Note: This is test shard 1 of 3.
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20251211 20:52:17.730787 416 test_util.cc:276] Using random seed: 1376004281
2025-12-11T20:52:17Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-12-11T20:52:17Z Disabled control of system clock
I20251211 20:52:17.764048 416 external_mini_cluster.cc:1366] Running /tmp/dist-test-task5TWsQc/build/asan/bin/kudu
/tmp/dist-test-task5TWsQc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.62:41123
--webserver_interface=127.0.104.62
--webserver_port=0
--builtin_ntp_servers=127.0.104.20:41333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.62:41123,127.0.104.61:40749,127.0.104.60:42087
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20251211 20:52:18.102699 424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:52:18.103068 424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:52:18.103197 424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:52:18.112217 424 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251211 20:52:18.112362 424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:52:18.112447 424 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251211 20:52:18.112514 424 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251211 20:52:18.123246 424 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.20:41333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.62:41123,127.0.104.61:40749,127.0.104.60:42087
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.62:41123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.104.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:52:18.125005 424 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:52:18.126963 424 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:52:18.137353 429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:52:18.136658 430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:52:18.138878 432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:52:18.139474 424 server_base.cc:1047] running on GCE node
I20251211 20:52:18.140359 424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:52:18.141891 424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:52:18.143126 424 hybrid_clock.cc:648] HybridClock initialized: now 1765486338143042 us; error 75 us; skew 500 ppm
I20251211 20:52:18.143558 424 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:52:18.146059 424 webserver.cc:492] Webserver started at http://127.0.104.62:37413/ using document root <none> and password file <none>
I20251211 20:52:18.146857 424 fs_manager.cc:362] Metadata directory not provided
I20251211 20:52:18.146965 424 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:52:18.147316 424 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251211 20:52:18.149806 424 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/data/instance:
uuid: "1830b0c58cfe495989f3d911dda0409a"
format_stamp: "Formatted at 2025-12-11 20:52:18 on dist-test-slave-fp9b"
I20251211 20:52:18.150645 424 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/wal/instance:
uuid: "1830b0c58cfe495989f3d911dda0409a"
format_stamp: "Formatted at 2025-12-11 20:52:18 on dist-test-slave-fp9b"
I20251211 20:52:18.156340 424 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20251211 20:52:18.160192 438 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:52:18.161620 424 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20251211 20:52:18.161798 424 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/data,/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/wal
uuid: "1830b0c58cfe495989f3d911dda0409a"
format_stamp: "Formatted at 2025-12-11 20:52:18 on dist-test-slave-fp9b"
I20251211 20:52:18.161996 424 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:52:18.194167 424 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:52:18.195297 424 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:52:18.195627 424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:52:18.216493 424 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.62:41123
I20251211 20:52:18.216527 489 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.62:41123 every 8 connection(s)
I20251211 20:52:18.217993 424 server_base.cc:1179] Dumped server information to /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/data/info.pb
I20251211 20:52:18.220919 416 external_mini_cluster.cc:1428] Started /tmp/dist-test-task5TWsQc/build/asan/bin/kudu as pid 424
I20251211 20:52:18.221179 416 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/wal/instance
I20251211 20:52:18.222611 490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:18.223737 416 external_mini_cluster.cc:1366] Running /tmp/dist-test-task5TWsQc/build/asan/bin/kudu
/tmp/dist-test-task5TWsQc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.61:40749
--webserver_interface=127.0.104.61
--webserver_port=0
--builtin_ntp_servers=127.0.104.20:41333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.62:41123,127.0.104.61:40749,127.0.104.60:42087
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20251211 20:52:18.229329 490 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } has no permanent_uuid. Determining permanent_uuid...
I20251211 20:52:18.254777 490 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } has no permanent_uuid. Determining permanent_uuid...
W20251211 20:52:18.255932 440 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.61:40749: connect: Connection refused (error 111)
W20251211 20:52:18.258016 490 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.61:40749: Network error: Client connection negotiation failed: client connection to 127.0.104.61:40749: connect: Connection refused (error 111)
I20251211 20:52:18.307410 490 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } attempt: 1
W20251211 20:52:18.309382 490 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.61:40749: Network error: Client connection negotiation failed: client connection to 127.0.104.61:40749: connect: Connection refused (error 111)
I20251211 20:52:18.377676 490 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } attempt: 2
W20251211 20:52:18.379477 490 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.61:40749: Network error: Client connection negotiation failed: client connection to 127.0.104.61:40749: connect: Connection refused (error 111)
I20251211 20:52:18.470790 490 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } attempt: 3
W20251211 20:52:18.472487 490 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.61:40749: Network error: Client connection negotiation failed: client connection to 127.0.104.61:40749: connect: Connection refused (error 111)
W20251211 20:52:18.583830 492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:52:18.584177 492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:52:18.584295 492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:52:18.593245 492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251211 20:52:18.593382 492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:52:18.593474 492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251211 20:52:18.593608 492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251211 20:52:18.604282 492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.20:41333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.62:41123,127.0.104.61:40749,127.0.104.60:42087
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.61:40749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.104.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:52:18.605944 492 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:52:18.607836 492 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251211 20:52:18.615741 490 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } attempt: 4
W20251211 20:52:18.617812 490 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.61:40749: Network error: Client connection negotiation failed: client connection to 127.0.104.61:40749: connect: Connection refused (error 111)
W20251211 20:52:18.618983 500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:52:18.619479 503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:52:18.619127 501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:52:18.620469 492 server_base.cc:1047] running on GCE node
I20251211 20:52:18.621376 492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:52:18.622931 492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:52:18.624147 492 hybrid_clock.cc:648] HybridClock initialized: now 1765486338624094 us; error 48 us; skew 500 ppm
I20251211 20:52:18.624523 492 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:52:18.626811 492 webserver.cc:492] Webserver started at http://127.0.104.61:42143/ using document root <none> and password file <none>
I20251211 20:52:18.627561 492 fs_manager.cc:362] Metadata directory not provided
I20251211 20:52:18.627696 492 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:52:18.628053 492 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251211 20:52:18.630514 492 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/data/instance:
uuid: "11f5f94732f049d0be4c97cebfffcb05"
format_stamp: "Formatted at 2025-12-11 20:52:18 on dist-test-slave-fp9b"
I20251211 20:52:18.631269 492 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/wal/instance:
uuid: "11f5f94732f049d0be4c97cebfffcb05"
format_stamp: "Formatted at 2025-12-11 20:52:18 on dist-test-slave-fp9b"
I20251211 20:52:18.637181 492 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.003s sys 0.001s
I20251211 20:52:18.640738 509 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:52:18.642186 492 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20251211 20:52:18.642369 492 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/data,/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/wal
uuid: "11f5f94732f049d0be4c97cebfffcb05"
format_stamp: "Formatted at 2025-12-11 20:52:18 on dist-test-slave-fp9b"
I20251211 20:52:18.642637 492 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:52:18.656181 492 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:52:18.657080 492 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:52:18.657351 492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:52:18.678546 492 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.61:40749
I20251211 20:52:18.678601 560 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.61:40749 every 8 connection(s)
I20251211 20:52:18.680068 492 server_base.cc:1179] Dumped server information to /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/data/info.pb
I20251211 20:52:18.685094 561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:18.690232 416 external_mini_cluster.cc:1428] Started /tmp/dist-test-task5TWsQc/build/asan/bin/kudu as pid 492
I20251211 20:52:18.690426 416 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/wal/instance
I20251211 20:52:18.691807 416 external_mini_cluster.cc:1366] Running /tmp/dist-test-task5TWsQc/build/asan/bin/kudu
/tmp/dist-test-task5TWsQc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.60:42087
--webserver_interface=127.0.104.60
--webserver_port=0
--builtin_ntp_servers=127.0.104.20:41333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.62:41123,127.0.104.61:40749,127.0.104.60:42087
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20251211 20:52:18.691310 561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } has no permanent_uuid. Determining permanent_uuid...
I20251211 20:52:18.708992 561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } has no permanent_uuid. Determining permanent_uuid...
I20251211 20:52:18.722172 561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } has no permanent_uuid. Determining permanent_uuid...
W20251211 20:52:18.724316 512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.60:42087: connect: Connection refused (error 111)
W20251211 20:52:18.726078 561 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.60:42087: Network error: Client connection negotiation failed: client connection to 127.0.104.60:42087: connect: Connection refused (error 111)
I20251211 20:52:18.775393 561 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } attempt: 1
W20251211 20:52:18.777482 561 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.60:42087: Network error: Client connection negotiation failed: client connection to 127.0.104.60:42087: connect: Connection refused (error 111)
I20251211 20:52:18.845744 561 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } attempt: 2
W20251211 20:52:18.847569 561 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.60:42087: Network error: Client connection negotiation failed: client connection to 127.0.104.60:42087: connect: Connection refused (error 111)
I20251211 20:52:18.917122 490 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } attempt: 5
I20251211 20:52:18.923985 490 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } has no permanent_uuid. Determining permanent_uuid...
W20251211 20:52:18.926008 490 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.60:42087: Network error: Client connection negotiation failed: client connection to 127.0.104.60:42087: connect: Connection refused (error 111)
I20251211 20:52:18.938911 561 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } attempt: 3
W20251211 20:52:18.940959 561 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.60:42087: Network error: Client connection negotiation failed: client connection to 127.0.104.60:42087: connect: Connection refused (error 111)
I20251211 20:52:18.977254 490 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } attempt: 1
W20251211 20:52:18.979238 490 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.60:42087: Network error: Client connection negotiation failed: client connection to 127.0.104.60:42087: connect: Connection refused (error 111)
I20251211 20:52:19.047544 490 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } attempt: 2
W20251211 20:52:19.049466 490 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.60:42087: Network error: Client connection negotiation failed: client connection to 127.0.104.60:42087: connect: Connection refused (error 111)
W20251211 20:52:19.050765 564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:52:19.051069 564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:52:19.051177 564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:52:19.059499 564 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251211 20:52:19.059650 564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:52:19.059731 564 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251211 20:52:19.059790 564 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251211 20:52:19.070238 564 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.20:41333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.62:41123,127.0.104.61:40749,127.0.104.60:42087
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.60:42087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.104.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:52:19.071939 564 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:52:19.073760 564 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:52:19.082508 572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:52:19.082612 571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:52:19.082787 574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:52:19.083698 564 server_base.cc:1047] running on GCE node
I20251211 20:52:19.084214 561 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } attempt: 4
I20251211 20:52:19.084565 564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20251211 20:52:19.085870 561 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.60:42087: Network error: Client connection negotiation failed: client connection to 127.0.104.60:42087: connect: Connection refused (error 111)
I20251211 20:52:19.086201 564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:52:19.087464 564 hybrid_clock.cc:648] HybridClock initialized: now 1765486339087382 us; error 68 us; skew 500 ppm
I20251211 20:52:19.087888 564 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:52:19.090287 564 webserver.cc:492] Webserver started at http://127.0.104.60:40729/ using document root <none> and password file <none>
I20251211 20:52:19.091094 564 fs_manager.cc:362] Metadata directory not provided
I20251211 20:52:19.091187 564 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:52:19.091552 564 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251211 20:52:19.094031 564 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/data/instance:
uuid: "9230f94a6a1a49cead51aeea9f069087"
format_stamp: "Formatted at 2025-12-11 20:52:19 on dist-test-slave-fp9b"
I20251211 20:52:19.094913 564 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/wal/instance:
uuid: "9230f94a6a1a49cead51aeea9f069087"
format_stamp: "Formatted at 2025-12-11 20:52:19 on dist-test-slave-fp9b"
I20251211 20:52:19.103025 564 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.005s sys 0.000s
I20251211 20:52:19.106614 580 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:52:19.108139 564 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20251211 20:52:19.108312 564 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/data,/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/wal
uuid: "9230f94a6a1a49cead51aeea9f069087"
format_stamp: "Formatted at 2025-12-11 20:52:19 on dist-test-slave-fp9b"
I20251211 20:52:19.108533 564 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:52:19.129624 564 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:52:19.130510 564 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:52:19.130762 564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:52:19.152482 564 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.60:42087
I20251211 20:52:19.152499 631 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.60:42087 every 8 connection(s)
I20251211 20:52:19.153951 564 server_base.cc:1179] Dumped server information to /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/data/info.pb
I20251211 20:52:19.155669 490 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } attempt: 3
I20251211 20:52:19.159029 632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:19.162516 416 external_mini_cluster.cc:1428] Started /tmp/dist-test-task5TWsQc/build/asan/bin/kudu as pid 564
I20251211 20:52:19.162714 416 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/wal/instance
I20251211 20:52:19.167079 632 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } has no permanent_uuid. Determining permanent_uuid...
I20251211 20:52:19.177167 490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a: Bootstrap starting.
I20251211 20:52:19.181679 490 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:19.183128 490 log.cc:826] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a: Log is configured to *not* fsync() on all Append() calls
I20251211 20:52:19.183773 632 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } has no permanent_uuid. Determining permanent_uuid...
I20251211 20:52:19.186810 490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a: No bootstrap required, opened a new log
I20251211 20:52:19.195837 490 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:19.196403 490 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:19.196578 490 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1830b0c58cfe495989f3d911dda0409a, State: Initialized, Role: FOLLOWER
I20251211 20:52:19.197532 490 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:19.202062 632 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } has no permanent_uuid. Determining permanent_uuid...
I20251211 20:52:19.204988 640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } } }
I20251211 20:52:19.205293 640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [sys.catalog]: This master's current role is: FOLLOWER
I20251211 20:52:19.206986 490 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [sys.catalog]: configured and running, proceeding with master startup.
I20251211 20:52:19.214264 632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Bootstrap starting.
I20251211 20:52:19.218473 632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:19.219959 632 log.cc:826] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Log is configured to *not* fsync() on all Append() calls
I20251211 20:52:19.218917 424 master_runner.cc:186] Error getting master registration for 127.0.104.61:40749: OK, instance_id { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" instance_seqno: 1765486338672958 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20251211 20:52:19.224946 424 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20251211 20:52:19.228384 632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: No bootstrap required, opened a new log
W20251211 20:52:19.234115 651 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251211 20:52:19.234404 651 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251211 20:52:19.236063 632 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:19.236474 632 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:19.236596 632 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9230f94a6a1a49cead51aeea9f069087, State: Initialized, Role: FOLLOWER
I20251211 20:52:19.237366 632 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:19.238437 654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } } }
I20251211 20:52:19.238761 654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [sys.catalog]: This master's current role is: FOLLOWER
I20251211 20:52:19.239882 632 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [sys.catalog]: configured and running, proceeding with master startup.
W20251211 20:52:19.256032 665 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251211 20:52:19.256183 665 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251211 20:52:19.256685 564 master_runner.cc:186] Error getting master registration for 127.0.104.61:40749: OK, instance_id { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" instance_seqno: 1765486338672958 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20251211 20:52:19.263132 564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20251211 20:52:19.346417 654 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:19.346771 654 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:19.348856 654 leader_election.cc:290] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1830b0c58cfe495989f3d911dda0409a (127.0.104.62:41123), 11f5f94732f049d0be4c97cebfffcb05 (127.0.104.61:40749)
I20251211 20:52:19.350169 465 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9230f94a6a1a49cead51aeea9f069087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1830b0c58cfe495989f3d911dda0409a" is_pre_election: true
I20251211 20:52:19.350751 465 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9230f94a6a1a49cead51aeea9f069087 in term 0.
I20251211 20:52:19.351464 584 leader_election.cc:304] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1830b0c58cfe495989f3d911dda0409a, 9230f94a6a1a49cead51aeea9f069087; no voters:
I20251211 20:52:19.352063 654 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:19.352222 654 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:19.352320 654 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:19.355674 654 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:19.357142 465 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9230f94a6a1a49cead51aeea9f069087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1830b0c58cfe495989f3d911dda0409a"
I20251211 20:52:19.357172 654 leader_election.cc:290] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [CANDIDATE]: Term 1 election: Requested vote from peers 1830b0c58cfe495989f3d911dda0409a (127.0.104.62:41123), 11f5f94732f049d0be4c97cebfffcb05 (127.0.104.61:40749)
I20251211 20:52:19.357372 465 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:19.361076 465 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9230f94a6a1a49cead51aeea9f069087 in term 1.
I20251211 20:52:19.361729 584 leader_election.cc:304] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1830b0c58cfe495989f3d911dda0409a, 9230f94a6a1a49cead51aeea9f069087; no voters:
I20251211 20:52:19.362537 654 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:19.364390 654 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [term 1 LEADER]: Becoming Leader. State: Replica: 9230f94a6a1a49cead51aeea9f069087, State: Running, Role: LEADER
I20251211 20:52:19.365068 654 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:19.370576 667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9230f94a6a1a49cead51aeea9f069087. Latest consensus state: current_term: 1 leader_uuid: "9230f94a6a1a49cead51aeea9f069087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } } }
I20251211 20:52:19.370955 667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [sys.catalog]: This master's current role is: LEADER
I20251211 20:52:19.372228 669 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251211 20:52:19.374622 669 catalog_manager.cc:1494] Initializing Kudu cluster ID...
W20251211 20:52:19.382925 586 tablet.cc:2378] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251211 20:52:19.385243 561 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } attempt: 5
I20251211 20:52:19.387504 465 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 1 FOLLOWER]: Refusing update from remote peer 9230f94a6a1a49cead51aeea9f069087: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251211 20:52:19.388834 667 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:19.401800 640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9230f94a6a1a49cead51aeea9f069087. Latest consensus state: current_term: 1 leader_uuid: "9230f94a6a1a49cead51aeea9f069087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } } }
I20251211 20:52:19.402078 640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [sys.catalog]: This master's current role is: FOLLOWER
I20251211 20:52:19.402549 561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Bootstrap starting.
I20251211 20:52:19.406883 561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:19.410619 561 log.cc:826] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Log is configured to *not* fsync() on all Append() calls
I20251211 20:52:19.410928 670 mvcc.cc:204] Tried to move back new op lower bound from 7231432046114066432 to 7231432046045335552. Current Snapshot: MvccSnapshot[applied={T|T < 7231432046114066432}]
I20251211 20:52:19.412290 671 mvcc.cc:204] Tried to move back new op lower bound from 7231432046114066432 to 7231432046045335552. Current Snapshot: MvccSnapshot[applied={T|T < 7231432046114066432}]
I20251211 20:52:19.412873 667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9230f94a6a1a49cead51aeea9f069087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } } }
I20251211 20:52:19.413100 667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [sys.catalog]: This master's current role is: LEADER
I20251211 20:52:19.415383 669 catalog_manager.cc:1357] Generated new cluster ID: 521ed6ceec724e0ab1d4cde545c10f31
I20251211 20:52:19.415506 669 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251211 20:52:19.421216 640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9230f94a6a1a49cead51aeea9f069087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } } }
I20251211 20:52:19.421459 640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [sys.catalog]: This master's current role is: FOLLOWER
I20251211 20:52:19.431917 561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: No bootstrap required, opened a new log
I20251211 20:52:19.447893 561 raft_consensus.cc:359] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:19.448369 561 raft_consensus.cc:385] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:19.448536 561 raft_consensus.cc:740] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f5f94732f049d0be4c97cebfffcb05, State: Initialized, Role: FOLLOWER
I20251211 20:52:19.449353 561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:19.464520 676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } } }
I20251211 20:52:19.467986 676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [sys.catalog]: This master's current role is: FOLLOWER
I20251211 20:52:19.464577 561 sys_catalog.cc:565] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [sys.catalog]: configured and running, proceeding with master startup.
I20251211 20:52:19.464881 534 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:19.464630 535 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9230f94a6a1a49cead51aeea9f069087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f5f94732f049d0be4c97cebfffcb05"
I20251211 20:52:19.464581 536 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9230f94a6a1a49cead51aeea9f069087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f5f94732f049d0be4c97cebfffcb05" is_pre_election: true
I20251211 20:52:19.480453 534 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 1 FOLLOWER]: Refusing update from remote peer 9230f94a6a1a49cead51aeea9f069087: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251211 20:52:19.481503 654 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:19.503708 669 catalog_manager.cc:1380] Generated new certificate authority record
I20251211 20:52:19.505504 669 catalog_manager.cc:1514] Loading token signing keys...
I20251211 20:52:19.514925 667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9230f94a6a1a49cead51aeea9f069087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } } }
I20251211 20:52:19.515185 667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [sys.catalog]: This master's current role is: LEADER
I20251211 20:52:19.518934 676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9230f94a6a1a49cead51aeea9f069087. Latest consensus state: current_term: 1 leader_uuid: "9230f94a6a1a49cead51aeea9f069087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } } }
I20251211 20:52:19.519186 676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [sys.catalog]: This master's current role is: FOLLOWER
I20251211 20:52:19.519572 680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9230f94a6a1a49cead51aeea9f069087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } } }
I20251211 20:52:19.520519 680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [sys.catalog]: This master's current role is: FOLLOWER
I20251211 20:52:19.539539 669 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Generated new TSK 0
I20251211 20:52:19.540844 669 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251211 20:52:19.554783 416 external_mini_cluster.cc:1366] Running /tmp/dist-test-task5TWsQc/build/asan/bin/kudu
/tmp/dist-test-task5TWsQc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.104.1:0
--local_ip_for_outbound_sockets=127.0.104.1
--webserver_interface=127.0.104.1
--webserver_port=0
--tserver_master_addrs=127.0.104.62:41123,127.0.104.61:40749,127.0.104.60:42087
--builtin_ntp_servers=127.0.104.20:41333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20251211 20:52:19.566527 692 catalog_manager.cc:1269] Loaded cluster ID: 521ed6ceec724e0ab1d4cde545c10f31
I20251211 20:52:19.566637 692 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: loading cluster ID for follower catalog manager: success
I20251211 20:52:19.571162 692 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: acquiring CA information for follower catalog manager: success
I20251211 20:52:19.574286 692 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20251211 20:52:19.988835 694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:52:19.989178 694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:52:19.989315 694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:52:19.998068 694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:52:19.998261 694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.104.1
I20251211 20:52:20.008172 694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.20:41333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.104.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.104.1
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.104.62:41123,127.0.104.61:40749,127.0.104.60:42087
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.104.1
--log_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:52:20.009846 694 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:52:20.011729 694 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:52:20.021584 700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:52:20.021611 699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:52:20.022549 702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:52:20.023717 694 server_base.cc:1047] running on GCE node
I20251211 20:52:20.024338 694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:52:20.025309 694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:52:20.026530 694 hybrid_clock.cc:648] HybridClock initialized: now 1765486340026473 us; error 49 us; skew 500 ppm
I20251211 20:52:20.026839 694 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:52:20.029088 694 webserver.cc:492] Webserver started at http://127.0.104.1:41669/ using document root <none> and password file <none>
I20251211 20:52:20.029878 694 fs_manager.cc:362] Metadata directory not provided
I20251211 20:52:20.029980 694 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:52:20.030332 694 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251211 20:52:20.033126 694 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/data/instance:
uuid: "6f8d784fd1cd492bbe3809403ec34919"
format_stamp: "Formatted at 2025-12-11 20:52:20 on dist-test-slave-fp9b"
I20251211 20:52:20.034161 694 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/instance:
uuid: "6f8d784fd1cd492bbe3809403ec34919"
format_stamp: "Formatted at 2025-12-11 20:52:20 on dist-test-slave-fp9b"
I20251211 20:52:20.040040 694 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20251211 20:52:20.043962 708 raft_consensus.cc:493] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9230f94a6a1a49cead51aeea9f069087)
I20251211 20:52:20.044266 708 raft_consensus.cc:515] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:20.044391 709 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:52:20.045683 708 leader_election.cc:290] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1830b0c58cfe495989f3d911dda0409a (127.0.104.62:41123), 9230f94a6a1a49cead51aeea9f069087 (127.0.104.60:42087)
I20251211 20:52:20.046224 694 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20251211 20:52:20.046325 465 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11f5f94732f049d0be4c97cebfffcb05" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1830b0c58cfe495989f3d911dda0409a" is_pre_election: true
I20251211 20:52:20.046413 694 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/data,/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal
uuid: "6f8d784fd1cd492bbe3809403ec34919"
format_stamp: "Formatted at 2025-12-11 20:52:20 on dist-test-slave-fp9b"
I20251211 20:52:20.046590 607 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11f5f94732f049d0be4c97cebfffcb05" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9230f94a6a1a49cead51aeea9f069087" is_pre_election: true
I20251211 20:52:20.046633 694 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:52:20.047616 513 leader_election.cc:304] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f5f94732f049d0be4c97cebfffcb05; no voters: 1830b0c58cfe495989f3d911dda0409a, 9230f94a6a1a49cead51aeea9f069087
I20251211 20:52:20.047995 708 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:52:20.076449 694 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:52:20.077349 694 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:52:20.077615 694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:52:20.078588 694 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:52:20.080286 694 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251211 20:52:20.080392 694 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251211 20:52:20.080492 694 ts_tablet_manager.cc:616] Registered 0 tablets
I20251211 20:52:20.080562 694 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251211 20:52:20.114467 694 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.1:42177
I20251211 20:52:20.114543 821 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.1:42177 every 8 connection(s)
I20251211 20:52:20.115947 694 server_base.cc:1179] Dumped server information to /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/data/info.pb
I20251211 20:52:20.121739 416 external_mini_cluster.cc:1428] Started /tmp/dist-test-task5TWsQc/build/asan/bin/kudu as pid 694
I20251211 20:52:20.121959 416 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/instance
I20251211 20:52:20.128129 416 external_mini_cluster.cc:1366] Running /tmp/dist-test-task5TWsQc/build/asan/bin/kudu
/tmp/dist-test-task5TWsQc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.104.2:0
--local_ip_for_outbound_sockets=127.0.104.2
--webserver_interface=127.0.104.2
--webserver_port=0
--tserver_master_addrs=127.0.104.62:41123,127.0.104.61:40749,127.0.104.60:42087
--builtin_ntp_servers=127.0.104.20:41333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20251211 20:52:20.144603 822 heartbeater.cc:344] Connected to a master server at 127.0.104.62:41123
I20251211 20:52:20.145999 822 heartbeater.cc:461] Registering TS with master...
I20251211 20:52:20.146026 824 heartbeater.cc:344] Connected to a master server at 127.0.104.61:40749
I20251211 20:52:20.146222 824 heartbeater.cc:461] Registering TS with master...
I20251211 20:52:20.147293 824 heartbeater.cc:507] Master 127.0.104.61:40749 requested a full tablet report, sending...
I20251211 20:52:20.147557 822 heartbeater.cc:507] Master 127.0.104.62:41123 requested a full tablet report, sending...
I20251211 20:52:20.149830 526 ts_manager.cc:194] Registered new tserver with Master: 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:20.150084 455 ts_manager.cc:194] Registered new tserver with Master: 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:20.150583 823 heartbeater.cc:344] Connected to a master server at 127.0.104.60:42087
I20251211 20:52:20.150728 823 heartbeater.cc:461] Registering TS with master...
I20251211 20:52:20.151278 823 heartbeater.cc:507] Master 127.0.104.60:42087 requested a full tablet report, sending...
I20251211 20:52:20.153154 597 ts_manager.cc:194] Registered new tserver with Master: 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:20.155205 597 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.104.1:41823
I20251211 20:52:20.236342 651 catalog_manager.cc:1269] Loaded cluster ID: 521ed6ceec724e0ab1d4cde545c10f31
I20251211 20:52:20.236474 651 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a: loading cluster ID for follower catalog manager: success
I20251211 20:52:20.239504 651 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a: acquiring CA information for follower catalog manager: success
I20251211 20:52:20.241964 651 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20251211 20:52:20.480670 832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:52:20.480983 832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:52:20.481106 832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:52:20.489842 832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:52:20.490039 832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.104.2
I20251211 20:52:20.500042 832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.20:41333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.104.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.104.2
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.104.62:41123,127.0.104.61:40749,127.0.104.60:42087
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.104.2
--log_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:52:20.501714 832 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:52:20.503403 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:52:20.513064 838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:52:20.513032 837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:52:20.513520 840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:52:20.514278 832 server_base.cc:1047] running on GCE node
I20251211 20:52:20.514879 832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:52:20.515784 832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:52:20.516992 832 hybrid_clock.cc:648] HybridClock initialized: now 1765486340516937 us; error 48 us; skew 500 ppm
I20251211 20:52:20.517290 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:52:20.519615 832 webserver.cc:492] Webserver started at http://127.0.104.2:41559/ using document root <none> and password file <none>
I20251211 20:52:20.520360 832 fs_manager.cc:362] Metadata directory not provided
I20251211 20:52:20.520463 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:52:20.521283 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251211 20:52:20.523721 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/data/instance:
uuid: "eb0f1933d9a74fccaa4618047fc05c41"
format_stamp: "Formatted at 2025-12-11 20:52:20 on dist-test-slave-fp9b"
I20251211 20:52:20.524475 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/instance:
uuid: "eb0f1933d9a74fccaa4618047fc05c41"
format_stamp: "Formatted at 2025-12-11 20:52:20 on dist-test-slave-fp9b"
I20251211 20:52:20.529598 832 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20251211 20:52:20.533087 846 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:52:20.534430 832 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20251211 20:52:20.534600 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/data,/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal
uuid: "eb0f1933d9a74fccaa4618047fc05c41"
format_stamp: "Formatted at 2025-12-11 20:52:20 on dist-test-slave-fp9b"
I20251211 20:52:20.534827 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:52:20.549015 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:52:20.549881 832 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:52:20.550160 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:52:20.551770 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:52:20.553362 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251211 20:52:20.553455 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251211 20:52:20.553551 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251211 20:52:20.553615 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251211 20:52:20.587121 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.2:34691
I20251211 20:52:20.587164 958 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.2:34691 every 8 connection(s)
I20251211 20:52:20.588474 832 server_base.cc:1179] Dumped server information to /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/data/info.pb
I20251211 20:52:20.598824 416 external_mini_cluster.cc:1428] Started /tmp/dist-test-task5TWsQc/build/asan/bin/kudu as pid 832
I20251211 20:52:20.599057 416 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/instance
I20251211 20:52:20.604349 416 external_mini_cluster.cc:1366] Running /tmp/dist-test-task5TWsQc/build/asan/bin/kudu
/tmp/dist-test-task5TWsQc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.104.3:0
--local_ip_for_outbound_sockets=127.0.104.3
--webserver_interface=127.0.104.3
--webserver_port=0
--tserver_master_addrs=127.0.104.62:41123,127.0.104.61:40749,127.0.104.60:42087
--builtin_ntp_servers=127.0.104.20:41333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20251211 20:52:20.619485 959 heartbeater.cc:344] Connected to a master server at 127.0.104.62:41123
I20251211 20:52:20.619520 961 heartbeater.cc:344] Connected to a master server at 127.0.104.61:40749
I20251211 20:52:20.619647 960 heartbeater.cc:344] Connected to a master server at 127.0.104.60:42087
I20251211 20:52:20.619935 961 heartbeater.cc:461] Registering TS with master...
I20251211 20:52:20.619933 959 heartbeater.cc:461] Registering TS with master...
I20251211 20:52:20.619935 960 heartbeater.cc:461] Registering TS with master...
I20251211 20:52:20.621183 959 heartbeater.cc:507] Master 127.0.104.62:41123 requested a full tablet report, sending...
I20251211 20:52:20.621380 960 heartbeater.cc:507] Master 127.0.104.60:42087 requested a full tablet report, sending...
I20251211 20:52:20.621592 961 heartbeater.cc:507] Master 127.0.104.61:40749 requested a full tablet report, sending...
I20251211 20:52:20.622895 597 ts_manager.cc:194] Registered new tserver with Master: eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:20.623385 526 ts_manager.cc:194] Registered new tserver with Master: eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:20.624017 597 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.104.2:36041
I20251211 20:52:20.624046 455 ts_manager.cc:194] Registered new tserver with Master: eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:20.627842 967 raft_consensus.cc:493] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9230f94a6a1a49cead51aeea9f069087)
I20251211 20:52:20.628026 967 raft_consensus.cc:515] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:20.628697 967 leader_election.cc:290] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1830b0c58cfe495989f3d911dda0409a (127.0.104.62:41123), 9230f94a6a1a49cead51aeea9f069087 (127.0.104.60:42087)
I20251211 20:52:20.629138 465 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11f5f94732f049d0be4c97cebfffcb05" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1830b0c58cfe495989f3d911dda0409a" is_pre_election: true
I20251211 20:52:20.629616 607 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11f5f94732f049d0be4c97cebfffcb05" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9230f94a6a1a49cead51aeea9f069087" is_pre_election: true
I20251211 20:52:20.630089 512 leader_election.cc:304] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f5f94732f049d0be4c97cebfffcb05; no voters: 1830b0c58cfe495989f3d911dda0409a, 9230f94a6a1a49cead51aeea9f069087
I20251211 20:52:20.630373 967 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:52:20.658219 823 heartbeater.cc:499] Master 127.0.104.60:42087 was elected leader, sending a full tablet report...
W20251211 20:52:20.949069 966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:52:20.949369 966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:52:20.949502 966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:52:20.957906 966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:52:20.958105 966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.104.3
I20251211 20:52:20.968303 966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.20:41333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.104.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.104.3
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.104.62:41123,127.0.104.61:40749,127.0.104.60:42087
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.104.3
--log_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:52:20.969905 966 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:52:20.971740 966 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:52:20.980921 972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:52:20.981271 973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:52:20.981961 975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:52:20.982318 966 server_base.cc:1047] running on GCE node
I20251211 20:52:20.982916 966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:52:20.983881 966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:52:20.985085 966 hybrid_clock.cc:648] HybridClock initialized: now 1765486340985034 us; error 47 us; skew 500 ppm
I20251211 20:52:20.985384 966 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:52:20.987574 966 webserver.cc:492] Webserver started at http://127.0.104.3:38257/ using document root <none> and password file <none>
I20251211 20:52:20.988344 966 fs_manager.cc:362] Metadata directory not provided
I20251211 20:52:20.988459 966 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:52:20.988802 966 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251211 20:52:20.991106 966 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/data/instance:
uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
format_stamp: "Formatted at 2025-12-11 20:52:20 on dist-test-slave-fp9b"
I20251211 20:52:20.991904 966 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/instance:
uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
format_stamp: "Formatted at 2025-12-11 20:52:20 on dist-test-slave-fp9b"
I20251211 20:52:20.997107 966 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20251211 20:52:21.001194 981 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:52:21.002701 966 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20251211 20:52:21.002861 966 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/data,/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal
uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
format_stamp: "Formatted at 2025-12-11 20:52:20 on dist-test-slave-fp9b"
I20251211 20:52:21.003048 966 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:52:21.041271 966 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:52:21.042164 966 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:52:21.042443 966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:52:21.043434 966 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:52:21.045245 966 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251211 20:52:21.045336 966 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251211 20:52:21.045405 966 ts_tablet_manager.cc:616] Registered 0 tablets
I20251211 20:52:21.045469 966 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251211 20:52:21.080024 966 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.3:36845
I20251211 20:52:21.080098 1093 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.3:36845 every 8 connection(s)
I20251211 20:52:21.081357 966 server_base.cc:1179] Dumped server information to /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/data/info.pb
I20251211 20:52:21.083802 416 external_mini_cluster.cc:1428] Started /tmp/dist-test-task5TWsQc/build/asan/bin/kudu as pid 966
I20251211 20:52:21.083989 416 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/instance
I20251211 20:52:21.098783 1095 heartbeater.cc:344] Connected to a master server at 127.0.104.60:42087
I20251211 20:52:21.099285 1095 heartbeater.cc:461] Registering TS with master...
I20251211 20:52:21.100566 1095 heartbeater.cc:507] Master 127.0.104.60:42087 requested a full tablet report, sending...
I20251211 20:52:21.102475 597 ts_manager.cc:194] Registered new tserver with Master: e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:21.102859 1094 heartbeater.cc:344] Connected to a master server at 127.0.104.62:41123
I20251211 20:52:21.102994 1094 heartbeater.cc:461] Registering TS with master...
I20251211 20:52:21.103423 1094 heartbeater.cc:507] Master 127.0.104.62:41123 requested a full tablet report, sending...
I20251211 20:52:21.103440 597 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.104.3:41803
I20251211 20:52:21.104498 1096 heartbeater.cc:344] Connected to a master server at 127.0.104.61:40749
I20251211 20:52:21.104617 1096 heartbeater.cc:461] Registering TS with master...
I20251211 20:52:21.104772 455 ts_manager.cc:194] Registered new tserver with Master: e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:21.105782 1096 heartbeater.cc:507] Master 127.0.104.61:40749 requested a full tablet report, sending...
I20251211 20:52:21.106868 526 ts_manager.cc:194] Registered new tserver with Master: e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:21.109925 416 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20251211 20:52:21.126466 960 heartbeater.cc:499] Master 127.0.104.60:42087 was elected leader, sending a full tablet report...
I20251211 20:52:21.147986 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_912e1e2f9656404b8b3a01a6e5ebd466"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:21.148239 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_975785c8697c4a72af52d97904eb884a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:21.148795 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_0bd6bd2f476a4cf78c2916708fc414e6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:21.149431 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_f4db346503934dba9aa7535bc522e594"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:21.149857 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_912e1e2f9656404b8b3a01a6e5ebd466 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:21.150070 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f4db346503934dba9aa7535bc522e594 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.151084 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_0cd35b930b9441fcaacfaf5cb01025eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:21.151706 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0cd35b930b9441fcaacfaf5cb01025eb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.152767 416 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-task5TWsQc/build/asan/bin/kudu pid 424
I20251211 20:52:21.153216 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_f7f3a5440e6441a39c27987ee2c6f84e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:21.153846 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f7f3a5440e6441a39c27987ee2c6f84e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.160840 485 generic_service.cc:196] Checking for leaks (request via RPC)
W20251211 20:52:21.161649 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_975785c8697c4a72af52d97904eb884a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.170006 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_4ab591f2d9c54a3c93d2d0514e417bcc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:21.170866 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4ab591f2d9c54a3c93d2d0514e417bcc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.171887 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_5475d18bbfc54f03a6d62babaf2d5ae6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:21.172780 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5475d18bbfc54f03a6d62babaf2d5ae6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:21.174558 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0bd6bd2f476a4cf78c2916708fc414e6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.174407 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_bb4d73eec58a4034aa65d53e54beb39c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:21.176277 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bb4d73eec58a4034aa65d53e54beb39c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.175520 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_814fe47fa1fd41f2a99a6a6ab72ce837"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:21.178025 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_814fe47fa1fd41f2a99a6a6ab72ce837 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.216346 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_912e1e2f9656404b8b3a01a6e5ebd466" } new_table_name: "default.Table_9a500be9b1c94dbb99529a71b1f5b56d" modify_external_catalogs: true
I20251211 20:52:21.236105 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f7f3a5440e6441a39c27987ee2c6f84e" } new_table_name: "default.Table_54365c25842c4ec391e069b55177da23" modify_external_catalogs: true
I20251211 20:52:21.237360 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_975785c8697c4a72af52d97904eb884a" } new_table_name: "default.Table_b37d05b3e8eb4425bb5cb72d46ce9e91" modify_external_catalogs: true
I20251211 20:52:21.238144 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_5475d18bbfc54f03a6d62babaf2d5ae6" } new_table_name: "default.Table_ce8eb57ae5624218b6cac53848e34194" modify_external_catalogs: true
I20251211 20:52:21.238440 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4ab591f2d9c54a3c93d2d0514e417bcc" } modify_external_catalogs: true
I20251211 20:52:21.238623 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0cd35b930b9441fcaacfaf5cb01025eb" } new_table_name: "default.Table_ebc736f835124930b0060d37924294af" modify_external_catalogs: true
I20251211 20:52:21.238662 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4ab591f2d9c54a3c93d2d0514e417bcc" } modify_external_catalogs: true
I20251211 20:52:21.239351 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_814fe47fa1fd41f2a99a6a6ab72ce837" } modify_external_catalogs: true
I20251211 20:52:21.239521 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_814fe47fa1fd41f2a99a6a6ab72ce837" } modify_external_catalogs: true
I20251211 20:52:21.242067 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f4db346503934dba9aa7535bc522e594" } modify_external_catalogs: true
I20251211 20:52:21.242281 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f4db346503934dba9aa7535bc522e594" } modify_external_catalogs: true
I20251211 20:52:21.245466 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0bd6bd2f476a4cf78c2916708fc414e6" } modify_external_catalogs: true
I20251211 20:52:21.245684 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0bd6bd2f476a4cf78c2916708fc414e6" } modify_external_catalogs: true
I20251211 20:52:21.246893 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_bb4d73eec58a4034aa65d53e54beb39c" } modify_external_catalogs: true
I20251211 20:52:21.247058 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_bb4d73eec58a4034aa65d53e54beb39c" } modify_external_catalogs: true
W20251211 20:52:21.257145 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d42e5cce248342f59d67f7c9aa6f66a2 consensus state has no leader
I20251211 20:52:21.269317 590 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 5d9d79cd0b934432838ecc588af9c6c8
W20251211 20:52:21.269402 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 00d68e696250404d880b8f97af217cbd has no consensus state
W20251211 20:52:21.270372 592 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f5467114696046fbb2f311856b7a4d73 has no consensus state
W20251211 20:52:21.270954 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8140bfd13c50450f9ecb677de47b635e has no consensus state
I20251211 20:52:21.271260 591 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 9773d5b6a19240fc943b56cc8e12b521
W20251211 20:52:21.271525 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 03ebcee3069f44f99761cc13e34512ed has no consensus state
I20251211 20:52:21.271878 588 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 41197d2f1e5547159b70445fd4995baf
I20251211 20:52:21.275849 589 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet f166b88d9f0f45dc9205ee4199129f67
I20251211 20:52:21.276204 595 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet e86f2f4af0be415d93b65df14122271f
I20251211 20:52:21.281188 894 tablet_service.cc:1505] Processing CreateTablet for tablet d42e5cce248342f59d67f7c9aa6f66a2 (DEFAULT_TABLE table=default.Table_912e1e2f9656404b8b3a01a6e5ebd466 [id=1473b367a32b480f842106bf0f4e41fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.283138 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d42e5cce248342f59d67f7c9aa6f66a2. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:21.288964 582 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet d42e5cce248342f59d67f7c9aa6f66a2 (table default.Table_9a500be9b1c94dbb99529a71b1f5b56d [id=1473b367a32b480f842106bf0f4e41fb], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet d42e5cce248342f59d67f7c9aa6f66a2 consensus state has no leader
I20251211 20:52:21.291214 893 tablet_service.cc:1505] Processing CreateTablet for tablet 03ebcee3069f44f99761cc13e34512ed (DEFAULT_TABLE table=default.Table_ebc736f835124930b0060d37924294af [id=7b11b994218b4802a07f78be9ed38877]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.291826 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03ebcee3069f44f99761cc13e34512ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.293812 892 tablet_service.cc:1505] Processing CreateTablet for tablet 00d68e696250404d880b8f97af217cbd (DEFAULT_TABLE table=default.Table_54365c25842c4ec391e069b55177da23 [id=ebae22b7d617403c84c023d0628f8ad8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.294389 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00d68e696250404d880b8f97af217cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.295250 891 tablet_service.cc:1505] Processing CreateTablet for tablet f5467114696046fbb2f311856b7a4d73 (DEFAULT_TABLE table=default.Table_b37d05b3e8eb4425bb5cb72d46ce9e91 [id=4e18dbcc527d4b12ba6978ca125d5085]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.295756 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5467114696046fbb2f311856b7a4d73. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.297331 890 tablet_service.cc:1505] Processing CreateTablet for tablet 8140bfd13c50450f9ecb677de47b635e (DEFAULT_TABLE table=default.Table_ce8eb57ae5624218b6cac53848e34194 [id=b8c401dfdb5f4f7584dca0989287ce31]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.297981 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8140bfd13c50450f9ecb677de47b635e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.307480 1160 tablet_bootstrap.cc:492] T f5467114696046fbb2f311856b7a4d73 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:21.311424 1160 tablet_bootstrap.cc:654] T f5467114696046fbb2f311856b7a4d73 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:21.313616 1160 log.cc:826] T f5467114696046fbb2f311856b7a4d73 P eb0f1933d9a74fccaa4618047fc05c41: Log is configured to *not* fsync() on all Append() calls
I20251211 20:52:21.324810 1026 tablet_service.cc:1505] Processing CreateTablet for tablet f5467114696046fbb2f311856b7a4d73 (DEFAULT_TABLE table=default.Table_b37d05b3e8eb4425bb5cb72d46ce9e91 [id=4e18dbcc527d4b12ba6978ca125d5085]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.326196 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5467114696046fbb2f311856b7a4d73. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.325825 756 tablet_service.cc:1505] Processing CreateTablet for tablet 03ebcee3069f44f99761cc13e34512ed (DEFAULT_TABLE table=default.Table_ebc736f835124930b0060d37924294af [id=7b11b994218b4802a07f78be9ed38877]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.327723 757 tablet_service.cc:1505] Processing CreateTablet for tablet d42e5cce248342f59d67f7c9aa6f66a2 (DEFAULT_TABLE table=default.Table_912e1e2f9656404b8b3a01a6e5ebd466 [id=1473b367a32b480f842106bf0f4e41fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.328112 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 8140bfd13c50450f9ecb677de47b635e (DEFAULT_TABLE table=default.Table_ce8eb57ae5624218b6cac53848e34194 [id=b8c401dfdb5f4f7584dca0989287ce31]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.328461 755 tablet_service.cc:1505] Processing CreateTablet for tablet 00d68e696250404d880b8f97af217cbd (DEFAULT_TABLE table=default.Table_54365c25842c4ec391e069b55177da23 [id=ebae22b7d617403c84c023d0628f8ad8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.328688 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8140bfd13c50450f9ecb677de47b635e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.329161 753 tablet_service.cc:1505] Processing CreateTablet for tablet 8140bfd13c50450f9ecb677de47b635e (DEFAULT_TABLE table=default.Table_ce8eb57ae5624218b6cac53848e34194 [id=b8c401dfdb5f4f7584dca0989287ce31]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.331991 754 tablet_service.cc:1505] Processing CreateTablet for tablet f5467114696046fbb2f311856b7a4d73 (DEFAULT_TABLE table=default.Table_b37d05b3e8eb4425bb5cb72d46ce9e91 [id=4e18dbcc527d4b12ba6978ca125d5085]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.332254 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 00d68e696250404d880b8f97af217cbd (DEFAULT_TABLE table=default.Table_54365c25842c4ec391e069b55177da23 [id=ebae22b7d617403c84c023d0628f8ad8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.332768 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00d68e696250404d880b8f97af217cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.335459 596 master_service.cc:954] ReplaceTablet: received request to replace tablet f5467114696046fbb2f311856b7a4d73 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:21.336022 1029 tablet_service.cc:1505] Processing CreateTablet for tablet d42e5cce248342f59d67f7c9aa6f66a2 (DEFAULT_TABLE table=default.Table_912e1e2f9656404b8b3a01a6e5ebd466 [id=1473b367a32b480f842106bf0f4e41fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.336498 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d42e5cce248342f59d67f7c9aa6f66a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.336982 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 8140bfd13c50450f9ecb677de47b635e from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:21.338009 1028 tablet_service.cc:1505] Processing CreateTablet for tablet 03ebcee3069f44f99761cc13e34512ed (DEFAULT_TABLE table=default.Table_ebc736f835124930b0060d37924294af [id=7b11b994218b4802a07f78be9ed38877]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.338486 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03ebcee3069f44f99761cc13e34512ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.340368 589 master_service.cc:954] ReplaceTablet: received request to replace tablet f5467114696046fbb2f311856b7a4d73 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:21.340678 588 master_service.cc:954] ReplaceTablet: received request to replace tablet 8140bfd13c50450f9ecb677de47b635e from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:21.340945 593 master_service.cc:954] ReplaceTablet: received request to replace tablet d42e5cce248342f59d67f7c9aa6f66a2 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:21.341806 1160 tablet_bootstrap.cc:492] T f5467114696046fbb2f311856b7a4d73 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:21.342101 1160 ts_tablet_manager.cc:1403] T f5467114696046fbb2f311856b7a4d73 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.035s user 0.006s sys 0.006s
I20251211 20:52:21.348727 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet d42e5cce248342f59d67f7c9aa6f66a2
I20251211 20:52:21.349071 1160 raft_consensus.cc:359] T f5467114696046fbb2f311856b7a4d73 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.349500 1160 raft_consensus.cc:385] T f5467114696046fbb2f311856b7a4d73 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:21.349572 593 catalog_manager.cc:6724] ReplaceTablet: tablet d42e5cce248342f59d67f7c9aa6f66a2 deleted and replaced by tablet 997fd184e900424bb5eabfde8c676d0e
I20251211 20:52:21.349611 1160 raft_consensus.cc:740] T f5467114696046fbb2f311856b7a4d73 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:21.350174 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet d42e5cce248342f59d67f7c9aa6f66a2 with delete_type TABLET_DATA_DELETED (replaced by tablet 997fd184e900424bb5eabfde8c676d0e) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:21.350411 1160 consensus_queue.cc:260] T f5467114696046fbb2f311856b7a4d73 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.350677 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 8140bfd13c50450f9ecb677de47b635e
I20251211 20:52:21.351089 595 catalog_manager.cc:6724] ReplaceTablet: tablet 8140bfd13c50450f9ecb677de47b635e deleted and replaced by tablet b67de345cf44441abe9e1431b64c4ba4
I20251211 20:52:21.352981 891 tablet_service.cc:1552] Processing DeleteTablet for tablet d42e5cce248342f59d67f7c9aa6f66a2 with delete_type TABLET_DATA_DELETED (replaced by tablet 997fd184e900424bb5eabfde8c676d0e) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:21.353240 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet 8140bfd13c50450f9ecb677de47b635e with delete_type TABLET_DATA_DELETED (replaced by tablet b67de345cf44441abe9e1431b64c4ba4) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:21.353600 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 8140bfd13c50450f9ecb677de47b635e with delete_type TABLET_DATA_DELETED (replaced by tablet b67de345cf44441abe9e1431b64c4ba4) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:21.353973 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet f5467114696046fbb2f311856b7a4d73
I20251211 20:52:21.354444 596 catalog_manager.cc:6724] ReplaceTablet: tablet f5467114696046fbb2f311856b7a4d73 deleted and replaced by tablet 12313c06b33749589ceb08ade798fc59
I20251211 20:52:21.357086 893 tablet_service.cc:1552] Processing DeleteTablet for tablet f5467114696046fbb2f311856b7a4d73 with delete_type TABLET_DATA_DELETED (replaced by tablet 12313c06b33749589ceb08ade798fc59) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:21.356215 752 tablet_service.cc:1552] Processing DeleteTablet for tablet d42e5cce248342f59d67f7c9aa6f66a2 with delete_type TABLET_DATA_DELETED (replaced by tablet 997fd184e900424bb5eabfde8c676d0e) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:21.359570 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet f5467114696046fbb2f311856b7a4d73 with delete_type TABLET_DATA_DELETED (replaced by tablet 12313c06b33749589ceb08ade798fc59) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:21.363909 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03ebcee3069f44f99761cc13e34512ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.364676 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5467114696046fbb2f311856b7a4d73. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.365934 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00d68e696250404d880b8f97af217cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.367811 893 tablet_service.cc:1505] Processing CreateTablet for tablet 997fd184e900424bb5eabfde8c676d0e (DEFAULT_TABLE table=default.Table_9a500be9b1c94dbb99529a71b1f5b56d [id=1473b367a32b480f842106bf0f4e41fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.368268 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 997fd184e900424bb5eabfde8c676d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.371773 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8140bfd13c50450f9ecb677de47b635e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.373261 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 8140bfd13c50450f9ecb677de47b635e with delete_type TABLET_DATA_DELETED (replaced by tablet b67de345cf44441abe9e1431b64c4ba4) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:21.373422 750 tablet_service.cc:1552] Processing DeleteTablet for tablet f5467114696046fbb2f311856b7a4d73 with delete_type TABLET_DATA_DELETED (replaced by tablet 12313c06b33749589ceb08ade798fc59) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:21.374008 749 tablet_service.cc:1505] Processing CreateTablet for tablet 997fd184e900424bb5eabfde8c676d0e (DEFAULT_TABLE table=default.Table_9a500be9b1c94dbb99529a71b1f5b56d [id=1473b367a32b480f842106bf0f4e41fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.374619 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 997fd184e900424bb5eabfde8c676d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.375774 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d42e5cce248342f59d67f7c9aa6f66a2. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:21.379792 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet d42e5cce248342f59d67f7c9aa6f66a2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d42e5cce248342f59d67f7c9aa6f66a2 already in progress: creating tablet
I20251211 20:52:21.380658 894 tablet_service.cc:1505] Processing CreateTablet for tablet 12313c06b33749589ceb08ade798fc59 (DEFAULT_TABLE table=default.Table_b37d05b3e8eb4425bb5cb72d46ce9e91 [id=4e18dbcc527d4b12ba6978ca125d5085]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.381054 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12313c06b33749589ceb08ade798fc59. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:21.388149 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet d42e5cce248342f59d67f7c9aa6f66a2 because the tablet was not found. No further retry: Not found: Tablet not found: d42e5cce248342f59d67f7c9aa6f66a2
I20251211 20:52:21.388222 1024 tablet_service.cc:1505] Processing CreateTablet for tablet 997fd184e900424bb5eabfde8c676d0e (DEFAULT_TABLE table=default.Table_9a500be9b1c94dbb99529a71b1f5b56d [id=1473b367a32b480f842106bf0f4e41fb]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:21.388630 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 8140bfd13c50450f9ecb677de47b635e because the tablet was not found. No further retry: Not found: Tablet not found: 8140bfd13c50450f9ecb677de47b635e
I20251211 20:52:21.388723 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 997fd184e900424bb5eabfde8c676d0e. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:21.388960 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet f5467114696046fbb2f311856b7a4d73 because the tablet was not found. No further retry: Not found: Tablet not found: f5467114696046fbb2f311856b7a4d73
I20251211 20:52:21.389883 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 12313c06b33749589ceb08ade798fc59 (DEFAULT_TABLE table=default.Table_b37d05b3e8eb4425bb5cb72d46ce9e91 [id=4e18dbcc527d4b12ba6978ca125d5085]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.390435 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12313c06b33749589ceb08ade798fc59. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.397158 893 tablet_service.cc:1505] Processing CreateTablet for tablet b67de345cf44441abe9e1431b64c4ba4 (DEFAULT_TABLE table=default.Table_ce8eb57ae5624218b6cac53848e34194 [id=b8c401dfdb5f4f7584dca0989287ce31]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.397615 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b67de345cf44441abe9e1431b64c4ba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.398319 1160 ts_tablet_manager.cc:1434] T f5467114696046fbb2f311856b7a4d73 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.056s user 0.009s sys 0.009s
I20251211 20:52:21.399392 1160 tablet_bootstrap.cc:492] T 00d68e696250404d880b8f97af217cbd P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:21.401219 1160 tablet_bootstrap.cc:654] T 00d68e696250404d880b8f97af217cbd P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:21.412103 416 external_mini_cluster.cc:1658] Killing /tmp/dist-test-task5TWsQc/build/asan/bin/kudu with pid 424
I20251211 20:52:21.415459 1168 tablet_bootstrap.cc:492] T 8140bfd13c50450f9ecb677de47b635e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:21.416786 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b142a71a5da94b88925a2e142715ff4d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:21.417680 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b142a71a5da94b88925a2e142715ff4d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.422111 1022 tablet_service.cc:1505] Processing CreateTablet for tablet b67de345cf44441abe9e1431b64c4ba4 (DEFAULT_TABLE table=default.Table_ce8eb57ae5624218b6cac53848e34194 [id=b8c401dfdb5f4f7584dca0989287ce31]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.422865 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b67de345cf44441abe9e1431b64c4ba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.425127 1168 tablet_bootstrap.cc:654] T 8140bfd13c50450f9ecb677de47b635e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:21.426692 1168 log.cc:826] T 8140bfd13c50450f9ecb677de47b635e P 6f8d784fd1cd492bbe3809403ec34919: Log is configured to *not* fsync() on all Append() calls
I20251211 20:52:21.428601 1160 tablet_bootstrap.cc:492] T 00d68e696250404d880b8f97af217cbd P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:21.428817 1160 ts_tablet_manager.cc:1403] T 00d68e696250404d880b8f97af217cbd P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.030s user 0.010s sys 0.001s
I20251211 20:52:21.429874 1160 raft_consensus.cc:359] T 00d68e696250404d880b8f97af217cbd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.430125 1160 raft_consensus.cc:385] T 00d68e696250404d880b8f97af217cbd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:21.430254 1160 raft_consensus.cc:740] T 00d68e696250404d880b8f97af217cbd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:21.430584 1160 consensus_queue.cc:260] T 00d68e696250404d880b8f97af217cbd P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.431150 1160 ts_tablet_manager.cc:1434] T 00d68e696250404d880b8f97af217cbd P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:21.432727 1160 tablet_bootstrap.cc:492] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:21.434402 1160 tablet_bootstrap.cc:654] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:21.427006 1166 tablet_bootstrap.cc:492] T 8140bfd13c50450f9ecb677de47b635e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:21.439999 757 tablet_service.cc:1505] Processing CreateTablet for tablet 12313c06b33749589ceb08ade798fc59 (DEFAULT_TABLE table=default.Table_b37d05b3e8eb4425bb5cb72d46ce9e91 [id=4e18dbcc527d4b12ba6978ca125d5085]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.440220 596 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 00d68e696250404d880b8f97af217cbd (table default.Table_54365c25842c4ec391e069b55177da23 [id=ebae22b7d617403c84c023d0628f8ad8]). Expected version 1 got 0
I20251211 20:52:21.440495 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12313c06b33749589ceb08ade798fc59. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.441052 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_3e00c6ddf63b4b28ba106f5cf6862fa9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:21.441648 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3e00c6ddf63b4b28ba106f5cf6862fa9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.441844 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 638fee757fc444f8a76b5189c103c40c (DEFAULT_TABLE table=default.Table_b142a71a5da94b88925a2e142715ff4d [id=14a0dbb8486246b09f3e06fc2120cd28]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.442216 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 638fee757fc444f8a76b5189c103c40c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.443747 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_1bf24f49f75941369cea32e12a5e81be"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:21.443989 754 tablet_service.cc:1505] Processing CreateTablet for tablet b67de345cf44441abe9e1431b64c4ba4 (DEFAULT_TABLE table=default.Table_ce8eb57ae5624218b6cac53848e34194 [id=b8c401dfdb5f4f7584dca0989287ce31]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:21.444322 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1bf24f49f75941369cea32e12a5e81be in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.444825 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b67de345cf44441abe9e1431b64c4ba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.448276 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_d51d6227f8af4d3086ab98f9e9ca1eee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:21.448843 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d51d6227f8af4d3086ab98f9e9ca1eee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.449438 756 tablet_service.cc:1505] Processing CreateTablet for tablet 638fee757fc444f8a76b5189c103c40c (DEFAULT_TABLE table=default.Table_b142a71a5da94b88925a2e142715ff4d [id=14a0dbb8486246b09f3e06fc2120cd28]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.449846 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 638fee757fc444f8a76b5189c103c40c. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:21.451159 959 heartbeater.cc:646] Failed to heartbeat to 127.0.104.62:41123 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv got EOF from 127.0.104.62:41123 (error 108)
I20251211 20:52:21.451267 1166 tablet_bootstrap.cc:654] T 8140bfd13c50450f9ecb677de47b635e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:21.452080 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_563667ee5f6047ccbddbc84347af40bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:21.452618 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_563667ee5f6047ccbddbc84347af40bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.453215 1166 log.cc:826] T 8140bfd13c50450f9ecb677de47b635e P e715a811a82446ce8d5bb4aafd5b81c7: Log is configured to *not* fsync() on all Append() calls
I20251211 20:52:21.455164 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_34aee520cbc4471a9f0639847077f275"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:21.455327 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet d42e5cce248342f59d67f7c9aa6f66a2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d42e5cce248342f59d67f7c9aa6f66a2 already in progress: creating tablet
W20251211 20:52:21.455794 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_34aee520cbc4471a9f0639847077f275 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:21.457499 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 8140bfd13c50450f9ecb677de47b635e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8140bfd13c50450f9ecb677de47b635e already in progress: creating tablet
W20251211 20:52:21.458380 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet f5467114696046fbb2f311856b7a4d73 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f5467114696046fbb2f311856b7a4d73 already in progress: creating tablet
I20251211 20:52:21.458493 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_673da12cd31d4ba789de37d39fd60988"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:21.458776 1164 tablet_replica.cc:333] T f5467114696046fbb2f311856b7a4d73 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
W20251211 20:52:21.459091 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_673da12cd31d4ba789de37d39fd60988 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:21.461217 822 heartbeater.cc:646] Failed to heartbeat to 127.0.104.62:41123 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv got EOF from 127.0.104.62:41123 (error 108)
I20251211 20:52:21.461413 1164 raft_consensus.cc:2243] T f5467114696046fbb2f311856b7a4d73 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:21.461886 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_096ebbdfa0ac4a208414db520448f78a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:21.462042 584 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.104.62:41123 (error 108)
I20251211 20:52:21.462486 1164 raft_consensus.cc:2272] T f5467114696046fbb2f311856b7a4d73 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:21.462508 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_096ebbdfa0ac4a208414db520448f78a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.464440 1164 ts_tablet_manager.cc:1916] T f5467114696046fbb2f311856b7a4d73 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:21.465751 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_7a3a07a264fc4bed96455f83ae51fb05"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:21.466389 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a3a07a264fc4bed96455f83ae51fb05 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.467085 1166 tablet_bootstrap.cc:492] T 8140bfd13c50450f9ecb677de47b635e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:21.467407 1166 ts_tablet_manager.cc:1403] T 8140bfd13c50450f9ecb677de47b635e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.073s user 0.007s sys 0.000s
I20251211 20:52:21.469538 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_7ce0e2ed6ea4463cac10ba89d3718c5c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:21.470248 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7ce0e2ed6ea4463cac10ba89d3718c5c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:21.473654 1171 negotiation.cc:337] Failed RPC negotiation. Trace:
1211 20:52:21.451823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.104.62:41123 (local address 127.0.104.2:54839)
1211 20:52:21.460866 (+ 9043us) negotiation.cc:107] Waiting for socket to connect
1211 20:52:21.460888 (+ 22us) client_negotiation.cc:174] Beginning negotiation
1211 20:52:21.461024 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
1211 20:52:21.471731 (+ 10707us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.104.62:41123: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":8923,"thread_start_us":8806,"threads_started":1}
I20251211 20:52:21.474082 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_54365c25842c4ec391e069b55177da23" } new_table_name: "default.Table_5ef3d8d2746a4016ad8efac84ba492b5" modify_external_catalogs: true
I20251211 20:52:21.478403 894 tablet_service.cc:1505] Processing CreateTablet for tablet 638fee757fc444f8a76b5189c103c40c (DEFAULT_TABLE table=default.Table_b142a71a5da94b88925a2e142715ff4d [id=14a0dbb8486246b09f3e06fc2120cd28]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.478978 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 638fee757fc444f8a76b5189c103c40c. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:21.479741 584 consensus_peers.cc:597] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 -> Peer 1830b0c58cfe495989f3d911dda0409a (127.0.104.62:41123): Couldn't send request to peer 1830b0c58cfe495989f3d911dda0409a. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.62:41123: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:21.482764 1164 ts_tablet_manager.cc:1929] T f5467114696046fbb2f311856b7a4d73 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:21.482946 1164 log.cc:1199] T f5467114696046fbb2f311856b7a4d73 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/f5467114696046fbb2f311856b7a4d73
I20251211 20:52:21.483449 1164 ts_tablet_manager.cc:1950] T f5467114696046fbb2f311856b7a4d73 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:21.488595 1166 raft_consensus.cc:359] T 8140bfd13c50450f9ecb677de47b635e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.489136 1166 raft_consensus.cc:385] T 8140bfd13c50450f9ecb677de47b635e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:21.489257 1166 raft_consensus.cc:740] T 8140bfd13c50450f9ecb677de47b635e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:21.491968 1166 consensus_queue.cc:260] T 8140bfd13c50450f9ecb677de47b635e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:21.493880 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 8140bfd13c50450f9ecb677de47b635e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8140bfd13c50450f9ecb677de47b635e already in progress: creating tablet
I20251211 20:52:21.494441 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet f5467114696046fbb2f311856b7a4d73 (table default.Table_b37d05b3e8eb4425bb5cb72d46ce9e91 [id=4e18dbcc527d4b12ba6978ca125d5085]) successfully deleted
I20251211 20:52:21.496954 1166 ts_tablet_manager.cc:1434] T 8140bfd13c50450f9ecb677de47b635e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.029s user 0.012s sys 0.004s
I20251211 20:52:21.497296 1095 heartbeater.cc:499] Master 127.0.104.60:42087 was elected leader, sending a full tablet report...
I20251211 20:52:21.499049 1166 tablet_bootstrap.cc:492] T 03ebcee3069f44f99761cc13e34512ed P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:21.499241 1094 heartbeater.cc:646] Failed to heartbeat to 127.0.104.62:41123 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.104.62:41123: connect: Connection refused (error 111)
I20251211 20:52:21.501238 1166 tablet_bootstrap.cc:654] T 03ebcee3069f44f99761cc13e34512ed P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:21.509449 1166 tablet_bootstrap.cc:492] T 03ebcee3069f44f99761cc13e34512ed P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:21.509639 1166 ts_tablet_manager.cc:1403] T 03ebcee3069f44f99761cc13e34512ed P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.008s
I20251211 20:52:21.510715 1166 raft_consensus.cc:359] T 03ebcee3069f44f99761cc13e34512ed P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.510924 1166 raft_consensus.cc:385] T 03ebcee3069f44f99761cc13e34512ed P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:21.511003 1166 raft_consensus.cc:740] T 03ebcee3069f44f99761cc13e34512ed P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:21.511278 1166 consensus_queue.cc:260] T 03ebcee3069f44f99761cc13e34512ed P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.511807 1166 ts_tablet_manager.cc:1434] T 03ebcee3069f44f99761cc13e34512ed P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20251211 20:52:21.512097 1166 tablet_bootstrap.cc:492] T 00d68e696250404d880b8f97af217cbd P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:21.514401 1166 tablet_bootstrap.cc:654] T 00d68e696250404d880b8f97af217cbd P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:21.517068 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 00d68e696250404d880b8f97af217cbd consensus state has no leader
I20251211 20:52:21.523947 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9a500be9b1c94dbb99529a71b1f5b56d" } new_table_name: "default.Table_8cbb799adf2747bba3e7edf97a7f1f59" modify_external_catalogs: true
I20251211 20:52:21.526351 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b142a71a5da94b88925a2e142715ff4d" } new_table_name: "default.Table_fbca1745cad8402fba1c38f2c3b0a9f4" modify_external_catalogs: true
I20251211 20:52:21.528134 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b37d05b3e8eb4425bb5cb72d46ce9e91" } modify_external_catalogs: true
I20251211 20:52:21.528365 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b37d05b3e8eb4425bb5cb72d46ce9e91" } modify_external_catalogs: true
I20251211 20:52:21.530045 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ce8eb57ae5624218b6cac53848e34194" } new_table_name: "default.Table_1b829e0ef7ef4b17be671db434f3858c" modify_external_catalogs: true
I20251211 20:52:21.531687 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ebc736f835124930b0060d37924294af" } modify_external_catalogs: true
I20251211 20:52:21.531903 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ebc736f835124930b0060d37924294af" } modify_external_catalogs: true
I20251211 20:52:21.523480 588 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 00d68e696250404d880b8f97af217cbd (table default.Table_5ef3d8d2746a4016ad8efac84ba492b5 [id=ebae22b7d617403c84c023d0628f8ad8]). Expected version 2 got 0
I20251211 20:52:21.563244 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_1bf24f49f75941369cea32e12a5e81be" } modify_external_catalogs: true
I20251211 20:52:21.563653 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_1bf24f49f75941369cea32e12a5e81be" } modify_external_catalogs: true
I20251211 20:52:21.565186 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_563667ee5f6047ccbddbc84347af40bb" } modify_external_catalogs: true
I20251211 20:52:21.565416 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_563667ee5f6047ccbddbc84347af40bb" } modify_external_catalogs: true
I20251211 20:52:21.566887 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_34aee520cbc4471a9f0639847077f275" } modify_external_catalogs: true
I20251211 20:52:21.567082 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_34aee520cbc4471a9f0639847077f275" } modify_external_catalogs: true
I20251211 20:52:21.577580 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_096ebbdfa0ac4a208414db520448f78a" } new_table_name: "default.Table_40256d3973c74566a09b1d235062d284" modify_external_catalogs: true
W20251211 20:52:21.584408 1097 tablet.cc:2378] T 03ebcee3069f44f99761cc13e34512ed P e715a811a82446ce8d5bb4aafd5b81c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251211 20:52:21.585755 1166 tablet_bootstrap.cc:492] T 00d68e696250404d880b8f97af217cbd P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:21.585983 1166 ts_tablet_manager.cc:1403] T 00d68e696250404d880b8f97af217cbd P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.074s user 0.016s sys 0.043s
I20251211 20:52:21.586633 1166 raft_consensus.cc:359] T 00d68e696250404d880b8f97af217cbd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.586884 1166 raft_consensus.cc:385] T 00d68e696250404d880b8f97af217cbd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:21.587006 1166 raft_consensus.cc:740] T 00d68e696250404d880b8f97af217cbd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:21.587298 1166 consensus_queue.cc:260] T 00d68e696250404d880b8f97af217cbd P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.587865 1160 tablet_bootstrap.cc:492] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:21.588086 1160 ts_tablet_manager.cc:1403] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.155s user 0.034s sys 0.071s
I20251211 20:52:21.588788 588 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet eaba03b935d243c19fc2c346798119f9
I20251211 20:52:21.588948 1160 raft_consensus.cc:359] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:21.589046 588 master_service.cc:954] ReplaceTablet: received request to replace tablet b67de345cf44441abe9e1431b64c4ba4 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:21.589162 1160 raft_consensus.cc:385] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:21.589365 1160 raft_consensus.cc:740] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:21.589697 1160 consensus_queue.cc:260] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:21.590237 1160 ts_tablet_manager.cc:1434] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251211 20:52:21.591470 589 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 24179a60b2304fb4be40ae266cee9cab
I20251211 20:52:21.591841 1160 tablet_bootstrap.cc:492] T 8140bfd13c50450f9ecb677de47b635e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:21.593676 1160 tablet_bootstrap.cc:654] T 8140bfd13c50450f9ecb677de47b635e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:21.594868 1166 ts_tablet_manager.cc:1434] T 00d68e696250404d880b8f97af217cbd P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20251211 20:52:21.597211 1166 tablet_bootstrap.cc:492] T d42e5cce248342f59d67f7c9aa6f66a2 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:21.598141 1168 tablet_bootstrap.cc:492] T 8140bfd13c50450f9ecb677de47b635e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
W20251211 20:52:21.598347 962 tablet.cc:2378] T 00d68e696250404d880b8f97af217cbd P eb0f1933d9a74fccaa4618047fc05c41: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251211 20:52:21.598605 1168 ts_tablet_manager.cc:1403] T 8140bfd13c50450f9ecb677de47b635e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.184s user 0.031s sys 0.105s
I20251211 20:52:21.599143 1166 tablet_bootstrap.cc:654] T d42e5cce248342f59d67f7c9aa6f66a2 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:21.603078 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 49193f7bd8ac4f20a6ca980a9e1eb830 has no consensus state
I20251211 20:52:21.603302 596 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 3678756490fa4fa6a953264710107a04
I20251211 20:52:21.603459 597 master_service.cc:954] ReplaceTablet: received request to replace tablet 03ebcee3069f44f99761cc13e34512ed from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:21.603525 596 master_service.cc:954] ReplaceTablet: received request to replace tablet 00d68e696250404d880b8f97af217cbd from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:21.604564 1160 tablet_bootstrap.cc:492] T 8140bfd13c50450f9ecb677de47b635e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:21.604817 1160 ts_tablet_manager.cc:1403] T 8140bfd13c50450f9ecb677de47b635e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20251211 20:52:21.605743 1160 raft_consensus.cc:359] T 8140bfd13c50450f9ecb677de47b635e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.606009 1160 raft_consensus.cc:385] T 8140bfd13c50450f9ecb677de47b635e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:21.606148 1160 raft_consensus.cc:740] T 8140bfd13c50450f9ecb677de47b635e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:21.606472 1160 consensus_queue.cc:260] T 8140bfd13c50450f9ecb677de47b635e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.607656 1166 tablet_bootstrap.cc:492] T d42e5cce248342f59d67f7c9aa6f66a2 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:21.607822 1166 ts_tablet_manager.cc:1403] T d42e5cce248342f59d67f7c9aa6f66a2 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.001s
I20251211 20:52:21.608645 1166 raft_consensus.cc:359] T d42e5cce248342f59d67f7c9aa6f66a2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:21.608840 1166 raft_consensus.cc:385] T d42e5cce248342f59d67f7c9aa6f66a2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:21.608937 1166 raft_consensus.cc:740] T d42e5cce248342f59d67f7c9aa6f66a2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:21.608944 1160 ts_tablet_manager.cc:1434] T 8140bfd13c50450f9ecb677de47b635e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251211 20:52:21.609176 1166 consensus_queue.cc:260] T d42e5cce248342f59d67f7c9aa6f66a2 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:21.611160 1160 tablet_bootstrap.cc:492] T 03ebcee3069f44f99761cc13e34512ed P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:21.612514 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet b67de345cf44441abe9e1431b64c4ba4
I20251211 20:52:21.613016 588 catalog_manager.cc:6724] ReplaceTablet: tablet b67de345cf44441abe9e1431b64c4ba4 deleted and replaced by tablet d0df24ce65234eeba97f9fdd351b58ad
I20251211 20:52:21.613245 588 master_service.cc:954] ReplaceTablet: received request to replace tablet 638fee757fc444f8a76b5189c103c40c from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:21.613329 1160 tablet_bootstrap.cc:654] T 03ebcee3069f44f99761cc13e34512ed P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:21.613390 756 tablet_service.cc:1552] Processing DeleteTablet for tablet b67de345cf44441abe9e1431b64c4ba4 with delete_type TABLET_DATA_DELETED (replaced by tablet d0df24ce65234eeba97f9fdd351b58ad) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:21.615458 1160 tablet_bootstrap.cc:492] T 03ebcee3069f44f99761cc13e34512ed P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:21.615682 1160 ts_tablet_manager.cc:1403] T 03ebcee3069f44f99761cc13e34512ed P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
W20251211 20:52:21.616146 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet b67de345cf44441abe9e1431b64c4ba4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b67de345cf44441abe9e1431b64c4ba4 already in progress: creating tablet
I20251211 20:52:21.616473 1160 raft_consensus.cc:359] T 03ebcee3069f44f99761cc13e34512ed P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.616274 1168 raft_consensus.cc:359] T 8140bfd13c50450f9ecb677de47b635e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.616675 1160 raft_consensus.cc:385] T 03ebcee3069f44f99761cc13e34512ed P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:21.616669 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet b67de345cf44441abe9e1431b64c4ba4 with delete_type TABLET_DATA_DELETED (replaced by tablet d0df24ce65234eeba97f9fdd351b58ad) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:21.616739 1168 raft_consensus.cc:385] T 8140bfd13c50450f9ecb677de47b635e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:21.616753 1160 raft_consensus.cc:740] T 03ebcee3069f44f99761cc13e34512ed P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:21.617031 1160 consensus_queue.cc:260] T 03ebcee3069f44f99761cc13e34512ed P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.617292 1168 raft_consensus.cc:740] T 8140bfd13c50450f9ecb677de47b635e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
W20251211 20:52:21.617512 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet b67de345cf44441abe9e1431b64c4ba4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b67de345cf44441abe9e1431b64c4ba4 already in progress: creating tablet
I20251211 20:52:21.618173 1168 consensus_queue.cc:260] T 8140bfd13c50450f9ecb677de47b635e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.618748 894 tablet_service.cc:1552] Processing DeleteTablet for tablet b67de345cf44441abe9e1431b64c4ba4 with delete_type TABLET_DATA_DELETED (replaced by tablet d0df24ce65234eeba97f9fdd351b58ad) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:21.619400 1160 ts_tablet_manager.cc:1434] T 03ebcee3069f44f99761cc13e34512ed P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:21.619756 1160 tablet_bootstrap.cc:492] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:21.620159 1166 ts_tablet_manager.cc:1434] T d42e5cce248342f59d67f7c9aa6f66a2 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20251211 20:52:21.620668 1166 tablet_bootstrap.cc:492] T f5467114696046fbb2f311856b7a4d73 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:21.621516 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet b67de345cf44441abe9e1431b64c4ba4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b67de345cf44441abe9e1431b64c4ba4 already in progress: creating tablet
I20251211 20:52:21.621580 1160 tablet_bootstrap.cc:654] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:21.624979 595 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 997fd184e900424bb5eabfde8c676d0e (table default.Table_9a500be9b1c94dbb99529a71b1f5b56d [id=1473b367a32b480f842106bf0f4e41fb]). Expected version 1 got 0
I20251211 20:52:21.625258 595 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 00d68e696250404d880b8f97af217cbd (table default.Table_5ef3d8d2746a4016ad8efac84ba492b5 [id=ebae22b7d617403c84c023d0628f8ad8]). Expected version 2 got 0
I20251211 20:52:21.626267 597 master_service.cc:954] ReplaceTablet: received request to replace tablet 03ebcee3069f44f99761cc13e34512ed from {username='slave'} at 127.0.0.1:44772
W20251211 20:52:21.626901 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 997fd184e900424bb5eabfde8c676d0e consensus state has no leader
W20251211 20:52:21.627126 592 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 638fee757fc444f8a76b5189c103c40c consensus state has no leader
I20251211 20:52:21.627470 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 12313c06b33749589ceb08ade798fc59
I20251211 20:52:21.627720 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 03ebcee3069f44f99761cc13e34512ed
I20251211 20:52:21.627780 1168 ts_tablet_manager.cc:1434] T 8140bfd13c50450f9ecb677de47b635e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.029s user 0.012s sys 0.003s
W20251211 20:52:21.628453 591 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d0df24ce65234eeba97f9fdd351b58ad has no consensus state
W20251211 20:52:21.630461 825 tablet.cc:2378] T 8140bfd13c50450f9ecb677de47b635e P 6f8d784fd1cd492bbe3809403ec34919: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251211 20:52:21.631846 1162 raft_consensus.cc:493] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:21.632087 1162 raft_consensus.cc:515] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:21.632784 1168 tablet_bootstrap.cc:492] T 997fd184e900424bb5eabfde8c676d0e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:21.634076 1162 leader_election.cc:290] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:21.634461 1178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9230f94a6a1a49cead51aeea9f069087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } } }
I20251211 20:52:21.634677 1178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [sys.catalog]: This master's current role is: LEADER
I20251211 20:52:21.634771 1168 tablet_bootstrap.cc:654] T 997fd184e900424bb5eabfde8c676d0e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:21.637832 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 03ebcee3069f44f99761cc13e34512ed with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:21.638152 1164 tablet_replica.cc:333] T 03ebcee3069f44f99761cc13e34512ed P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:21.638278 891 tablet_service.cc:1505] Processing CreateTablet for tablet 49193f7bd8ac4f20a6ca980a9e1eb830 (DEFAULT_TABLE table=default.Table_40256d3973c74566a09b1d235062d284 [id=1f4650c549d94e2c9ef753ff96670aa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.638152 894 tablet_service.cc:1505] Processing CreateTablet for tablet 72d7d2f78ab64bbebb63239b9ecb366e (DEFAULT_TABLE table=default.Table_d51d6227f8af4d3086ab98f9e9ca1eee [id=cda4c0ad8a0347be88a0518491e6af49]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.638414 893 tablet_service.cc:1505] Processing CreateTablet for tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce (DEFAULT_TABLE table=default.Table_7ce0e2ed6ea4463cac10ba89d3718c5c [id=0e7104bb15484108b86dacf1f1f2c359]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.638567 889 tablet_service.cc:1505] Processing CreateTablet for tablet 274c10c2aa46479dab5edbb483e5b883 (DEFAULT_TABLE table=default.Table_7a3a07a264fc4bed96455f83ae51fb05 [id=5a91ece54cfa446880f72a42678f4f26]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.638747 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49193f7bd8ac4f20a6ca980a9e1eb830. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.639075 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72d7d2f78ab64bbebb63239b9ecb366e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.639045 890 tablet_service.cc:1505] Processing CreateTablet for tablet ae3be9d91e5d4551bc706b2043a9edf5 (DEFAULT_TABLE table=default.Table_673da12cd31d4ba789de37d39fd60988 [id=b9fb32193afe4e5faa21db3c195030ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.639240 892 tablet_service.cc:1552] Processing DeleteTablet for tablet d42e5cce248342f59d67f7c9aa6f66a2 with delete_type TABLET_DATA_DELETED (replaced by tablet 997fd184e900424bb5eabfde8c676d0e) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:21.639353 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 274c10c2aa46479dab5edbb483e5b883. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.639461 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 12313c06b33749589ceb08ade798fc59 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:21.640808 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_872ae36ce97749d39238287355dadbe1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:21.641276 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae3be9d91e5d4551bc706b2043a9edf5. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:21.641402 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_872ae36ce97749d39238287355dadbe1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.644223 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.646160 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 00d68e696250404d880b8f97af217cbd
I20251211 20:52:21.646564 596 catalog_manager.cc:6724] ReplaceTablet: tablet 00d68e696250404d880b8f97af217cbd deleted and replaced by tablet dc7e8db4b51c415e86d8c24a76b8cb1e
I20251211 20:52:21.648833 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 638fee757fc444f8a76b5189c103c40c
I20251211 20:52:21.649242 588 catalog_manager.cc:6724] ReplaceTablet: tablet 638fee757fc444f8a76b5189c103c40c deleted and replaced by tablet 62fb046960024a2b9d70dc46ae52d53f
I20251211 20:52:21.651937 1024 tablet_service.cc:1505] Processing CreateTablet for tablet 49193f7bd8ac4f20a6ca980a9e1eb830 (DEFAULT_TABLE table=default.Table_40256d3973c74566a09b1d235062d284 [id=1f4650c549d94e2c9ef753ff96670aa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.652415 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49193f7bd8ac4f20a6ca980a9e1eb830. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.655952 1025 tablet_service.cc:1505] Processing CreateTablet for tablet ae3be9d91e5d4551bc706b2043a9edf5 (DEFAULT_TABLE table=default.Table_673da12cd31d4ba789de37d39fd60988 [id=b9fb32193afe4e5faa21db3c195030ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.656208 1166 tablet_bootstrap.cc:654] T f5467114696046fbb2f311856b7a4d73 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:21.656494 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 03ebcee3069f44f99761cc13e34512ed with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:21.656760 1028 tablet_service.cc:1552] Processing DeleteTablet for tablet 12313c06b33749589ceb08ade798fc59 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:21.656845 1163 tablet_replica.cc:333] T 03ebcee3069f44f99761cc13e34512ed P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:21.656982 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet f5467114696046fbb2f311856b7a4d73 with delete_type TABLET_DATA_DELETED (replaced by tablet 12313c06b33749589ceb08ade798fc59) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:21.657213 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet b67de345cf44441abe9e1431b64c4ba4 with delete_type TABLET_DATA_DELETED (replaced by tablet d0df24ce65234eeba97f9fdd351b58ad) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:21.657450 1163 raft_consensus.cc:2243] T 03ebcee3069f44f99761cc13e34512ed P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:21.657497 1021 tablet_service.cc:1552] Processing DeleteTablet for tablet d42e5cce248342f59d67f7c9aa6f66a2 with delete_type TABLET_DATA_DELETED (replaced by tablet 997fd184e900424bb5eabfde8c676d0e) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:21.657884 1163 raft_consensus.cc:2272] T 03ebcee3069f44f99761cc13e34512ed P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:21.658030 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 274c10c2aa46479dab5edbb483e5b883 (DEFAULT_TABLE table=default.Table_7a3a07a264fc4bed96455f83ae51fb05 [id=5a91ece54cfa446880f72a42678f4f26]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.658995 1023 tablet_service.cc:1505] Processing CreateTablet for tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce (DEFAULT_TABLE table=default.Table_7ce0e2ed6ea4463cac10ba89d3718c5c [id=0e7104bb15484108b86dacf1f1f2c359]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.659622 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 72d7d2f78ab64bbebb63239b9ecb366e (DEFAULT_TABLE table=default.Table_d51d6227f8af4d3086ab98f9e9ca1eee [id=cda4c0ad8a0347be88a0518491e6af49]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.656328 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae3be9d91e5d4551bc706b2043a9edf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.659940 756 tablet_service.cc:1505] Processing CreateTablet for tablet 72d7d2f78ab64bbebb63239b9ecb366e (DEFAULT_TABLE table=default.Table_d51d6227f8af4d3086ab98f9e9ca1eee [id=cda4c0ad8a0347be88a0518491e6af49]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.660436 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72d7d2f78ab64bbebb63239b9ecb366e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.663832 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.667258 756 tablet_service.cc:1505] Processing CreateTablet for tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce (DEFAULT_TABLE table=default.Table_7ce0e2ed6ea4463cac10ba89d3718c5c [id=0e7104bb15484108b86dacf1f1f2c359]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.667798 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.671314 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 274c10c2aa46479dab5edbb483e5b883. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.671766 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72d7d2f78ab64bbebb63239b9ecb366e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.674237 754 tablet_service.cc:1505] Processing CreateTablet for tablet 49193f7bd8ac4f20a6ca980a9e1eb830 (DEFAULT_TABLE table=default.Table_40256d3973c74566a09b1d235062d284 [id=1f4650c549d94e2c9ef753ff96670aa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.674701 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49193f7bd8ac4f20a6ca980a9e1eb830. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.681407 754 tablet_service.cc:1505] Processing CreateTablet for tablet ae3be9d91e5d4551bc706b2043a9edf5 (DEFAULT_TABLE table=default.Table_673da12cd31d4ba789de37d39fd60988 [id=b9fb32193afe4e5faa21db3c195030ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.681826 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae3be9d91e5d4551bc706b2043a9edf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.685618 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 03ebcee3069f44f99761cc13e34512ed with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:21.686358 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 12313c06b33749589ceb08ade798fc59 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:21.687420 1163 ts_tablet_manager.cc:1916] T 03ebcee3069f44f99761cc13e34512ed P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:21.691073 756 tablet_service.cc:1505] Processing CreateTablet for tablet 274c10c2aa46479dab5edbb483e5b883 (DEFAULT_TABLE table=default.Table_7a3a07a264fc4bed96455f83ae51fb05 [id=5a91ece54cfa446880f72a42678f4f26]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.691532 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 274c10c2aa46479dab5edbb483e5b883. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.693804 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 638fee757fc444f8a76b5189c103c40c with delete_type TABLET_DATA_DELETED (replaced by tablet 62fb046960024a2b9d70dc46ae52d53f) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:21.693964 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 03ebcee3069f44f99761cc13e34512ed because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 03ebcee3069f44f99761cc13e34512ed already in progress: creating tablet
W20251211 20:52:21.694190 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 12313c06b33749589ceb08ade798fc59 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12313c06b33749589ceb08ade798fc59 already in progress: creating tablet
I20251211 20:52:21.694456 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 00d68e696250404d880b8f97af217cbd with delete_type TABLET_DATA_DELETED (replaced by tablet dc7e8db4b51c415e86d8c24a76b8cb1e) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:21.694504 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 638fee757fc444f8a76b5189c103c40c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 638fee757fc444f8a76b5189c103c40c already in progress: creating tablet
I20251211 20:52:21.695570 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 00d68e696250404d880b8f97af217cbd with delete_type TABLET_DATA_DELETED (replaced by tablet dc7e8db4b51c415e86d8c24a76b8cb1e) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:21.695861 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 638fee757fc444f8a76b5189c103c40c with delete_type TABLET_DATA_DELETED (replaced by tablet 62fb046960024a2b9d70dc46ae52d53f) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:21.696043 1021 tablet_service.cc:1552] Processing DeleteTablet for tablet 8140bfd13c50450f9ecb677de47b635e with delete_type TABLET_DATA_DELETED (replaced by tablet b67de345cf44441abe9e1431b64c4ba4) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:21.697258 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 00d68e696250404d880b8f97af217cbd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 00d68e696250404d880b8f97af217cbd already in progress: creating tablet
I20251211 20:52:21.701619 1164 raft_consensus.cc:2243] T 03ebcee3069f44f99761cc13e34512ed P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:21.701874 1164 raft_consensus.cc:2272] T 03ebcee3069f44f99761cc13e34512ed P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:21.706925 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 00d68e696250404d880b8f97af217cbd with delete_type TABLET_DATA_DELETED (replaced by tablet dc7e8db4b51c415e86d8c24a76b8cb1e) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:21.710785 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 638fee757fc444f8a76b5189c103c40c with delete_type TABLET_DATA_DELETED (replaced by tablet 62fb046960024a2b9d70dc46ae52d53f) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:21.711483 1164 ts_tablet_manager.cc:1916] T 03ebcee3069f44f99761cc13e34512ed P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:21.717519 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 8140bfd13c50450f9ecb677de47b635e with delete_type TABLET_DATA_DELETED (replaced by tablet b67de345cf44441abe9e1431b64c4ba4) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:21.718220 1165 tablet_replica.cc:333] T 8140bfd13c50450f9ecb677de47b635e P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:21.718743 1165 raft_consensus.cc:2243] T 8140bfd13c50450f9ecb677de47b635e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:21.718778 891 tablet_service.cc:1505] Processing CreateTablet for tablet 8e9eded009e44c1a9544d4471c04c5c8 (DEFAULT_TABLE table=default.Table_3e00c6ddf63b4b28ba106f5cf6862fa9 [id=383392d835da4c50848068d83261cecc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.719174 1165 raft_consensus.cc:2272] T 8140bfd13c50450f9ecb677de47b635e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:21.720232 756 tablet_service.cc:1505] Processing CreateTablet for tablet 8e9eded009e44c1a9544d4471c04c5c8 (DEFAULT_TABLE table=default.Table_3e00c6ddf63b4b28ba106f5cf6862fa9 [id=383392d835da4c50848068d83261cecc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.720222 894 tablet_service.cc:1505] Processing CreateTablet for tablet d0df24ce65234eeba97f9fdd351b58ad (DEFAULT_TABLE table=default.Table_1b829e0ef7ef4b17be671db434f3858c [id=b8c401dfdb5f4f7584dca0989287ce31]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.720660 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e9eded009e44c1a9544d4471c04c5c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.720863 754 tablet_service.cc:1505] Processing CreateTablet for tablet d0df24ce65234eeba97f9fdd351b58ad (DEFAULT_TABLE table=default.Table_1b829e0ef7ef4b17be671db434f3858c [id=b8c401dfdb5f4f7584dca0989287ce31]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.721252 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0df24ce65234eeba97f9fdd351b58ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.723960 1023 tablet_service.cc:1505] Processing CreateTablet for tablet d0df24ce65234eeba97f9fdd351b58ad (DEFAULT_TABLE table=default.Table_1b829e0ef7ef4b17be671db434f3858c [id=b8c401dfdb5f4f7584dca0989287ce31]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.724377 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0df24ce65234eeba97f9fdd351b58ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.725131 1165 ts_tablet_manager.cc:1916] T 8140bfd13c50450f9ecb677de47b635e P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:21.728816 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 8e9eded009e44c1a9544d4471c04c5c8 (DEFAULT_TABLE table=default.Table_3e00c6ddf63b4b28ba106f5cf6862fa9 [id=383392d835da4c50848068d83261cecc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.729265 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e9eded009e44c1a9544d4471c04c5c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.733340 1163 ts_tablet_manager.cc:1929] T 03ebcee3069f44f99761cc13e34512ed P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:21.733548 1163 log.cc:1199] T 03ebcee3069f44f99761cc13e34512ed P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/03ebcee3069f44f99761cc13e34512ed
I20251211 20:52:21.734033 1163 ts_tablet_manager.cc:1950] T 03ebcee3069f44f99761cc13e34512ed P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:21.742151 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_39f957a2d1ac44218e3be30c9b936747"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:21.742354 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e9eded009e44c1a9544d4471c04c5c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.742458 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d42e5cce248342f59d67f7c9aa6f66a2" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:21.742797 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_5e9fc3da4e46438bbf2fd34a24bef2c3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:21.743021 1049 raft_consensus.cc:2468] T d42e5cce248342f59d67f7c9aa6f66a2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 0.
W20251211 20:52:21.743335 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5e9fc3da4e46438bbf2fd34a24bef2c3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:21.742807 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_39f957a2d1ac44218e3be30c9b936747 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.746942 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_a4067eb4ef9d4baba9b72ba67a175f58"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:21.747324 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_d51d6227f8af4d3086ab98f9e9ca1eee" } new_table_name: "default.Table_f439572c5b1d4997af54e0ab3c036023" modify_external_catalogs: true
W20251211 20:52:21.747499 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a4067eb4ef9d4baba9b72ba67a175f58 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.747778 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0df24ce65234eeba97f9fdd351b58ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.747833 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b411ab6a2d934ad8aa183b348dc24d85"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:21.748320 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b411ab6a2d934ad8aa183b348dc24d85 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.748625 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_ce67c2bcef6d4a6baecc11975d852ba8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:21.749119 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ce67c2bcef6d4a6baecc11975d852ba8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.750972 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_0c955f27bf344b1cbe27484262636529"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:21.751471 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c955f27bf344b1cbe27484262636529 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.751781 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 8140bfd13c50450f9ecb677de47b635e with delete_type TABLET_DATA_DELETED (replaced by tablet b67de345cf44441abe9e1431b64c4ba4) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:21.752822 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_9f357fec3afd4c58b2930fea17141463"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:21.753294 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f357fec3afd4c58b2930fea17141463 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.754297 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_9af853b8388442319882d871dc136367"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:21.754787 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9af853b8388442319882d871dc136367 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.756213 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 03ebcee3069f44f99761cc13e34512ed (table default.Table_ebc736f835124930b0060d37924294af [id=7b11b994218b4802a07f78be9ed38877]) successfully deleted
I20251211 20:52:21.756419 1160 tablet_bootstrap.cc:492] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:21.756567 1163 tablet_replica.cc:333] T d42e5cce248342f59d67f7c9aa6f66a2 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:21.756609 1160 ts_tablet_manager.cc:1403] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.137s user 0.007s sys 0.014s
I20251211 20:52:21.756767 1163 raft_consensus.cc:2243] T d42e5cce248342f59d67f7c9aa6f66a2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
W20251211 20:52:21.756882 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 12313c06b33749589ceb08ade798fc59 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12313c06b33749589ceb08ade798fc59 already in progress: creating tablet
I20251211 20:52:21.756945 1163 raft_consensus.cc:2272] T d42e5cce248342f59d67f7c9aa6f66a2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:21.757109 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet f5467114696046fbb2f311856b7a4d73 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f5467114696046fbb2f311856b7a4d73 already in progress: creating tablet
W20251211 20:52:21.757303 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet b67de345cf44441abe9e1431b64c4ba4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b67de345cf44441abe9e1431b64c4ba4 already in progress: creating tablet
I20251211 20:52:21.757484 1160 raft_consensus.cc:359] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.757699 1160 raft_consensus.cc:385] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:21.757786 1160 raft_consensus.cc:740] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:21.758307 1163 ts_tablet_manager.cc:1916] T d42e5cce248342f59d67f7c9aa6f66a2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:21.758249 847 leader_election.cc:304] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41; no voters:
I20251211 20:52:21.758651 1162 raft_consensus.cc:2804] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:21.758812 1162 raft_consensus.cc:493] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:21.758912 1162 raft_consensus.cc:3060] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:21.760399 1166 tablet_bootstrap.cc:492] T f5467114696046fbb2f311856b7a4d73 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:21.760586 1166 ts_tablet_manager.cc:1403] T f5467114696046fbb2f311856b7a4d73 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.140s user 0.035s sys 0.029s
I20251211 20:52:21.761329 1166 raft_consensus.cc:359] T f5467114696046fbb2f311856b7a4d73 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.761533 1166 raft_consensus.cc:385] T f5467114696046fbb2f311856b7a4d73 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:21.761602 1166 raft_consensus.cc:740] T f5467114696046fbb2f311856b7a4d73 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:21.761873 1166 consensus_queue.cc:260] T f5467114696046fbb2f311856b7a4d73 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.762216 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_cbe68d84efef4745822f6aef5a723f11"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:21.762365 1166 ts_tablet_manager.cc:1434] T f5467114696046fbb2f311856b7a4d73 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:21.762591 1164 ts_tablet_manager.cc:1929] T 03ebcee3069f44f99761cc13e34512ed P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:21.762672 1166 tablet_bootstrap.cc:492] T 997fd184e900424bb5eabfde8c676d0e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:21.762890 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cbe68d84efef4745822f6aef5a723f11 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.764351 1166 tablet_bootstrap.cc:654] T 997fd184e900424bb5eabfde8c676d0e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:21.764896 1162 raft_consensus.cc:515] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:21.765719 1162 leader_election.cc:290] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Requested vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:21.766491 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d42e5cce248342f59d67f7c9aa6f66a2" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
W20251211 20:52:21.767416 847 leader_election.cc:343] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Illegal state: Tablet not RUNNING: STOPPED
I20251211 20:52:21.767706 1160 consensus_queue.cc:260] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.768375 1160 ts_tablet_manager.cc:1434] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20251211 20:52:21.764681 1165 ts_tablet_manager.cc:1929] T 8140bfd13c50450f9ecb677de47b635e P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:21.772202 1165 log.cc:1199] T 8140bfd13c50450f9ecb677de47b635e P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/8140bfd13c50450f9ecb677de47b635e
I20251211 20:52:21.772336 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d42e5cce248342f59d67f7c9aa6f66a2" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:21.772461 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d42e5cce248342f59d67f7c9aa6f66a2" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:21.772830 1165 ts_tablet_manager.cc:1950] T 8140bfd13c50450f9ecb677de47b635e P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:21.773926 1163 ts_tablet_manager.cc:1929] T d42e5cce248342f59d67f7c9aa6f66a2 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:21.774151 1163 log.cc:1199] T d42e5cce248342f59d67f7c9aa6f66a2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/d42e5cce248342f59d67f7c9aa6f66a2
I20251211 20:52:21.774572 1163 ts_tablet_manager.cc:1950] T d42e5cce248342f59d67f7c9aa6f66a2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
W20251211 20:52:21.774569 847 leader_election.cc:343] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:21.774729 847 leader_election.cc:304] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:21.775771 1162 raft_consensus.cc:2749] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20251211 20:52:21.775919 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet d42e5cce248342f59d67f7c9aa6f66a2 (table default.Table_8cbb799adf2747bba3e7edf97a7f1f59 [id=1473b367a32b480f842106bf0f4e41fb]) successfully deleted
I20251211 20:52:21.776434 1163 tablet_replica.cc:333] T 00d68e696250404d880b8f97af217cbd P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:21.776679 1160 tablet_bootstrap.cc:492] T b67de345cf44441abe9e1431b64c4ba4 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:21.776684 1163 raft_consensus.cc:2243] T 00d68e696250404d880b8f97af217cbd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:21.776976 1163 raft_consensus.cc:2272] T 00d68e696250404d880b8f97af217cbd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:21.776986 1168 tablet_bootstrap.cc:492] T 997fd184e900424bb5eabfde8c676d0e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:21.777134 1168 ts_tablet_manager.cc:1403] T 997fd184e900424bb5eabfde8c676d0e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.144s user 0.003s sys 0.016s
I20251211 20:52:21.777871 1163 ts_tablet_manager.cc:1916] T 00d68e696250404d880b8f97af217cbd P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:21.777989 1168 raft_consensus.cc:359] T 997fd184e900424bb5eabfde8c676d0e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.778198 1168 raft_consensus.cc:385] T 997fd184e900424bb5eabfde8c676d0e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:21.778304 1168 raft_consensus.cc:740] T 997fd184e900424bb5eabfde8c676d0e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:21.778504 1160 tablet_bootstrap.cc:654] T b67de345cf44441abe9e1431b64c4ba4 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:21.778573 1168 consensus_queue.cc:260] T 997fd184e900424bb5eabfde8c676d0e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.779449 1164 log.cc:1199] T 03ebcee3069f44f99761cc13e34512ed P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/03ebcee3069f44f99761cc13e34512ed
I20251211 20:52:21.780449 1168 ts_tablet_manager.cc:1434] T 997fd184e900424bb5eabfde8c676d0e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251211 20:52:21.780763 1168 tablet_bootstrap.cc:492] T 00d68e696250404d880b8f97af217cbd P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:21.782574 1168 tablet_bootstrap.cc:654] T 00d68e696250404d880b8f97af217cbd P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:21.783715 1166 tablet_bootstrap.cc:492] T 997fd184e900424bb5eabfde8c676d0e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:21.783874 1166 ts_tablet_manager.cc:1403] T 997fd184e900424bb5eabfde8c676d0e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.021s user 0.015s sys 0.006s
I20251211 20:52:21.784576 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 8140bfd13c50450f9ecb677de47b635e (table default.Table_1b829e0ef7ef4b17be671db434f3858c [id=b8c401dfdb5f4f7584dca0989287ce31]) successfully deleted
I20251211 20:52:21.784773 1166 raft_consensus.cc:359] T 997fd184e900424bb5eabfde8c676d0e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.784972 1166 raft_consensus.cc:385] T 997fd184e900424bb5eabfde8c676d0e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:21.785063 1166 raft_consensus.cc:740] T 997fd184e900424bb5eabfde8c676d0e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:21.785301 1166 consensus_queue.cc:260] T 997fd184e900424bb5eabfde8c676d0e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.785799 1166 ts_tablet_manager.cc:1434] T 997fd184e900424bb5eabfde8c676d0e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:21.786073 1166 tablet_bootstrap.cc:492] T 12313c06b33749589ceb08ade798fc59 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:21.787606 1166 tablet_bootstrap.cc:654] T 12313c06b33749589ceb08ade798fc59 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:21.793773 1020 tablet_service.cc:1505] Processing CreateTablet for tablet dc7e8db4b51c415e86d8c24a76b8cb1e (DEFAULT_TABLE table=default.Table_5ef3d8d2746a4016ad8efac84ba492b5 [id=ebae22b7d617403c84c023d0628f8ad8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.779881 1164 ts_tablet_manager.cc:1950] T 03ebcee3069f44f99761cc13e34512ed P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:21.794217 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc7e8db4b51c415e86d8c24a76b8cb1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.797623 1168 tablet_bootstrap.cc:492] T 00d68e696250404d880b8f97af217cbd P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:21.797837 1168 ts_tablet_manager.cc:1403] T 00d68e696250404d880b8f97af217cbd P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.017s user 0.013s sys 0.002s
I20251211 20:52:21.794144 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 62fb046960024a2b9d70dc46ae52d53f (DEFAULT_TABLE table=default.Table_fbca1745cad8402fba1c38f2c3b0a9f4 [id=14a0dbb8486246b09f3e06fc2120cd28]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.798373 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62fb046960024a2b9d70dc46ae52d53f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.798946 1168 raft_consensus.cc:359] T 00d68e696250404d880b8f97af217cbd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.799188 1168 raft_consensus.cc:385] T 00d68e696250404d880b8f97af217cbd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:21.799292 1168 raft_consensus.cc:740] T 00d68e696250404d880b8f97af217cbd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:21.799543 1023 tablet_service.cc:1505] Processing CreateTablet for tablet deacd1f8f3bf4b1f923b61b312492104 (DEFAULT_TABLE table=default.Table_872ae36ce97749d39238287355dadbe1 [id=98bd7ff031214ba7a5e974da798e1248]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.799607 1168 consensus_queue.cc:260] T 00d68e696250404d880b8f97af217cbd P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.800117 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet deacd1f8f3bf4b1f923b61b312492104. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.801487 1168 ts_tablet_manager.cc:1434] T 00d68e696250404d880b8f97af217cbd P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20251211 20:52:21.801734 754 tablet_service.cc:1505] Processing CreateTablet for tablet dc7e8db4b51c415e86d8c24a76b8cb1e (DEFAULT_TABLE table=default.Table_5ef3d8d2746a4016ad8efac84ba492b5 [id=ebae22b7d617403c84c023d0628f8ad8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.801976 756 tablet_service.cc:1505] Processing CreateTablet for tablet deacd1f8f3bf4b1f923b61b312492104 (DEFAULT_TABLE table=default.Table_872ae36ce97749d39238287355dadbe1 [id=98bd7ff031214ba7a5e974da798e1248]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.802121 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc7e8db4b51c415e86d8c24a76b8cb1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.802294 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet deacd1f8f3bf4b1f923b61b312492104. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.803179 1168 tablet_bootstrap.cc:492] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:21.805284 1168 tablet_bootstrap.cc:654] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:21.808439 757 tablet_service.cc:1505] Processing CreateTablet for tablet 62fb046960024a2b9d70dc46ae52d53f (DEFAULT_TABLE table=default.Table_fbca1745cad8402fba1c38f2c3b0a9f4 [id=14a0dbb8486246b09f3e06fc2120cd28]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.808841 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62fb046960024a2b9d70dc46ae52d53f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.811947 1168 tablet_bootstrap.cc:492] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:21.812088 1168 ts_tablet_manager.cc:1403] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20251211 20:52:21.812992 1168 raft_consensus.cc:359] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.813174 1168 raft_consensus.cc:385] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:21.813247 1168 raft_consensus.cc:740] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:21.813472 1168 consensus_queue.cc:260] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.816041 1168 ts_tablet_manager.cc:1434] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:21.817984 1168 tablet_bootstrap.cc:492] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:21.820411 1168 tablet_bootstrap.cc:654] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:21.825340 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 72d7d2f78ab64bbebb63239b9ecb366e consensus state has no leader
I20251211 20:52:21.833518 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_3e00c6ddf63b4b28ba106f5cf6862fa9" } new_table_name: "default.Table_7faeeac71cd74273819ae03fe5272ee4" modify_external_catalogs: true
I20251211 20:52:21.837667 594 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 997fd184e900424bb5eabfde8c676d0e (table default.Table_8cbb799adf2747bba3e7edf97a7f1f59 [id=1473b367a32b480f842106bf0f4e41fb]). Expected version 2 got 0
I20251211 20:52:21.837764 893 tablet_service.cc:1505] Processing CreateTablet for tablet 62fb046960024a2b9d70dc46ae52d53f (DEFAULT_TABLE table=default.Table_fbca1745cad8402fba1c38f2c3b0a9f4 [id=14a0dbb8486246b09f3e06fc2120cd28]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.838601 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62fb046960024a2b9d70dc46ae52d53f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.838531 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8cbb799adf2747bba3e7edf97a7f1f59" } modify_external_catalogs: true
I20251211 20:52:21.838743 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8cbb799adf2747bba3e7edf97a7f1f59" } modify_external_catalogs: true
I20251211 20:52:21.839993 1164 tablet_replica.cc:333] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:21.840195 1164 raft_consensus.cc:2243] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:21.840406 1164 raft_consensus.cc:2272] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:21.840435 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_1b829e0ef7ef4b17be671db434f3858c" } modify_external_catalogs: true
I20251211 20:52:21.840631 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_1b829e0ef7ef4b17be671db434f3858c" } modify_external_catalogs: true
I20251211 20:52:21.842056 1162 raft_consensus.cc:493] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:21.842530 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_7ce0e2ed6ea4463cac10ba89d3718c5c" } modify_external_catalogs: true
I20251211 20:52:21.842715 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_7ce0e2ed6ea4463cac10ba89d3718c5c" } modify_external_catalogs: true
I20251211 20:52:21.843669 1162 raft_consensus.cc:515] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.844383 1162 leader_election.cc:290] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:21.845085 894 tablet_service.cc:1505] Processing CreateTablet for tablet dc7e8db4b51c415e86d8c24a76b8cb1e (DEFAULT_TABLE table=default.Table_5ef3d8d2746a4016ad8efac84ba492b5 [id=ebae22b7d617403c84c023d0628f8ad8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.846386 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "997fd184e900424bb5eabfde8c676d0e" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:21.846777 777 raft_consensus.cc:2468] T 997fd184e900424bb5eabfde8c676d0e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 0.
I20251211 20:52:21.847254 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc7e8db4b51c415e86d8c24a76b8cb1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.848349 891 tablet_service.cc:1505] Processing CreateTablet for tablet deacd1f8f3bf4b1f923b61b312492104 (DEFAULT_TABLE table=default.Table_872ae36ce97749d39238287355dadbe1 [id=98bd7ff031214ba7a5e974da798e1248]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:21.849570 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "997fd184e900424bb5eabfde8c676d0e" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:21.849838 1049 raft_consensus.cc:2468] T 997fd184e900424bb5eabfde8c676d0e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 0.
I20251211 20:52:21.849920 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 03ebcee3069f44f99761cc13e34512ed (table default.Table_ebc736f835124930b0060d37924294af [id=7b11b994218b4802a07f78be9ed38877]) successfully deleted
I20251211 20:52:21.850354 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_39f957a2d1ac44218e3be30c9b936747" } new_table_name: "default.Table_512673b662ef4de0a2c2328d3310f4bf" modify_external_catalogs: true
I20251211 20:52:21.851442 847 leader_election.cc:304] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters:
I20251211 20:52:21.851785 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_7a3a07a264fc4bed96455f83ae51fb05" } modify_external_catalogs: true
I20251211 20:52:21.851958 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_7a3a07a264fc4bed96455f83ae51fb05" } modify_external_catalogs: true
I20251211 20:52:21.852622 1162 raft_consensus.cc:2804] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:21.852730 1162 raft_consensus.cc:493] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:21.852826 1162 raft_consensus.cc:3060] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:21.853271 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a4067eb4ef9d4baba9b72ba67a175f58" } new_table_name: "default.Table_5fa0a824d9234eaebcbbd089e47d3f75" modify_external_catalogs: true
I20251211 20:52:21.856320 1164 ts_tablet_manager.cc:1916] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:21.857457 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_5e9fc3da4e46438bbf2fd34a24bef2c3" } new_table_name: "default.Table_4f9b0d5d6ec647df87f89a25344ca9f8" modify_external_catalogs: true
I20251211 20:52:21.861057 1162 raft_consensus.cc:515] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.863937 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet deacd1f8f3bf4b1f923b61b312492104. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:21.868321 1162 leader_election.cc:290] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Requested vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:21.874868 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "997fd184e900424bb5eabfde8c676d0e" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:21.875149 1049 raft_consensus.cc:3060] T 997fd184e900424bb5eabfde8c676d0e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:21.878710 1049 raft_consensus.cc:2468] T 997fd184e900424bb5eabfde8c676d0e P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 1.
I20251211 20:52:21.879376 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "997fd184e900424bb5eabfde8c676d0e" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:21.879693 777 raft_consensus.cc:3060] T 997fd184e900424bb5eabfde8c676d0e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:21.883262 777 raft_consensus.cc:2468] T 997fd184e900424bb5eabfde8c676d0e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 1.
I20251211 20:52:21.874689 1164 ts_tablet_manager.cc:1929] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:21.896008 1164 log.cc:1199] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/d42e5cce248342f59d67f7c9aa6f66a2
I20251211 20:52:21.896471 1164 ts_tablet_manager.cc:1950] T d42e5cce248342f59d67f7c9aa6f66a2 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:21.892194 847 leader_election.cc:304] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters:
I20251211 20:52:21.897235 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet f5467114696046fbb2f311856b7a4d73 with delete_type TABLET_DATA_DELETED (replaced by tablet 12313c06b33749589ceb08ade798fc59) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:21.897284 1162 raft_consensus.cc:2804] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:21.898505 1164 tablet_replica.cc:333] T 00d68e696250404d880b8f97af217cbd P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:21.898747 1164 raft_consensus.cc:2243] T 00d68e696250404d880b8f97af217cbd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:21.898909 1164 raft_consensus.cc:2272] T 00d68e696250404d880b8f97af217cbd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:21.899699 1164 ts_tablet_manager.cc:1916] T 00d68e696250404d880b8f97af217cbd P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:21.899644 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet d42e5cce248342f59d67f7c9aa6f66a2 (table default.Table_8cbb799adf2747bba3e7edf97a7f1f59 [id=1473b367a32b480f842106bf0f4e41fb]) successfully deleted
W20251211 20:52:21.900002 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 12313c06b33749589ceb08ade798fc59 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12313c06b33749589ceb08ade798fc59 already in progress: creating tablet
I20251211 20:52:21.906417 1164 ts_tablet_manager.cc:1929] T 00d68e696250404d880b8f97af217cbd P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:21.906621 1164 log.cc:1199] T 00d68e696250404d880b8f97af217cbd P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/00d68e696250404d880b8f97af217cbd
I20251211 20:52:21.907027 1164 ts_tablet_manager.cc:1950] T 00d68e696250404d880b8f97af217cbd P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:21.908200 1164 tablet_replica.cc:333] T 8140bfd13c50450f9ecb677de47b635e P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:21.908174 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 00d68e696250404d880b8f97af217cbd (table default.Table_5ef3d8d2746a4016ad8efac84ba492b5 [id=ebae22b7d617403c84c023d0628f8ad8]) successfully deleted
I20251211 20:52:21.908392 1164 raft_consensus.cc:2243] T 8140bfd13c50450f9ecb677de47b635e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:21.908579 1164 raft_consensus.cc:2272] T 8140bfd13c50450f9ecb677de47b635e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:21.908689 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 638fee757fc444f8a76b5189c103c40c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 638fee757fc444f8a76b5189c103c40c already in progress: creating tablet
I20251211 20:52:21.909999 1160 tablet_bootstrap.cc:492] T b67de345cf44441abe9e1431b64c4ba4 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:21.910184 1160 ts_tablet_manager.cc:1403] T b67de345cf44441abe9e1431b64c4ba4 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.134s user 0.013s sys 0.026s
I20251211 20:52:21.911026 1160 raft_consensus.cc:359] T b67de345cf44441abe9e1431b64c4ba4 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.911234 1160 raft_consensus.cc:385] T b67de345cf44441abe9e1431b64c4ba4 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:21.911324 1160 raft_consensus.cc:740] T b67de345cf44441abe9e1431b64c4ba4 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:21.911617 1160 consensus_queue.cc:260] T b67de345cf44441abe9e1431b64c4ba4 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.911980 1162 raft_consensus.cc:697] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41 [term 1 LEADER]: Becoming Leader. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Running, Role: LEADER
I20251211 20:52:21.912405 1162 consensus_queue.cc:237] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.916560 1160 ts_tablet_manager.cc:1434] T b67de345cf44441abe9e1431b64c4ba4 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20251211 20:52:21.916929 1160 tablet_bootstrap.cc:492] T 12313c06b33749589ceb08ade798fc59 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:21.919559 1160 tablet_bootstrap.cc:654] T 12313c06b33749589ceb08ade798fc59 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:21.918758 1163 ts_tablet_manager.cc:1929] T 00d68e696250404d880b8f97af217cbd P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:21.923156 1164 ts_tablet_manager.cc:1916] T 8140bfd13c50450f9ecb677de47b635e P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:21.923425 1163 log.cc:1199] T 00d68e696250404d880b8f97af217cbd P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/00d68e696250404d880b8f97af217cbd
I20251211 20:52:21.924022 1163 ts_tablet_manager.cc:1950] T 00d68e696250404d880b8f97af217cbd P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:21.925750 1163 tablet_replica.cc:333] T 8140bfd13c50450f9ecb677de47b635e P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
W20251211 20:52:21.928208 592 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8e9eded009e44c1a9544d4471c04c5c8 consensus state has no leader
I20251211 20:52:21.930002 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet d0df24ce65234eeba97f9fdd351b58ad
I20251211 20:52:21.930819 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_5ef3d8d2746a4016ad8efac84ba492b5" } modify_external_catalogs: true
I20251211 20:52:21.931013 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_5ef3d8d2746a4016ad8efac84ba492b5" } modify_external_catalogs: true
I20251211 20:52:21.931825 1163 raft_consensus.cc:2243] T 8140bfd13c50450f9ecb677de47b635e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:21.932044 1163 raft_consensus.cc:2272] T 8140bfd13c50450f9ecb677de47b635e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:21.932526 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d703a5a42b2f47d0a7ffe1bb534a1b05 has no consensus state
I20251211 20:52:21.932767 1163 ts_tablet_manager.cc:1916] T 8140bfd13c50450f9ecb677de47b635e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:21.932811 597 master_service.cc:954] ReplaceTablet: received request to replace tablet 49193f7bd8ac4f20a6ca980a9e1eb830 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:21.934082 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 274c10c2aa46479dab5edbb483e5b883
I20251211 20:52:21.934656 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 72d7d2f78ab64bbebb63239b9ecb366e from {username='slave'} at 127.0.0.1:44772
W20251211 20:52:21.935905 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 01ada50100bc48e39d1bc67082da750e has no consensus state
I20251211 20:52:21.936175 593 master_service.cc:954] ReplaceTablet: received request to replace tablet 12313c06b33749589ceb08ade798fc59 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:21.936601 593 master_service.cc:954] ReplaceTablet: received request to replace tablet 274c10c2aa46479dab5edbb483e5b883 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:21.936961 593 master_service.cc:954] ReplaceTablet: received request to replace tablet d42e5cce248342f59d67f7c9aa6f66a2 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:21.937405 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_8f2aa060e015406f805718a21dc1dd01"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:21.938083 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f2aa060e015406f805718a21dc1dd01 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:21.939373 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce
I20251211 20:52:21.941195 1166 tablet_bootstrap.cc:492] T 12313c06b33749589ceb08ade798fc59 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:21.941361 1168 tablet_bootstrap.cc:492] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:21.941368 1164 ts_tablet_manager.cc:1929] T 8140bfd13c50450f9ecb677de47b635e P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:21.941489 1164 log.cc:1199] T 8140bfd13c50450f9ecb677de47b635e P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/8140bfd13c50450f9ecb677de47b635e
I20251211 20:52:21.941484 1166 ts_tablet_manager.cc:1403] T 12313c06b33749589ceb08ade798fc59 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.155s user 0.044s sys 0.079s
I20251211 20:52:21.941672 1168 ts_tablet_manager.cc:1403] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.124s user 0.052s sys 0.066s
I20251211 20:52:21.941862 1164 ts_tablet_manager.cc:1950] T 8140bfd13c50450f9ecb677de47b635e P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:21.942502 1168 raft_consensus.cc:359] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.942514 1166 raft_consensus.cc:359] T 12313c06b33749589ceb08ade798fc59 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.942718 1168 raft_consensus.cc:385] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:21.942798 1168 raft_consensus.cc:740] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:21.942845 1166 raft_consensus.cc:385] T 12313c06b33749589ceb08ade798fc59 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:21.943068 1163 ts_tablet_manager.cc:1929] T 8140bfd13c50450f9ecb677de47b635e P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:21.943080 1168 consensus_queue.cc:260] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.943228 1163 log.cc:1199] T 8140bfd13c50450f9ecb677de47b635e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/8140bfd13c50450f9ecb677de47b635e
I20251211 20:52:21.943027 1166 raft_consensus.cc:740] T 12313c06b33749589ceb08ade798fc59 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:21.943639 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 00d68e696250404d880b8f97af217cbd (table default.Table_5ef3d8d2746a4016ad8efac84ba492b5 [id=ebae22b7d617403c84c023d0628f8ad8]) successfully deleted
W20251211 20:52:21.943907 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 638fee757fc444f8a76b5189c103c40c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 638fee757fc444f8a76b5189c103c40c already in progress: creating tablet
I20251211 20:52:21.944227 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet d0df24ce65234eeba97f9fdd351b58ad with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:21.944451 1168 ts_tablet_manager.cc:1434] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:21.944610 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 274c10c2aa46479dab5edbb483e5b883 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:21.944801 1168 tablet_bootstrap.cc:492] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:21.945007 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:21.943897 1166 consensus_queue.cc:260] T 12313c06b33749589ceb08ade798fc59 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.945290 1163 ts_tablet_manager.cc:1950] T 8140bfd13c50450f9ecb677de47b635e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:21.945600 756 tablet_service.cc:1552] Processing DeleteTablet for tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:21.945812 754 tablet_service.cc:1552] Processing DeleteTablet for tablet d0df24ce65234eeba97f9fdd351b58ad with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:21.946043 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 274c10c2aa46479dab5edbb483e5b883 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:21.948248 1166 ts_tablet_manager.cc:1434] T 12313c06b33749589ceb08ade798fc59 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20251211 20:52:21.948895 893 tablet_service.cc:1552] Processing DeleteTablet for tablet d0df24ce65234eeba97f9fdd351b58ad with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:21.949147 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 274c10c2aa46479dab5edbb483e5b883 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:21.949353 891 tablet_service.cc:1552] Processing DeleteTablet for tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:21.949934 1166 tablet_bootstrap.cc:492] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:21.950718 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce already in progress: creating tablet
W20251211 20:52:21.950975 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet d0df24ce65234eeba97f9fdd351b58ad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d0df24ce65234eeba97f9fdd351b58ad already in progress: creating tablet
W20251211 20:52:21.951181 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 274c10c2aa46479dab5edbb483e5b883 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 274c10c2aa46479dab5edbb483e5b883 already in progress: creating tablet
I20251211 20:52:21.951720 1166 tablet_bootstrap.cc:654] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:21.952042 1168 tablet_bootstrap.cc:654] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:21.952667 1163 tablet_replica.cc:333] T f5467114696046fbb2f311856b7a4d73 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:21.952843 1163 raft_consensus.cc:2243] T f5467114696046fbb2f311856b7a4d73 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:21.952943 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 997fd184e900424bb5eabfde8c676d0e
I20251211 20:52:21.953020 1163 raft_consensus.cc:2272] T f5467114696046fbb2f311856b7a4d73 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:21.953766 1163 ts_tablet_manager.cc:1916] T f5467114696046fbb2f311856b7a4d73 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:21.954587 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 997fd184e900424bb5eabfde8c676d0e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:21.955705 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 997fd184e900424bb5eabfde8c676d0e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:21.956068 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 997fd184e900424bb5eabfde8c676d0e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:21.956326 1165 tablet_replica.cc:333] T 997fd184e900424bb5eabfde8c676d0e P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:21.956519 1165 raft_consensus.cc:2243] T 997fd184e900424bb5eabfde8c676d0e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:21.956696 1165 raft_consensus.cc:2272] T 997fd184e900424bb5eabfde8c676d0e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:21.956825 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet edc4300dc5a84173aa95b587d77fc95c has no consensus state
I20251211 20:52:21.957190 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 12313c06b33749589ceb08ade798fc59 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:21.957888 1165 ts_tablet_manager.cc:1916] T 997fd184e900424bb5eabfde8c676d0e P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:21.958101 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet d0df24ce65234eeba97f9fdd351b58ad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d0df24ce65234eeba97f9fdd351b58ad already in progress: creating tablet
I20251211 20:52:21.958614 891 tablet_service.cc:1552] Processing DeleteTablet for tablet b67de345cf44441abe9e1431b64c4ba4 with delete_type TABLET_DATA_DELETED (replaced by tablet d0df24ce65234eeba97f9fdd351b58ad) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:21.959215 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 274c10c2aa46479dab5edbb483e5b883 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 274c10c2aa46479dab5edbb483e5b883 already in progress: creating tablet
W20251211 20:52:21.959872 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce already in progress: creating tablet
I20251211 20:52:21.960150 1164 tablet_replica.cc:333] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:21.960332 1164 raft_consensus.cc:2243] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:21.960767 1164 pending_rounds.cc:70] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41: Trying to abort 1 pending ops.
I20251211 20:52:21.960976 1164 pending_rounds.cc:77] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432056478638080 op_type: NO_OP noop_request { }
I20251211 20:52:21.961144 1164 raft_consensus.cc:2889] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:21.961262 1164 raft_consensus.cc:2272] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:21.962644 1164 ts_tablet_manager.cc:1916] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:21.965378 1165 ts_tablet_manager.cc:1929] T 997fd184e900424bb5eabfde8c676d0e P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:21.965637 1165 log.cc:1199] T 997fd184e900424bb5eabfde8c676d0e P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/997fd184e900424bb5eabfde8c676d0e
I20251211 20:52:21.966084 1165 ts_tablet_manager.cc:1950] T 997fd184e900424bb5eabfde8c676d0e P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:21.966373 1163 ts_tablet_manager.cc:1929] T f5467114696046fbb2f311856b7a4d73 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:21.966511 1163 log.cc:1199] T f5467114696046fbb2f311856b7a4d73 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/f5467114696046fbb2f311856b7a4d73
I20251211 20:52:21.966864 1163 ts_tablet_manager.cc:1950] T f5467114696046fbb2f311856b7a4d73 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:21.967669 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 997fd184e900424bb5eabfde8c676d0e (table default.Table_8cbb799adf2747bba3e7edf97a7f1f59 [id=1473b367a32b480f842106bf0f4e41fb]) successfully deleted
I20251211 20:52:21.967957 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet f5467114696046fbb2f311856b7a4d73 (table default.Table_b37d05b3e8eb4425bb5cb72d46ce9e91 [id=4e18dbcc527d4b12ba6978ca125d5085]) successfully deleted
I20251211 20:52:21.969154 1164 ts_tablet_manager.cc:1929] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:21.969288 1164 log.cc:1199] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/997fd184e900424bb5eabfde8c676d0e
I20251211 20:52:21.969650 1164 ts_tablet_manager.cc:1950] T 997fd184e900424bb5eabfde8c676d0e P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
W20251211 20:52:21.969801 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet d0df24ce65234eeba97f9fdd351b58ad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d0df24ce65234eeba97f9fdd351b58ad already in progress: creating tablet
I20251211 20:52:21.971071 1163 tablet_replica.cc:333] T 997fd184e900424bb5eabfde8c676d0e P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:21.971280 1163 raft_consensus.cc:2243] T 997fd184e900424bb5eabfde8c676d0e P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:21.971459 1163 raft_consensus.cc:2272] T 997fd184e900424bb5eabfde8c676d0e P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:21.971907 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 997fd184e900424bb5eabfde8c676d0e (table default.Table_8cbb799adf2747bba3e7edf97a7f1f59 [id=1473b367a32b480f842106bf0f4e41fb]) successfully deleted
W20251211 20:52:21.973096 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 274c10c2aa46479dab5edbb483e5b883 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 274c10c2aa46479dab5edbb483e5b883 already in progress: creating tablet
W20251211 20:52:21.973357 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce already in progress: creating tablet
I20251211 20:52:21.974335 1163 ts_tablet_manager.cc:1916] T 997fd184e900424bb5eabfde8c676d0e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:21.974453 1160 tablet_bootstrap.cc:492] T 12313c06b33749589ceb08ade798fc59 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:21.974606 1160 ts_tablet_manager.cc:1403] T 12313c06b33749589ceb08ade798fc59 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.058s user 0.000s sys 0.003s
I20251211 20:52:21.975096 1166 tablet_bootstrap.cc:492] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:21.975237 1166 ts_tablet_manager.cc:1403] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.007s
I20251211 20:52:21.976043 1166 raft_consensus.cc:359] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.976236 1166 raft_consensus.cc:385] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:21.976336 1166 raft_consensus.cc:740] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:21.976600 1166 consensus_queue.cc:260] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.977187 1166 ts_tablet_manager.cc:1434] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:21.977582 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 49193f7bd8ac4f20a6ca980a9e1eb830
I20251211 20:52:21.978080 597 catalog_manager.cc:6724] ReplaceTablet: tablet 49193f7bd8ac4f20a6ca980a9e1eb830 deleted and replaced by tablet 5ed0a0141051462eb45ca3a66d41d009
I20251211 20:52:21.978652 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 72d7d2f78ab64bbebb63239b9ecb366e
I20251211 20:52:21.978960 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet dc7e8db4b51c415e86d8c24a76b8cb1e
I20251211 20:52:21.979027 595 catalog_manager.cc:6724] ReplaceTablet: tablet 72d7d2f78ab64bbebb63239b9ecb366e deleted and replaced by tablet 8a064789f89c424ca698e52333d73eea
I20251211 20:52:21.982125 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 49193f7bd8ac4f20a6ca980a9e1eb830 with delete_type TABLET_DATA_DELETED (replaced by tablet 5ed0a0141051462eb45ca3a66d41d009) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:21.982438 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 72d7d2f78ab64bbebb63239b9ecb366e with delete_type TABLET_DATA_DELETED (replaced by tablet 8a064789f89c424ca698e52333d73eea) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:21.982642 893 tablet_service.cc:1552] Processing DeleteTablet for tablet dc7e8db4b51c415e86d8c24a76b8cb1e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:21.983049 1160 raft_consensus.cc:359] T 12313c06b33749589ceb08ade798fc59 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.983150 416 external_mini_cluster.cc:1366] Running /tmp/dist-test-task5TWsQc/build/asan/bin/kudu
/tmp/dist-test-task5TWsQc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.62:41123
--webserver_interface=127.0.104.62
--webserver_port=37413
--builtin_ntp_servers=127.0.104.20:41333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.62:41123,127.0.104.61:40749,127.0.104.60:42087
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20251211 20:52:21.983251 1160 raft_consensus.cc:385] T 12313c06b33749589ceb08ade798fc59 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:21.983336 1160 raft_consensus.cc:740] T 12313c06b33749589ceb08ade798fc59 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:21.983621 1160 consensus_queue.cc:260] T 12313c06b33749589ceb08ade798fc59 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:21.984889 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 49193f7bd8ac4f20a6ca980a9e1eb830 with delete_type TABLET_DATA_DELETED (replaced by tablet 5ed0a0141051462eb45ca3a66d41d009) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:21.985136 756 tablet_service.cc:1552] Processing DeleteTablet for tablet dc7e8db4b51c415e86d8c24a76b8cb1e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:21.985221 1160 ts_tablet_manager.cc:1434] T 12313c06b33749589ceb08ade798fc59 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.011s user 0.001s sys 0.002s
I20251211 20:52:21.985291 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 00d68e696250404d880b8f97af217cbd with delete_type TABLET_DATA_DELETED (replaced by tablet dc7e8db4b51c415e86d8c24a76b8cb1e) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:21.985860 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 03ebcee3069f44f99761cc13e34512ed with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:21.985998 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 72d7d2f78ab64bbebb63239b9ecb366e with delete_type TABLET_DATA_DELETED (replaced by tablet 8a064789f89c424ca698e52333d73eea) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:21.986238 753 tablet_service.cc:1552] Processing DeleteTablet for tablet 997fd184e900424bb5eabfde8c676d0e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:21.986322 1160 tablet_bootstrap.cc:492] T 638fee757fc444f8a76b5189c103c40c P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:21.988073 1160 tablet_bootstrap.cc:654] T 638fee757fc444f8a76b5189c103c40c P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:21.989100 1164 tablet_replica.cc:333] T b67de345cf44441abe9e1431b64c4ba4 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:21.989784 1164 raft_consensus.cc:2243] T b67de345cf44441abe9e1431b64c4ba4 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:21.989974 1164 raft_consensus.cc:2272] T b67de345cf44441abe9e1431b64c4ba4 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:21.990048 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 12313c06b33749589ceb08ade798fc59 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:21.990789 1164 ts_tablet_manager.cc:1916] T b67de345cf44441abe9e1431b64c4ba4 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:21.990993 753 tablet_service.cc:1552] Processing DeleteTablet for tablet f5467114696046fbb2f311856b7a4d73 with delete_type TABLET_DATA_DELETED (replaced by tablet 12313c06b33749589ceb08ade798fc59) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:21.993089 1168 tablet_bootstrap.cc:492] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:21.993238 1168 ts_tablet_manager.cc:1403] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.049s user 0.001s sys 0.008s
W20251211 20:52:21.993209 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 49193f7bd8ac4f20a6ca980a9e1eb830 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 49193f7bd8ac4f20a6ca980a9e1eb830 already in progress: creating tablet
W20251211 20:52:21.993945 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet dc7e8db4b51c415e86d8c24a76b8cb1e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dc7e8db4b51c415e86d8c24a76b8cb1e already in progress: creating tablet
I20251211 20:52:21.994072 1168 raft_consensus.cc:359] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:21.994261 1168 raft_consensus.cc:385] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:21.994370 1168 raft_consensus.cc:740] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:21.994477 1165 tablet_replica.cc:333] T 00d68e696250404d880b8f97af217cbd P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:21.994611 1168 consensus_queue.cc:260] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:21.995270 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 49193f7bd8ac4f20a6ca980a9e1eb830 with delete_type TABLET_DATA_DELETED (replaced by tablet 5ed0a0141051462eb45ca3a66d41d009) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:21.995708 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 72d7d2f78ab64bbebb63239b9ecb366e with delete_type TABLET_DATA_DELETED (replaced by tablet 8a064789f89c424ca698e52333d73eea) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:21.996675 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet dc7e8db4b51c415e86d8c24a76b8cb1e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:21.998087 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet b67de345cf44441abe9e1431b64c4ba4 with delete_type TABLET_DATA_DELETED (replaced by tablet d0df24ce65234eeba97f9fdd351b58ad) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:21.998878 1164 ts_tablet_manager.cc:1929] T b67de345cf44441abe9e1431b64c4ba4 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:21.999028 1164 log.cc:1199] T b67de345cf44441abe9e1431b64c4ba4 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/b67de345cf44441abe9e1431b64c4ba4
I20251211 20:52:21.999389 1164 ts_tablet_manager.cc:1950] T b67de345cf44441abe9e1431b64c4ba4 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:22.000146 1163 ts_tablet_manager.cc:1929] T 997fd184e900424bb5eabfde8c676d0e P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.000326 1163 log.cc:1199] T 997fd184e900424bb5eabfde8c676d0e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/997fd184e900424bb5eabfde8c676d0e
I20251211 20:52:22.000639 1163 ts_tablet_manager.cc:1950] T 997fd184e900424bb5eabfde8c676d0e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:21.995071 1168 ts_tablet_manager.cc:1434] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:22.001940 1168 tablet_bootstrap.cc:492] T 638fee757fc444f8a76b5189c103c40c P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:22.003456 893 tablet_service.cc:1505] Processing CreateTablet for tablet 531a74c1fef34b2dba6d4a732e13ce38 (DEFAULT_TABLE table=default.Table_9f357fec3afd4c58b2930fea17141463 [id=4f44988d148e427589d8ee1b342f0590]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.003831 1168 tablet_bootstrap.cc:654] T 638fee757fc444f8a76b5189c103c40c P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:22.003919 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 531a74c1fef34b2dba6d4a732e13ce38. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.004439 1160 tablet_bootstrap.cc:492] T 638fee757fc444f8a76b5189c103c40c P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:22.004614 1160 ts_tablet_manager.cc:1403] T 638fee757fc444f8a76b5189c103c40c P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.001s
I20251211 20:52:21.994719 1165 raft_consensus.cc:2243] T 00d68e696250404d880b8f97af217cbd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.004863 1165 raft_consensus.cc:2272] T 00d68e696250404d880b8f97af217cbd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.005060 1163 tablet_replica.cc:333] T 12313c06b33749589ceb08ade798fc59 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:22.005235 1163 raft_consensus.cc:2243] T 12313c06b33749589ceb08ade798fc59 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.001722 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 997fd184e900424bb5eabfde8c676d0e (table default.Table_8cbb799adf2747bba3e7edf97a7f1f59 [id=1473b367a32b480f842106bf0f4e41fb]) successfully deleted
I20251211 20:52:22.005458 1163 raft_consensus.cc:2272] T 12313c06b33749589ceb08ade798fc59 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.005399 1160 raft_consensus.cc:359] T 638fee757fc444f8a76b5189c103c40c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.005610 1166 tablet_bootstrap.cc:492] T 638fee757fc444f8a76b5189c103c40c P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:22.005617 1160 raft_consensus.cc:385] T 638fee757fc444f8a76b5189c103c40c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:22.005699 1160 raft_consensus.cc:740] T 638fee757fc444f8a76b5189c103c40c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:22.005978 1160 consensus_queue.cc:260] T 638fee757fc444f8a76b5189c103c40c P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.006196 753 tablet_service.cc:1552] Processing DeleteTablet for tablet d42e5cce248342f59d67f7c9aa6f66a2 with delete_type TABLET_DATA_DELETED (replaced by tablet 997fd184e900424bb5eabfde8c676d0e) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:22.007252 1166 tablet_bootstrap.cc:654] T 638fee757fc444f8a76b5189c103c40c P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:22.008005 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet b67de345cf44441abe9e1431b64c4ba4 (table default.Table_1b829e0ef7ef4b17be671db434f3858c [id=b8c401dfdb5f4f7584dca0989287ce31]) successfully deleted
W20251211 20:52:22.008284 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 49193f7bd8ac4f20a6ca980a9e1eb830 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 49193f7bd8ac4f20a6ca980a9e1eb830 already in progress: creating tablet
W20251211 20:52:22.008565 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 72d7d2f78ab64bbebb63239b9ecb366e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 72d7d2f78ab64bbebb63239b9ecb366e already in progress: creating tablet
I20251211 20:52:22.008728 1163 ts_tablet_manager.cc:1916] T 12313c06b33749589ceb08ade798fc59 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:22.008939 1160 ts_tablet_manager.cc:1434] T 638fee757fc444f8a76b5189c103c40c P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251211 20:52:22.011173 1160 tablet_bootstrap.cc:492] T 72d7d2f78ab64bbebb63239b9ecb366e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:22.012591 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 638fee757fc444f8a76b5189c103c40c with delete_type TABLET_DATA_DELETED (replaced by tablet 62fb046960024a2b9d70dc46ae52d53f) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:22.013495 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 531a74c1fef34b2dba6d4a732e13ce38 (DEFAULT_TABLE table=default.Table_9f357fec3afd4c58b2930fea17141463 [id=4f44988d148e427589d8ee1b342f0590]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:22.013617 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet dc7e8db4b51c415e86d8c24a76b8cb1e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dc7e8db4b51c415e86d8c24a76b8cb1e already in progress: creating tablet
I20251211 20:52:22.013931 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 531a74c1fef34b2dba6d4a732e13ce38. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.013976 1164 tablet_replica.cc:333] T 12313c06b33749589ceb08ade798fc59 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:22.014144 1164 raft_consensus.cc:2243] T 12313c06b33749589ceb08ade798fc59 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.014336 1164 raft_consensus.cc:2272] T 12313c06b33749589ceb08ade798fc59 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.014655 894 tablet_service.cc:1505] Processing CreateTablet for tablet 35a49c9f560c47049bdcbeddd55ccb28 (DEFAULT_TABLE table=default.Table_0c955f27bf344b1cbe27484262636529 [id=16a2a090abb14c1a996e695ea2798579]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.014667 1160 tablet_bootstrap.cc:654] T 72d7d2f78ab64bbebb63239b9ecb366e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:22.015046 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35a49c9f560c47049bdcbeddd55ccb28. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.016198 1164 ts_tablet_manager.cc:1916] T 12313c06b33749589ceb08ade798fc59 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:22.020638 753 tablet_service.cc:1505] Processing CreateTablet for tablet 531a74c1fef34b2dba6d4a732e13ce38 (DEFAULT_TABLE table=default.Table_9f357fec3afd4c58b2930fea17141463 [id=4f44988d148e427589d8ee1b342f0590]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.021103 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 531a74c1fef34b2dba6d4a732e13ce38. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.022037 1164 ts_tablet_manager.cc:1929] T 12313c06b33749589ceb08ade798fc59 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.022156 1164 log.cc:1199] T 12313c06b33749589ceb08ade798fc59 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/12313c06b33749589ceb08ade798fc59
I20251211 20:52:22.022475 1164 ts_tablet_manager.cc:1950] T 12313c06b33749589ceb08ade798fc59 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:22.024127 1166 tablet_bootstrap.cc:492] T 638fee757fc444f8a76b5189c103c40c P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:22.024299 1166 ts_tablet_manager.cc:1403] T 638fee757fc444f8a76b5189c103c40c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.007s
I20251211 20:52:22.025087 1166 raft_consensus.cc:359] T 638fee757fc444f8a76b5189c103c40c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.025332 1166 raft_consensus.cc:385] T 638fee757fc444f8a76b5189c103c40c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:22.025418 1166 raft_consensus.cc:740] T 638fee757fc444f8a76b5189c103c40c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:22.025472 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 35a49c9f560c47049bdcbeddd55ccb28 (DEFAULT_TABLE table=default.Table_0c955f27bf344b1cbe27484262636529 [id=16a2a090abb14c1a996e695ea2798579]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.025686 1166 consensus_queue.cc:260] T 638fee757fc444f8a76b5189c103c40c P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.025954 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35a49c9f560c47049bdcbeddd55ccb28. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.025933 894 tablet_service.cc:1505] Processing CreateTablet for tablet aa685d1b051a4c93a2d313429d914ead (DEFAULT_TABLE table=default.Table_ce67c2bcef6d4a6baecc11975d852ba8 [id=f81bfe3e109f48ce86e40e77dcbb30e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.026176 1166 ts_tablet_manager.cc:1434] T 638fee757fc444f8a76b5189c103c40c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:22.026301 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa685d1b051a4c93a2d313429d914ead. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.026532 1166 tablet_bootstrap.cc:492] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:22.027940 1022 tablet_service.cc:1505] Processing CreateTablet for tablet aa685d1b051a4c93a2d313429d914ead (DEFAULT_TABLE table=default.Table_ce67c2bcef6d4a6baecc11975d852ba8 [id=f81bfe3e109f48ce86e40e77dcbb30e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.028328 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa685d1b051a4c93a2d313429d914ead. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.030078 1160 tablet_bootstrap.cc:492] T 72d7d2f78ab64bbebb63239b9ecb366e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:22.030226 1160 ts_tablet_manager.cc:1403] T 72d7d2f78ab64bbebb63239b9ecb366e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.000s
I20251211 20:52:22.031061 1160 raft_consensus.cc:359] T 72d7d2f78ab64bbebb63239b9ecb366e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.031255 1160 raft_consensus.cc:385] T 72d7d2f78ab64bbebb63239b9ecb366e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:22.031337 1160 raft_consensus.cc:740] T 72d7d2f78ab64bbebb63239b9ecb366e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:22.031548 1160 consensus_queue.cc:260] T 72d7d2f78ab64bbebb63239b9ecb366e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.032723 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_3fca4e8f1b6a427d8d016e27f88ea1ce"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:22.032905 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_1b3f670b29d647a38ef00101ce8df343"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.033420 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3fca4e8f1b6a427d8d016e27f88ea1ce in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.033651 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_a0c412985adf4382a7ea28c77654fd6c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.034073 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a0c412985adf4382a7ea28c77654fd6c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.034650 1160 ts_tablet_manager.cc:1434] T 72d7d2f78ab64bbebb63239b9ecb366e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:22.035396 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_9ac73058eb564b63871197be450065f0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:22.035429 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_3030ffcddf0240ae81dfe06ce7d31fe5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.035944 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3030ffcddf0240ae81dfe06ce7d31fe5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.036197 1160 tablet_bootstrap.cc:492] T 274c10c2aa46479dab5edbb483e5b883 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:22.037410 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9ac73058eb564b63871197be450065f0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.037971 1160 tablet_bootstrap.cc:654] T 274c10c2aa46479dab5edbb483e5b883 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:22.033440 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_114a85ac0466481baebe3666b39843cd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.038620 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_114a85ac0466481baebe3666b39843cd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.039335 1164 tablet_replica.cc:333] T 638fee757fc444f8a76b5189c103c40c P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:22.039526 1164 raft_consensus.cc:2243] T 638fee757fc444f8a76b5189c103c40c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.039960 1182 raft_consensus.cc:493] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:22.040205 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 12313c06b33749589ceb08ade798fc59 (table default.Table_b37d05b3e8eb4425bb5cb72d46ce9e91 [id=4e18dbcc527d4b12ba6978ca125d5085]) successfully deleted
I20251211 20:52:22.040150 1182 raft_consensus.cc:515] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:22.040632 1164 raft_consensus.cc:2272] T 638fee757fc444f8a76b5189c103c40c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.041396 1164 ts_tablet_manager.cc:1916] T 638fee757fc444f8a76b5189c103c40c P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:22.042276 1182 leader_election.cc:290] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:22.043377 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8f2aa060e015406f805718a21dc1dd01" } new_table_name: "default.Table_2645aa306b5c44f6bad5e2e67a91ebec" modify_external_catalogs: true
I20251211 20:52:22.040477 1166 tablet_bootstrap.cc:654] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:22.043781 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 72d7d2f78ab64bbebb63239b9ecb366e with delete_type TABLET_DATA_DELETED (replaced by tablet 8a064789f89c424ca698e52333d73eea) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:22.046066 757 tablet_service.cc:1505] Processing CreateTablet for tablet aa685d1b051a4c93a2d313429d914ead (DEFAULT_TABLE table=default.Table_ce67c2bcef6d4a6baecc11975d852ba8 [id=f81bfe3e109f48ce86e40e77dcbb30e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.046538 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa685d1b051a4c93a2d313429d914ead. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.047118 891 tablet_service.cc:1505] Processing CreateTablet for tablet f37f598a720d4161b13c03425e020fa5 (DEFAULT_TABLE table=default.Table_9af853b8388442319882d871dc136367 [id=0563c542d899496e8586946ff3a66d96]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.047531 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f37f598a720d4161b13c03425e020fa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.047515 754 tablet_service.cc:1505] Processing CreateTablet for tablet 35a49c9f560c47049bdcbeddd55ccb28 (DEFAULT_TABLE table=default.Table_0c955f27bf344b1cbe27484262636529 [id=16a2a090abb14c1a996e695ea2798579]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.050061 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35a49c9f560c47049bdcbeddd55ccb28. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.054633 893 tablet_service.cc:1505] Processing CreateTablet for tablet d703a5a42b2f47d0a7ffe1bb534a1b05 (DEFAULT_TABLE table=default.Table_512673b662ef4de0a2c2328d3310f4bf [id=e5caa22bce84422290707816a93bd3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.055104 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d703a5a42b2f47d0a7ffe1bb534a1b05. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.057333 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 638fee757fc444f8a76b5189c103c40c with delete_type TABLET_DATA_DELETED (replaced by tablet 62fb046960024a2b9d70dc46ae52d53f) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:22.057827 1024 tablet_service.cc:1505] Processing CreateTablet for tablet d703a5a42b2f47d0a7ffe1bb534a1b05 (DEFAULT_TABLE table=default.Table_512673b662ef4de0a2c2328d3310f4bf [id=e5caa22bce84422290707816a93bd3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.058261 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d703a5a42b2f47d0a7ffe1bb534a1b05. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.058102 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_104f49b10d1d45749449d0ff3009e1d9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.058821 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_104f49b10d1d45749449d0ff3009e1d9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.059281 1164 ts_tablet_manager.cc:1929] T 638fee757fc444f8a76b5189c103c40c P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.059428 1164 log.cc:1199] T 638fee757fc444f8a76b5189c103c40c P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/638fee757fc444f8a76b5189c103c40c
I20251211 20:52:22.059818 1164 ts_tablet_manager.cc:1950] T 638fee757fc444f8a76b5189c103c40c P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:22.058102 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b5a3abc502a64d6083d78344400b5eeb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.060492 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b5a3abc502a64d6083d78344400b5eeb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.061697 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 638fee757fc444f8a76b5189c103c40c (table default.Table_fbca1745cad8402fba1c38f2c3b0a9f4 [id=14a0dbb8486246b09f3e06fc2120cd28]) successfully deleted
I20251211 20:52:22.062114 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_a11e64052cf34ba6a6a31f4edb417c19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.062611 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a11e64052cf34ba6a6a31f4edb417c19 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.064018 1021 tablet_service.cc:1505] Processing CreateTablet for tablet f37f598a720d4161b13c03425e020fa5 (DEFAULT_TABLE table=default.Table_9af853b8388442319882d871dc136367 [id=0563c542d899496e8586946ff3a66d96]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.064483 1164 tablet_replica.cc:333] T 72d7d2f78ab64bbebb63239b9ecb366e P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:22.064585 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f37f598a720d4161b13c03425e020fa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.064678 1164 raft_consensus.cc:2243] T 72d7d2f78ab64bbebb63239b9ecb366e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.064889 1164 raft_consensus.cc:2272] T 72d7d2f78ab64bbebb63239b9ecb366e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.067231 1164 ts_tablet_manager.cc:1916] T 72d7d2f78ab64bbebb63239b9ecb366e P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:22.072461 752 tablet_service.cc:1505] Processing CreateTablet for tablet d703a5a42b2f47d0a7ffe1bb534a1b05 (DEFAULT_TABLE table=default.Table_512673b662ef4de0a2c2328d3310f4bf [id=e5caa22bce84422290707816a93bd3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.073033 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d703a5a42b2f47d0a7ffe1bb534a1b05. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.073323 1165 ts_tablet_manager.cc:1916] T 00d68e696250404d880b8f97af217cbd P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:22.077021 756 tablet_service.cc:1505] Processing CreateTablet for tablet f37f598a720d4161b13c03425e020fa5 (DEFAULT_TABLE table=default.Table_9af853b8388442319882d871dc136367 [id=0563c542d899496e8586946ff3a66d96]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.077521 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f37f598a720d4161b13c03425e020fa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.084388 1164 ts_tablet_manager.cc:1929] T 72d7d2f78ab64bbebb63239b9ecb366e P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.084540 1164 log.cc:1199] T 72d7d2f78ab64bbebb63239b9ecb366e P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/72d7d2f78ab64bbebb63239b9ecb366e
I20251211 20:52:22.084913 1164 ts_tablet_manager.cc:1950] T 72d7d2f78ab64bbebb63239b9ecb366e P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:22.086721 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 72d7d2f78ab64bbebb63239b9ecb366e (table default.Table_f439572c5b1d4997af54e0ab3c036023 [id=cda4c0ad8a0347be88a0518491e6af49]) successfully deleted
W20251211 20:52:22.033447 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1b3f670b29d647a38ef00101ce8df343 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:22.098237 710 leader_election.cc:343] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: f5467114696046fbb2f311856b7a4d73
W20251211 20:52:22.106295 592 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e69d53ac90a7463aa36ff15dadad7e0b has no consensus state
I20251211 20:52:22.108506 891 tablet_service.cc:1505] Processing CreateTablet for tablet b815bca893a8446e92d48973d9a0408e (DEFAULT_TABLE table=default.Table_b411ab6a2d934ad8aa183b348dc24d85 [id=fd269c697f9d4c5eb4a0d646f459c954]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.109022 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b815bca893a8446e92d48973d9a0408e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.118919 890 tablet_service.cc:1505] Processing CreateTablet for tablet 01ada50100bc48e39d1bc67082da750e (DEFAULT_TABLE table=default.Table_5fa0a824d9234eaebcbbd089e47d3f75 [id=a24d8c58b9d84cdea2b278ae9c6e0ba5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.119428 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01ada50100bc48e39d1bc67082da750e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.129550 1160 tablet_bootstrap.cc:492] T 274c10c2aa46479dab5edbb483e5b883 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:22.129767 1160 ts_tablet_manager.cc:1403] T 274c10c2aa46479dab5edbb483e5b883 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.094s user 0.003s sys 0.032s
I20251211 20:52:22.130582 1160 raft_consensus.cc:359] T 274c10c2aa46479dab5edbb483e5b883 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.130812 1160 raft_consensus.cc:385] T 274c10c2aa46479dab5edbb483e5b883 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:22.130887 1160 raft_consensus.cc:740] T 274c10c2aa46479dab5edbb483e5b883 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:22.131093 1160 consensus_queue.cc:260] T 274c10c2aa46479dab5edbb483e5b883 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.131381 1168 tablet_bootstrap.cc:492] T 638fee757fc444f8a76b5189c103c40c P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:22.131572 1168 ts_tablet_manager.cc:1403] T 638fee757fc444f8a76b5189c103c40c P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.130s user 0.038s sys 0.041s
I20251211 20:52:22.132352 1168 raft_consensus.cc:359] T 638fee757fc444f8a76b5189c103c40c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.132532 1168 raft_consensus.cc:385] T 638fee757fc444f8a76b5189c103c40c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:22.132594 1168 raft_consensus.cc:740] T 638fee757fc444f8a76b5189c103c40c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:22.132822 1168 consensus_queue.cc:260] T 638fee757fc444f8a76b5189c103c40c P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.133285 1168 ts_tablet_manager.cc:1434] T 638fee757fc444f8a76b5189c103c40c P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:22.133663 1160 ts_tablet_manager.cc:1434] T 274c10c2aa46479dab5edbb483e5b883 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251211 20:52:22.134897 1168 tablet_bootstrap.cc:492] T 12313c06b33749589ceb08ade798fc59 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:22.135090 1160 tablet_bootstrap.cc:492] T 49193f7bd8ac4f20a6ca980a9e1eb830 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:22.136924 1168 tablet_bootstrap.cc:654] T 12313c06b33749589ceb08ade798fc59 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:22.137038 1160 tablet_bootstrap.cc:654] T 49193f7bd8ac4f20a6ca980a9e1eb830 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:22.137609 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 01ada50100bc48e39d1bc67082da750e (DEFAULT_TABLE table=default.Table_5fa0a824d9234eaebcbbd089e47d3f75 [id=a24d8c58b9d84cdea2b278ae9c6e0ba5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.137681 755 tablet_service.cc:1505] Processing CreateTablet for tablet bc51bc0821a8484caee39c534eb4e6af (DEFAULT_TABLE table=default.Table_cbe68d84efef4745822f6aef5a723f11 [id=f9c1230a7d71435cac705d4675a9908d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.138064 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01ada50100bc48e39d1bc67082da750e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.138108 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc51bc0821a8484caee39c534eb4e6af. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.138978 1160 tablet_bootstrap.cc:492] T 49193f7bd8ac4f20a6ca980a9e1eb830 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:22.139120 1160 ts_tablet_manager.cc:1403] T 49193f7bd8ac4f20a6ca980a9e1eb830 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20251211 20:52:22.140024 1160 raft_consensus.cc:359] T 49193f7bd8ac4f20a6ca980a9e1eb830 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.140214 1160 raft_consensus.cc:385] T 49193f7bd8ac4f20a6ca980a9e1eb830 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:22.140306 1160 raft_consensus.cc:740] T 49193f7bd8ac4f20a6ca980a9e1eb830 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:22.140501 1160 consensus_queue.cc:260] T 49193f7bd8ac4f20a6ca980a9e1eb830 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.142786 1160 ts_tablet_manager.cc:1434] T 49193f7bd8ac4f20a6ca980a9e1eb830 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:22.143501 1166 tablet_bootstrap.cc:492] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:22.143738 1166 ts_tablet_manager.cc:1403] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.117s user 0.020s sys 0.016s
I20251211 20:52:22.144279 1174 raft_consensus.cc:493] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:22.144477 1174 raft_consensus.cc:515] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:22.144737 1166 raft_consensus.cc:359] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.144915 1166 raft_consensus.cc:385] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:22.145016 1166 raft_consensus.cc:740] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:22.145274 1166 consensus_queue.cc:260] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.146068 756 tablet_service.cc:1505] Processing CreateTablet for tablet b815bca893a8446e92d48973d9a0408e (DEFAULT_TABLE table=default.Table_b411ab6a2d934ad8aa183b348dc24d85 [id=fd269c697f9d4c5eb4a0d646f459c954]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.146461 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b815bca893a8446e92d48973d9a0408e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.146423 1174 leader_election.cc:290] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:22.147116 1160 tablet_bootstrap.cc:492] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:22.149055 891 tablet_service.cc:1505] Processing CreateTablet for tablet bc51bc0821a8484caee39c534eb4e6af (DEFAULT_TABLE table=default.Table_cbe68d84efef4745822f6aef5a723f11 [id=f9c1230a7d71435cac705d4675a9908d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.157066 1166 ts_tablet_manager.cc:1434] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.013s user 0.005s sys 0.002s
I20251211 20:52:22.157590 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc51bc0821a8484caee39c534eb4e6af. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.151552 1160 tablet_bootstrap.cc:654] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:22.157420 1166 tablet_bootstrap.cc:492] T 49193f7bd8ac4f20a6ca980a9e1eb830 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:22.160599 1166 tablet_bootstrap.cc:654] T 49193f7bd8ac4f20a6ca980a9e1eb830 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:22.160987 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9f357fec3afd4c58b2930fea17141463" } modify_external_catalogs: true
I20251211 20:52:22.161191 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9f357fec3afd4c58b2930fea17141463" } modify_external_catalogs: true
I20251211 20:52:22.161996 588 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_1b3f670b29d647a38ef00101ce8df343" } new_table_name: "default.Table_e5a162d6463e4ff5a4edaaae5a4d03cb" modify_external_catalogs: true
I20251211 20:52:22.163652 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ce67c2bcef6d4a6baecc11975d852ba8" } modify_external_catalogs: true
I20251211 20:52:22.163762 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_3030ffcddf0240ae81dfe06ce7d31fe5" } new_table_name: "default.Table_e6a96166bf33442989e15129d4d44631" modify_external_catalogs: true
I20251211 20:52:22.163839 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ce67c2bcef6d4a6baecc11975d852ba8" } modify_external_catalogs: true
I20251211 20:52:22.165127 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_512673b662ef4de0a2c2328d3310f4bf" } new_table_name: "default.Table_eac21915c0ae4694a106ce80dc011bbd" modify_external_catalogs: true
I20251211 20:52:22.165200 590 master_service.cc:954] ReplaceTablet: received request to replace tablet d0df24ce65234eeba97f9fdd351b58ad from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:22.165958 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b411ab6a2d934ad8aa183b348dc24d85" } modify_external_catalogs: true
I20251211 20:52:22.166132 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b411ab6a2d934ad8aa183b348dc24d85" } modify_external_catalogs: true
I20251211 20:52:22.166823 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_40256d3973c74566a09b1d235062d284" } modify_external_catalogs: true
I20251211 20:52:22.166991 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_40256d3973c74566a09b1d235062d284" } modify_external_catalogs: true
I20251211 20:52:22.167461 596 master_service.cc:954] ReplaceTablet: received request to replace tablet 274c10c2aa46479dab5edbb483e5b883 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:22.167436 894 tablet_service.cc:1505] Processing CreateTablet for tablet edc4300dc5a84173aa95b587d77fc95c (DEFAULT_TABLE table=default.Table_4f9b0d5d6ec647df87f89a25344ca9f8 [id=32fff752ffaa46e6af5221a3ecf2adc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.167881 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edc4300dc5a84173aa95b587d77fc95c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.168053 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_114a85ac0466481baebe3666b39843cd" } new_table_name: "default.Table_a0c567f392784be39ceb0176cf58cff8" modify_external_catalogs: true
I20251211 20:52:22.169154 597 master_service.cc:954] ReplaceTablet: received request to replace tablet 8e9eded009e44c1a9544d4471c04c5c8 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:22.169607 1160 tablet_bootstrap.cc:492] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:22.169797 1160 ts_tablet_manager.cc:1403] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.002s
I20251211 20:52:22.170646 1160 raft_consensus.cc:359] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.170861 1160 raft_consensus.cc:385] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:22.170944 1160 raft_consensus.cc:740] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:22.171191 1160 consensus_queue.cc:260] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.175805 1166 tablet_bootstrap.cc:492] T 49193f7bd8ac4f20a6ca980a9e1eb830 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:22.176060 1166 ts_tablet_manager.cc:1403] T 49193f7bd8ac4f20a6ca980a9e1eb830 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.002s
I20251211 20:52:22.176919 1166 raft_consensus.cc:359] T 49193f7bd8ac4f20a6ca980a9e1eb830 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.177235 1166 raft_consensus.cc:385] T 49193f7bd8ac4f20a6ca980a9e1eb830 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:22.177354 1166 raft_consensus.cc:740] T 49193f7bd8ac4f20a6ca980a9e1eb830 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:22.177637 1166 consensus_queue.cc:260] T 49193f7bd8ac4f20a6ca980a9e1eb830 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.178151 1166 ts_tablet_manager.cc:1434] T 49193f7bd8ac4f20a6ca980a9e1eb830 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:22.178609 1166 tablet_bootstrap.cc:492] T ae3be9d91e5d4551bc706b2043a9edf5 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:22.178601 1022 tablet_service.cc:1505] Processing CreateTablet for tablet bc51bc0821a8484caee39c534eb4e6af (DEFAULT_TABLE table=default.Table_cbe68d84efef4745822f6aef5a723f11 [id=f9c1230a7d71435cac705d4675a9908d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.180091 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc51bc0821a8484caee39c534eb4e6af. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.180341 1166 tablet_bootstrap.cc:654] T ae3be9d91e5d4551bc706b2043a9edf5 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:22.188443 1021 tablet_service.cc:1505] Processing CreateTablet for tablet b815bca893a8446e92d48973d9a0408e (DEFAULT_TABLE table=default.Table_b411ab6a2d934ad8aa183b348dc24d85 [id=fd269c697f9d4c5eb4a0d646f459c954]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.188974 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b815bca893a8446e92d48973d9a0408e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.198778 1163 ts_tablet_manager.cc:1929] T 12313c06b33749589ceb08ade798fc59 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.198961 1163 log.cc:1199] T 12313c06b33749589ceb08ade798fc59 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/12313c06b33749589ceb08ade798fc59
I20251211 20:52:22.199370 1163 ts_tablet_manager.cc:1950] T 12313c06b33749589ceb08ade798fc59 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:22.203085 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:22.203647 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 7f086c7c56fe4abb8df7d3c0978cce6e (DEFAULT_TABLE table=default.Table_1b3f670b29d647a38ef00101ce8df343 [id=872ff521abd740ee924f22e9c0ff8585]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.204317 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f086c7c56fe4abb8df7d3c0978cce6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.205406 1163 tablet_replica.cc:333] T 49193f7bd8ac4f20a6ca980a9e1eb830 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:22.205646 1163 raft_consensus.cc:2243] T 49193f7bd8ac4f20a6ca980a9e1eb830 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.205853 1163 raft_consensus.cc:2272] T 49193f7bd8ac4f20a6ca980a9e1eb830 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.207978 1163 ts_tablet_manager.cc:1916] T 49193f7bd8ac4f20a6ca980a9e1eb830 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:22.212514 1020 tablet_service.cc:1505] Processing CreateTablet for tablet acdb7dddf3e74ce1bc92f232b6180cca (DEFAULT_TABLE table=default.Table_104f49b10d1d45749449d0ff3009e1d9 [id=b68aac5147654707b677d020a2cc37cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.213008 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acdb7dddf3e74ce1bc92f232b6180cca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.220147 1022 tablet_service.cc:1505] Processing CreateTablet for tablet edc4300dc5a84173aa95b587d77fc95c (DEFAULT_TABLE table=default.Table_4f9b0d5d6ec647df87f89a25344ca9f8 [id=32fff752ffaa46e6af5221a3ecf2adc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.221298 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 297dc26ade384213b364a876c54bdd14 (DEFAULT_TABLE table=default.Table_114a85ac0466481baebe3666b39843cd [id=cb97ccf262fd4c41a180267fd0678cec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.221539 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edc4300dc5a84173aa95b587d77fc95c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.224483 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 297dc26ade384213b364a876c54bdd14. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.226387 1163 ts_tablet_manager.cc:1929] T 49193f7bd8ac4f20a6ca980a9e1eb830 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.226552 1163 log.cc:1199] T 49193f7bd8ac4f20a6ca980a9e1eb830 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/49193f7bd8ac4f20a6ca980a9e1eb830
I20251211 20:52:22.226961 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 8a064789f89c424ca698e52333d73eea (DEFAULT_TABLE table=default.Table_f439572c5b1d4997af54e0ab3c036023 [id=cda4c0ad8a0347be88a0518491e6af49]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.227355 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a064789f89c424ca698e52333d73eea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.228032 1029 tablet_service.cc:1505] Processing CreateTablet for tablet af3ca9ac988e458cb7068c10c006d587 (DEFAULT_TABLE table=default.Table_a11e64052cf34ba6a6a31f4edb417c19 [id=0afc1e783888443eb3f3e7848a04c207]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.228369 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af3ca9ac988e458cb7068c10c006d587. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.232069 1024 tablet_service.cc:1505] Processing CreateTablet for tablet 049cdecf12304211adbc1c1008dba989 (DEFAULT_TABLE table=default.Table_3030ffcddf0240ae81dfe06ce7d31fe5 [id=34edf86667d448c0b87e55627c67a704]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.232497 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 049cdecf12304211adbc1c1008dba989. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.235371 1029 tablet_service.cc:1505] Processing CreateTablet for tablet e69d53ac90a7463aa36ff15dadad7e0b (DEFAULT_TABLE table=default.Table_2645aa306b5c44f6bad5e2e67a91ebec [id=0ec7cf10eb8a47aca499b141913b38cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.235785 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e69d53ac90a7463aa36ff15dadad7e0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.236054 1163 ts_tablet_manager.cc:1950] T 49193f7bd8ac4f20a6ca980a9e1eb830 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:22.239179 1163 tablet_replica.cc:333] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:22.239437 1163 raft_consensus.cc:2243] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.239691 1163 raft_consensus.cc:2272] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.241438 1024 tablet_service.cc:1505] Processing CreateTablet for tablet 73aa2a3fb2f44bbca870fb7f4cf6087c (DEFAULT_TABLE table=default.Table_3fca4e8f1b6a427d8d016e27f88ea1ce [id=4f93b23edd4548aeb6bcb8f92878907d]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:22.241652 582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.61:40749, user_credentials={real_user=slave}} blocked reactor thread for 43794.6us
I20251211 20:52:22.241786 1163 ts_tablet_manager.cc:1916] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:22.242074 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73aa2a3fb2f44bbca870fb7f4cf6087c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.226948 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 5ed0a0141051462eb45ca3a66d41d009 (DEFAULT_TABLE table=default.Table_40256d3973c74566a09b1d235062d284 [id=1f4650c549d94e2c9ef753ff96670aa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.244786 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ed0a0141051462eb45ca3a66d41d009. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.248034 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 12313c06b33749589ceb08ade798fc59 (table default.Table_b37d05b3e8eb4425bb5cb72d46ce9e91 [id=4e18dbcc527d4b12ba6978ca125d5085]) successfully deleted
I20251211 20:52:22.248413 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 353da9106578450987a4f6544df8d7e6 (DEFAULT_TABLE table=default.Table_a0c412985adf4382a7ea28c77654fd6c [id=4a74b9a60cbb4198b55cee630b2f4bef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.248562 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 49193f7bd8ac4f20a6ca980a9e1eb830 (table default.Table_40256d3973c74566a09b1d235062d284 [id=1f4650c549d94e2c9ef753ff96670aa6]) successfully deleted
W20251211 20:52:22.248863 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 72d7d2f78ab64bbebb63239b9ecb366e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 72d7d2f78ab64bbebb63239b9ecb366e already in progress: creating tablet
I20251211 20:52:22.249053 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 353da9106578450987a4f6544df8d7e6. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:22.249102 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet dc7e8db4b51c415e86d8c24a76b8cb1e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dc7e8db4b51c415e86d8c24a76b8cb1e already in progress: creating tablet
I20251211 20:52:22.249826 1163 ts_tablet_manager.cc:1929] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.249951 1163 log.cc:1199] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/b67de345cf44441abe9e1431b64c4ba4
I20251211 20:52:22.250314 1163 ts_tablet_manager.cc:1950] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:22.250563 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 034f47d8564f4094a9d76aee2fea3119 (DEFAULT_TABLE table=default.Table_b5a3abc502a64d6083d78344400b5eeb [id=c58ec33f45c348329be4acee3d5f4f6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.250880 1163 tablet_replica.cc:333] T 638fee757fc444f8a76b5189c103c40c P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:22.250906 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 034f47d8564f4094a9d76aee2fea3119. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.251092 1163 raft_consensus.cc:2243] T 638fee757fc444f8a76b5189c103c40c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.251247 1163 raft_consensus.cc:2272] T 638fee757fc444f8a76b5189c103c40c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.256407 1028 tablet_service.cc:1505] Processing CreateTablet for tablet f4db14d082d34aef857e106992bb8950 (DEFAULT_TABLE table=default.Table_9ac73058eb564b63871197be450065f0 [id=ffb336d92b05442b80d70cd1f419431c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.256803 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4db14d082d34aef857e106992bb8950. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:22.256954 588 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7f086c7c56fe4abb8df7d3c0978cce6e consensus state has no leader
I20251211 20:52:22.257340 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4f9b0d5d6ec647df87f89a25344ca9f8" } modify_external_catalogs: true
I20251211 20:52:22.257508 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4f9b0d5d6ec647df87f89a25344ca9f8" } modify_external_catalogs: true
I20251211 20:52:22.258711 1163 ts_tablet_manager.cc:1916] T 638fee757fc444f8a76b5189c103c40c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:22.258854 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 049cdecf12304211adbc1c1008dba989 consensus state has no leader
I20251211 20:52:22.259284 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_21f7b3d63a0146a7ab330b1469611bfe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.259835 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_21f7b3d63a0146a7ab330b1469611bfe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:22.261029 591 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d703a5a42b2f47d0a7ffe1bb534a1b05 consensus state has no leader
W20251211 20:52:22.261399 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 297dc26ade384213b364a876c54bdd14 consensus state has no leader
I20251211 20:52:22.261889 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet aa685d1b051a4c93a2d313429d914ead
I20251211 20:52:22.262590 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 8e9eded009e44c1a9544d4471c04c5c8
I20251211 20:52:22.262961 597 catalog_manager.cc:6724] ReplaceTablet: tablet 8e9eded009e44c1a9544d4471c04c5c8 deleted and replaced by tablet 97a519a8ca7c4f5380a230a70ee84052
I20251211 20:52:22.263379 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 5ed0a0141051462eb45ca3a66d41d009
I20251211 20:52:22.264751 1163 ts_tablet_manager.cc:1929] T 638fee757fc444f8a76b5189c103c40c P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.264787 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 531a74c1fef34b2dba6d4a732e13ce38
I20251211 20:52:22.264909 1163 log.cc:1199] T 638fee757fc444f8a76b5189c103c40c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/638fee757fc444f8a76b5189c103c40c
I20251211 20:52:22.265275 1163 ts_tablet_manager.cc:1950] T 638fee757fc444f8a76b5189c103c40c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:22.266155 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet b815bca893a8446e92d48973d9a0408e
I20251211 20:52:22.266202 1163 tablet_replica.cc:333] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:22.266393 1163 raft_consensus.cc:2243] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.266571 1163 raft_consensus.cc:2272] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.276098 1163 ts_tablet_manager.cc:1916] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:22.282389 1163 ts_tablet_manager.cc:1929] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.282542 1163 log.cc:1199] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/cd13d4e7cf2c4abdbedbb5bbac5e05ce
I20251211 20:52:22.282982 1163 ts_tablet_manager.cc:1950] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:22.284791 1160 ts_tablet_manager.cc:1434] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:22.285416 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 274c10c2aa46479dab5edbb483e5b883 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:22.285698 1164 tablet_replica.cc:333] T 274c10c2aa46479dab5edbb483e5b883 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:22.285890 1164 raft_consensus.cc:2243] T 274c10c2aa46479dab5edbb483e5b883 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.286057 1164 raft_consensus.cc:2272] T 274c10c2aa46479dab5edbb483e5b883 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.288201 1164 ts_tablet_manager.cc:1916] T 274c10c2aa46479dab5edbb483e5b883 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:22.290599 583 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 7f086c7c56fe4abb8df7d3c0978cce6e (table default.Table_e5a162d6463e4ff5a4edaaae5a4d03cb [id=872ff521abd740ee924f22e9c0ff8585], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 7f086c7c56fe4abb8df7d3c0978cce6e consensus state has no leader
I20251211 20:52:22.290916 1160 tablet_bootstrap.cc:492] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:22.291087 893 tablet_service.cc:1505] Processing CreateTablet for tablet 7f086c7c56fe4abb8df7d3c0978cce6e (DEFAULT_TABLE table=default.Table_1b3f670b29d647a38ef00101ce8df343 [id=872ff521abd740ee924f22e9c0ff8585]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.292096 890 tablet_service.cc:1505] Processing CreateTablet for tablet acdb7dddf3e74ce1bc92f232b6180cca (DEFAULT_TABLE table=default.Table_104f49b10d1d45749449d0ff3009e1d9 [id=b68aac5147654707b677d020a2cc37cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.292557 891 tablet_service.cc:1505] Processing CreateTablet for tablet 297dc26ade384213b364a876c54bdd14 (DEFAULT_TABLE table=default.Table_114a85ac0466481baebe3666b39843cd [id=cb97ccf262fd4c41a180267fd0678cec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.293288 1166 tablet_bootstrap.cc:492] T ae3be9d91e5d4551bc706b2043a9edf5 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:22.293453 1166 ts_tablet_manager.cc:1403] T ae3be9d91e5d4551bc706b2043a9edf5 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.115s user 0.042s sys 0.044s
I20251211 20:52:22.294211 1166 raft_consensus.cc:359] T ae3be9d91e5d4551bc706b2043a9edf5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.294420 1166 raft_consensus.cc:385] T ae3be9d91e5d4551bc706b2043a9edf5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:22.294498 1166 raft_consensus.cc:740] T ae3be9d91e5d4551bc706b2043a9edf5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:22.294706 1166 consensus_queue.cc:260] T ae3be9d91e5d4551bc706b2043a9edf5 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.295151 1166 ts_tablet_manager.cc:1434] T ae3be9d91e5d4551bc706b2043a9edf5 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:22.297089 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_bcccedcd97444712a0f74e76db8cd74a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.297755 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bcccedcd97444712a0f74e76db8cd74a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.298300 1166 tablet_bootstrap.cc:492] T 274c10c2aa46479dab5edbb483e5b883 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:22.301275 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet b67de345cf44441abe9e1431b64c4ba4 (table default.Table_1b829e0ef7ef4b17be671db434f3858c [id=b8c401dfdb5f4f7584dca0989287ce31]) successfully deleted
I20251211 20:52:22.301363 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f086c7c56fe4abb8df7d3c0978cce6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.301800 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 638fee757fc444f8a76b5189c103c40c (table default.Table_fbca1745cad8402fba1c38f2c3b0a9f4 [id=14a0dbb8486246b09f3e06fc2120cd28]) successfully deleted
I20251211 20:52:22.302289 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce (table default.Table_7ce0e2ed6ea4463cac10ba89d3718c5c [id=0e7104bb15484108b86dacf1f1f2c359]) successfully deleted
I20251211 20:52:22.303112 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 8e9eded009e44c1a9544d4471c04c5c8 with delete_type TABLET_DATA_DELETED (replaced by tablet 97a519a8ca7c4f5380a230a70ee84052) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:22.303390 1028 tablet_service.cc:1552] Processing DeleteTablet for tablet 5ed0a0141051462eb45ca3a66d41d009 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:22.303504 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 49193f7bd8ac4f20a6ca980a9e1eb830 with delete_type TABLET_DATA_DELETED (replaced by tablet 5ed0a0141051462eb45ca3a66d41d009) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:22.304320 892 tablet_service.cc:1505] Processing CreateTablet for tablet 049cdecf12304211adbc1c1008dba989 (DEFAULT_TABLE table=default.Table_3030ffcddf0240ae81dfe06ce7d31fe5 [id=34edf86667d448c0b87e55627c67a704]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.304765 1166 tablet_bootstrap.cc:654] T 274c10c2aa46479dab5edbb483e5b883 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:22.303611 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet b815bca893a8446e92d48973d9a0408e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:22.303772 1019 tablet_service.cc:1552] Processing DeleteTablet for tablet 531a74c1fef34b2dba6d4a732e13ce38 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:22.305338 889 tablet_service.cc:1505] Processing CreateTablet for tablet 5ed0a0141051462eb45ca3a66d41d009 (DEFAULT_TABLE table=default.Table_40256d3973c74566a09b1d235062d284 [id=1f4650c549d94e2c9ef753ff96670aa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.305852 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet edc4300dc5a84173aa95b587d77fc95c
I20251211 20:52:22.306861 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acdb7dddf3e74ce1bc92f232b6180cca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.307646 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 297dc26ade384213b364a876c54bdd14. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.308208 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 049cdecf12304211adbc1c1008dba989. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.308418 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ed0a0141051462eb45ca3a66d41d009. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.308753 881 tablet_service.cc:1552] Processing DeleteTablet for tablet aa685d1b051a4c93a2d313429d914ead with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:22.310015 882 tablet_service.cc:1505] Processing CreateTablet for tablet f4db14d082d34aef857e106992bb8950 (DEFAULT_TABLE table=default.Table_9ac73058eb564b63871197be450065f0 [id=ffb336d92b05442b80d70cd1f419431c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.307749 888 tablet_service.cc:1505] Processing CreateTablet for tablet 8a064789f89c424ca698e52333d73eea (DEFAULT_TABLE table=default.Table_f439572c5b1d4997af54e0ab3c036023 [id=cda4c0ad8a0347be88a0518491e6af49]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.315708 1182 raft_consensus.cc:493] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:22.315795 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet aa685d1b051a4c93a2d313429d914ead with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:22.317101 749 tablet_service.cc:1505] Processing CreateTablet for tablet 297dc26ade384213b364a876c54bdd14 (DEFAULT_TABLE table=default.Table_114a85ac0466481baebe3666b39843cd [id=cb97ccf262fd4c41a180267fd0678cec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.317094 887 tablet_service.cc:1505] Processing CreateTablet for tablet af3ca9ac988e458cb7068c10c006d587 (DEFAULT_TABLE table=default.Table_a11e64052cf34ba6a6a31f4edb417c19 [id=0afc1e783888443eb3f3e7848a04c207]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.317273 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet edc4300dc5a84173aa95b587d77fc95c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:22.317330 883 tablet_service.cc:1505] Processing CreateTablet for tablet 034f47d8564f4094a9d76aee2fea3119 (DEFAULT_TABLE table=default.Table_b5a3abc502a64d6083d78344400b5eeb [id=c58ec33f45c348329be4acee3d5f4f6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.317538 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 297dc26ade384213b364a876c54bdd14. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.317605 589 master_service.cc:954] ReplaceTablet: received request to replace tablet 62fb046960024a2b9d70dc46ae52d53f from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:22.318025 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a064789f89c424ca698e52333d73eea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.318222 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_a7c6475971744685ace8304ecf702b38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.318791 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a7c6475971744685ace8304ecf702b38 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.320333 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 638fee757fc444f8a76b5189c103c40c with delete_type TABLET_DATA_DELETED (replaced by tablet 62fb046960024a2b9d70dc46ae52d53f) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:22.320955 877 tablet_service.cc:1552] Processing DeleteTablet for tablet b815bca893a8446e92d48973d9a0408e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:22.321192 878 tablet_service.cc:1552] Processing DeleteTablet for tablet 531a74c1fef34b2dba6d4a732e13ce38 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:22.321702 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_5e9e111d44b04f94b728307574520f89"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:22.321964 753 tablet_service.cc:1505] Processing CreateTablet for tablet edc4300dc5a84173aa95b587d77fc95c (DEFAULT_TABLE table=default.Table_4f9b0d5d6ec647df87f89a25344ca9f8 [id=32fff752ffaa46e6af5221a3ecf2adc9]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:22.322255 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5e9e111d44b04f94b728307574520f89 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.323760 885 tablet_service.cc:1505] Processing CreateTablet for tablet 73aa2a3fb2f44bbca870fb7f4cf6087c (DEFAULT_TABLE table=default.Table_3fca4e8f1b6a427d8d016e27f88ea1ce [id=4f93b23edd4548aeb6bcb8f92878907d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.324002 884 tablet_service.cc:1505] Processing CreateTablet for tablet 353da9106578450987a4f6544df8d7e6 (DEFAULT_TABLE table=default.Table_a0c412985adf4382a7ea28c77654fd6c [id=4a74b9a60cbb4198b55cee630b2f4bef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.324191 886 tablet_service.cc:1505] Processing CreateTablet for tablet e69d53ac90a7463aa36ff15dadad7e0b (DEFAULT_TABLE table=default.Table_2645aa306b5c44f6bad5e2e67a91ebec [id=0ec7cf10eb8a47aca499b141913b38cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.325464 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_7747db5b5c664662bfe1e87d1e206c91"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:22.325721 881 tablet_service.cc:1552] Processing DeleteTablet for tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:22.326100 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7747db5b5c664662bfe1e87d1e206c91 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.326203 751 tablet_service.cc:1505] Processing CreateTablet for tablet 8a064789f89c424ca698e52333d73eea (DEFAULT_TABLE table=default.Table_f439572c5b1d4997af54e0ab3c036023 [id=cda4c0ad8a0347be88a0518491e6af49]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.326553 1182 raft_consensus.cc:515] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:22.327078 879 tablet_service.cc:1552] Processing DeleteTablet for tablet 5ed0a0141051462eb45ca3a66d41d009 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:22.327886 757 tablet_service.cc:1505] Processing CreateTablet for tablet 01ada50100bc48e39d1bc67082da750e (DEFAULT_TABLE table=default.Table_5fa0a824d9234eaebcbbd089e47d3f75 [id=a24d8c58b9d84cdea2b278ae9c6e0ba5]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:22.328991 710 leader_election.cc:343] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 03ebcee3069f44f99761cc13e34512ed
W20251211 20:52:22.329907 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 8e9eded009e44c1a9544d4471c04c5c8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e9eded009e44c1a9544d4471c04c5c8 already in progress: creating tablet
W20251211 20:52:22.330158 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 5ed0a0141051462eb45ca3a66d41d009 because the tablet was not found. No further retry: Not found: Tablet not found: 5ed0a0141051462eb45ca3a66d41d009
I20251211 20:52:22.330237 1182 leader_election.cc:290] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:22.330367 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 49193f7bd8ac4f20a6ca980a9e1eb830 because the tablet was not found. No further retry: Not found: Tablet not found: 49193f7bd8ac4f20a6ca980a9e1eb830
W20251211 20:52:22.330556 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 531a74c1fef34b2dba6d4a732e13ce38 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 531a74c1fef34b2dba6d4a732e13ce38 already in progress: creating tablet
I20251211 20:52:22.327263 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 8e9eded009e44c1a9544d4471c04c5c8 with delete_type TABLET_DATA_DELETED (replaced by tablet 97a519a8ca7c4f5380a230a70ee84052) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:22.330775 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet b815bca893a8446e92d48973d9a0408e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b815bca893a8446e92d48973d9a0408e already in progress: creating tablet
W20251211 20:52:22.331272 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet aa685d1b051a4c93a2d313429d914ead because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aa685d1b051a4c93a2d313429d914ead already in progress: creating tablet
W20251211 20:52:22.331489 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet edc4300dc5a84173aa95b587d77fc95c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet edc4300dc5a84173aa95b587d77fc95c already in progress: creating tablet
I20251211 20:52:22.330053 750 tablet_service.cc:1505] Processing CreateTablet for tablet acdb7dddf3e74ce1bc92f232b6180cca (DEFAULT_TABLE table=default.Table_104f49b10d1d45749449d0ff3009e1d9 [id=b68aac5147654707b677d020a2cc37cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.332051 746 tablet_service.cc:1505] Processing CreateTablet for tablet e69d53ac90a7463aa36ff15dadad7e0b (DEFAULT_TABLE table=default.Table_2645aa306b5c44f6bad5e2e67a91ebec [id=0ec7cf10eb8a47aca499b141913b38cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.332280 1160 tablet_bootstrap.cc:654] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:22.332347 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_ce0d3130963649ef8ca062313c67f1cc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:22.332593 745 tablet_service.cc:1505] Processing CreateTablet for tablet 73aa2a3fb2f44bbca870fb7f4cf6087c (DEFAULT_TABLE table=default.Table_3fca4e8f1b6a427d8d016e27f88ea1ce [id=4f93b23edd4548aeb6bcb8f92878907d]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:22.333040 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ce0d3130963649ef8ca062313c67f1cc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.332679 748 tablet_service.cc:1505] Processing CreateTablet for tablet 049cdecf12304211adbc1c1008dba989 (DEFAULT_TABLE table=default.Table_3030ffcddf0240ae81dfe06ce7d31fe5 [id=34edf86667d448c0b87e55627c67a704]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.332444 752 tablet_service.cc:1505] Processing CreateTablet for tablet 7f086c7c56fe4abb8df7d3c0978cce6e (DEFAULT_TABLE table=default.Table_1b3f670b29d647a38ef00101ce8df343 [id=872ff521abd740ee924f22e9c0ff8585]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.336203 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4db14d082d34aef857e106992bb8950. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.337702 738 tablet_service.cc:1552] Processing DeleteTablet for tablet 531a74c1fef34b2dba6d4a732e13ce38 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:22.337745 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_86e66f9a19644d61b002c42103d4fda8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:22.338079 738 tablet_service.cc:1552] Processing DeleteTablet for tablet b815bca893a8446e92d48973d9a0408e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:22.338269 738 tablet_service.cc:1552] Processing DeleteTablet for tablet edc4300dc5a84173aa95b587d77fc95c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:22.338395 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_86e66f9a19644d61b002c42103d4fda8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.341566 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_fbca1745cad8402fba1c38f2c3b0a9f4" } new_table_name: "default.Table_02014bfadb7b434f960168f2db70b838" modify_external_catalogs: true
I20251211 20:52:22.343933 591 master_service.cc:954] ReplaceTablet: received request to replace tablet 274c10c2aa46479dab5edbb483e5b883 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:22.346146 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 034f47d8564f4094a9d76aee2fea3119. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.348177 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af3ca9ac988e458cb7068c10c006d587. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.348225 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 49193f7bd8ac4f20a6ca980a9e1eb830 with delete_type TABLET_DATA_DELETED (replaced by tablet 5ed0a0141051462eb45ca3a66d41d009) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:22.350512 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_a2a9717ed78046cab7c00077ac646665"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.351096 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a2a9717ed78046cab7c00077ac646665 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.352545 880 tablet_service.cc:1552] Processing DeleteTablet for tablet edc4300dc5a84173aa95b587d77fc95c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:22.342381 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 8e9eded009e44c1a9544d4471c04c5c8 with delete_type TABLET_DATA_DELETED (replaced by tablet 97a519a8ca7c4f5380a230a70ee84052) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:22.354009 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 353da9106578450987a4f6544df8d7e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.354378 754 tablet_service.cc:1505] Processing CreateTablet for tablet 5ed0a0141051462eb45ca3a66d41d009 (DEFAULT_TABLE table=default.Table_40256d3973c74566a09b1d235062d284 [id=1f4650c549d94e2c9ef753ff96670aa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.355667 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73aa2a3fb2f44bbca870fb7f4cf6087c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.357978 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e69d53ac90a7463aa36ff15dadad7e0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.360790 1166 tablet_bootstrap.cc:492] T 274c10c2aa46479dab5edbb483e5b883 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:22.360982 1166 ts_tablet_manager.cc:1403] T 274c10c2aa46479dab5edbb483e5b883 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.063s user 0.015s sys 0.011s
I20251211 20:52:22.361884 1166 raft_consensus.cc:359] T 274c10c2aa46479dab5edbb483e5b883 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.362098 1166 raft_consensus.cc:385] T 274c10c2aa46479dab5edbb483e5b883 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:22.362174 1166 raft_consensus.cc:740] T 274c10c2aa46479dab5edbb483e5b883 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:22.362959 1166 consensus_queue.cc:260] T 274c10c2aa46479dab5edbb483e5b883 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.363198 741 tablet_service.cc:1552] Processing DeleteTablet for tablet aa685d1b051a4c93a2d313429d914ead with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:22.363132 747 tablet_service.cc:1505] Processing CreateTablet for tablet af3ca9ac988e458cb7068c10c006d587 (DEFAULT_TABLE table=default.Table_a11e64052cf34ba6a6a31f4edb417c19 [id=0afc1e783888443eb3f3e7848a04c207]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.363480 739 tablet_service.cc:1552] Processing DeleteTablet for tablet 5ed0a0141051462eb45ca3a66d41d009 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:22.363667 1166 ts_tablet_manager.cc:1434] T 274c10c2aa46479dab5edbb483e5b883 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20251211 20:52:22.363798 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edc4300dc5a84173aa95b587d77fc95c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.364684 743 tablet_service.cc:1505] Processing CreateTablet for tablet 034f47d8564f4094a9d76aee2fea3119 (DEFAULT_TABLE table=default.Table_b5a3abc502a64d6083d78344400b5eeb [id=c58ec33f45c348329be4acee3d5f4f6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.364843 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_9d6a8e006ed241819eaecb22446277f5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:22.365077 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01ada50100bc48e39d1bc67082da750e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.365196 742 tablet_service.cc:1505] Processing CreateTablet for tablet f4db14d082d34aef857e106992bb8950 (DEFAULT_TABLE table=default.Table_9ac73058eb564b63871197be450065f0 [id=ffb336d92b05442b80d70cd1f419431c]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:22.365422 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9d6a8e006ed241819eaecb22446277f5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.365768 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ed0a0141051462eb45ca3a66d41d009. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.368120 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e69d53ac90a7463aa36ff15dadad7e0b. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:22.368114 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 62fb046960024a2b9d70dc46ae52d53f consensus state has no leader
I20251211 20:52:22.370815 1166 tablet_bootstrap.cc:492] T 72d7d2f78ab64bbebb63239b9ecb366e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:22.371832 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_e0a4d8f6e8a84693b285f1f8e6de6e42"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:22.372139 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f086c7c56fe4abb8df7d3c0978cce6e. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:22.372388 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e0a4d8f6e8a84693b285f1f8e6de6e42 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.372584 1166 tablet_bootstrap.cc:654] T 72d7d2f78ab64bbebb63239b9ecb366e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:22.375813 744 tablet_service.cc:1505] Processing CreateTablet for tablet 353da9106578450987a4f6544df8d7e6 (DEFAULT_TABLE table=default.Table_a0c412985adf4382a7ea28c77654fd6c [id=4a74b9a60cbb4198b55cee630b2f4bef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.376276 1165 ts_tablet_manager.cc:1929] T 00d68e696250404d880b8f97af217cbd P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.376550 1165 log.cc:1199] T 00d68e696250404d880b8f97af217cbd P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/00d68e696250404d880b8f97af217cbd
I20251211 20:52:22.377823 1168 tablet_bootstrap.cc:492] T 12313c06b33749589ceb08ade798fc59 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:22.380270 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 62fb046960024a2b9d70dc46ae52d53f
I20251211 20:52:22.380774 589 catalog_manager.cc:6724] ReplaceTablet: tablet 62fb046960024a2b9d70dc46ae52d53f deleted and replaced by tablet ec7df7b11ce34b8489048e6acdcb4e94
I20251211 20:52:22.384840 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 049cdecf12304211adbc1c1008dba989. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.386041 1168 ts_tablet_manager.cc:1403] T 12313c06b33749589ceb08ade798fc59 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.243s user 0.022s sys 0.018s
I20251211 20:52:22.388979 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a064789f89c424ca698e52333d73eea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.392158 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af3ca9ac988e458cb7068c10c006d587. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.392421 743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 034f47d8564f4094a9d76aee2fea3119. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.392642 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4db14d082d34aef857e106992bb8950. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.392833 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 353da9106578450987a4f6544df8d7e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.393077 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73aa2a3fb2f44bbca870fb7f4cf6087c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.393252 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acdb7dddf3e74ce1bc92f232b6180cca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.393494 1165 ts_tablet_manager.cc:1950] T 00d68e696250404d880b8f97af217cbd P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:22.394572 1165 tablet_replica.cc:333] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:22.394764 1165 raft_consensus.cc:2243] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.394918 1165 raft_consensus.cc:2272] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.395419 1165 ts_tablet_manager.cc:1916] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:22.401954 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet 62fb046960024a2b9d70dc46ae52d53f with delete_type TABLET_DATA_DELETED (replaced by tablet ec7df7b11ce34b8489048e6acdcb4e94) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:22.402710 1023 tablet_service.cc:1505] Processing CreateTablet for tablet e41f052151624df39a3c83744870c94e (DEFAULT_TABLE table=default.Table_bcccedcd97444712a0f74e76db8cd74a [id=3de8f4cc61014e96aa64e7d4dd768096]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.402846 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 00d68e696250404d880b8f97af217cbd (table default.Table_5ef3d8d2746a4016ad8efac84ba492b5 [id=ebae22b7d617403c84c023d0628f8ad8]) successfully deleted
I20251211 20:52:22.403129 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e41f052151624df39a3c83744870c94e. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:22.404548 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 62fb046960024a2b9d70dc46ae52d53f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 62fb046960024a2b9d70dc46ae52d53f already in progress: creating tablet
I20251211 20:52:22.405241 739 tablet_service.cc:1505] Processing CreateTablet for tablet 694518e6c1ae4032b06b99e9bde2e5dd (DEFAULT_TABLE table=default.Table_21f7b3d63a0146a7ab330b1469611bfe [id=d49d5216afba41d68497f42fc72b1bf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.405843 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 694518e6c1ae4032b06b99e9bde2e5dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.409111 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 694518e6c1ae4032b06b99e9bde2e5dd (DEFAULT_TABLE table=default.Table_21f7b3d63a0146a7ab330b1469611bfe [id=d49d5216afba41d68497f42fc72b1bf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.409478 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 694518e6c1ae4032b06b99e9bde2e5dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.411412 1168 raft_consensus.cc:359] T 12313c06b33749589ceb08ade798fc59 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:22.411671 1168 raft_consensus.cc:385] T 12313c06b33749589ceb08ade798fc59 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:22.411764 1168 raft_consensus.cc:740] T 12313c06b33749589ceb08ade798fc59 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:22.411743 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 274c10c2aa46479dab5edbb483e5b883 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:22.412009 1163 tablet_replica.cc:333] T 274c10c2aa46479dab5edbb483e5b883 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:22.411978 1168 consensus_queue.cc:260] T 12313c06b33749589ceb08ade798fc59 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:22.412199 1163 raft_consensus.cc:2243] T 274c10c2aa46479dab5edbb483e5b883 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.412407 1163 raft_consensus.cc:2272] T 274c10c2aa46479dab5edbb483e5b883 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.412444 1168 ts_tablet_manager.cc:1434] T 12313c06b33749589ceb08ade798fc59 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.026s user 0.002s sys 0.000s
I20251211 20:52:22.412748 1168 tablet_bootstrap.cc:492] T b67de345cf44441abe9e1431b64c4ba4 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:22.413539 1163 ts_tablet_manager.cc:1916] T 274c10c2aa46479dab5edbb483e5b883 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:22.413836 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 62fb046960024a2b9d70dc46ae52d53f with delete_type TABLET_DATA_DELETED (replaced by tablet ec7df7b11ce34b8489048e6acdcb4e94) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:22.413990 1021 tablet_service.cc:1505] Processing CreateTablet for tablet 97a519a8ca7c4f5380a230a70ee84052 (DEFAULT_TABLE table=default.Table_7faeeac71cd74273819ae03fe5272ee4 [id=383392d835da4c50848068d83261cecc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.414397 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97a519a8ca7c4f5380a230a70ee84052. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.425171 757 tablet_service.cc:1505] Processing CreateTablet for tablet e41f052151624df39a3c83744870c94e (DEFAULT_TABLE table=default.Table_bcccedcd97444712a0f74e76db8cd74a [id=3de8f4cc61014e96aa64e7d4dd768096]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.425619 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e41f052151624df39a3c83744870c94e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.427452 1163 ts_tablet_manager.cc:1929] T 274c10c2aa46479dab5edbb483e5b883 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.427646 1163 log.cc:1199] T 274c10c2aa46479dab5edbb483e5b883 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/274c10c2aa46479dab5edbb483e5b883
I20251211 20:52:22.427994 1163 ts_tablet_manager.cc:1950] T 274c10c2aa46479dab5edbb483e5b883 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:22.429617 1166 tablet_bootstrap.cc:492] T 72d7d2f78ab64bbebb63239b9ecb366e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:22.429792 1166 ts_tablet_manager.cc:1403] T 72d7d2f78ab64bbebb63239b9ecb366e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.059s user 0.013s sys 0.018s
I20251211 20:52:22.430598 1166 raft_consensus.cc:359] T 72d7d2f78ab64bbebb63239b9ecb366e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.430851 1166 raft_consensus.cc:385] T 72d7d2f78ab64bbebb63239b9ecb366e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:22.430954 1166 raft_consensus.cc:740] T 72d7d2f78ab64bbebb63239b9ecb366e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:22.435211 1166 consensus_queue.cc:260] T 72d7d2f78ab64bbebb63239b9ecb366e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.430454 879 tablet_service.cc:1552] Processing DeleteTablet for tablet 62fb046960024a2b9d70dc46ae52d53f with delete_type TABLET_DATA_DELETED (replaced by tablet ec7df7b11ce34b8489048e6acdcb4e94) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:22.437544 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 732c76659c0540bc8b6909b430ffb146 (DEFAULT_TABLE table=default.Table_a2a9717ed78046cab7c00077ac646665 [id=65dab4128d204227947dd4651a5aeb6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.438009 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 732c76659c0540bc8b6909b430ffb146. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.443737 594 master_service.cc:954] ReplaceTablet: received request to replace tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:22.444566 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 1951a67964834152afb31f95741d981a (DEFAULT_TABLE table=default.Table_7747db5b5c664662bfe1e87d1e206c91 [id=016a0bf9d2704b74af4e64d2f9a23878]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.444960 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1951a67964834152afb31f95741d981a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.430989 880 tablet_service.cc:1505] Processing CreateTablet for tablet 97a519a8ca7c4f5380a230a70ee84052 (DEFAULT_TABLE table=default.Table_7faeeac71cd74273819ae03fe5272ee4 [id=383392d835da4c50848068d83261cecc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.450649 1166 ts_tablet_manager.cc:1434] T 72d7d2f78ab64bbebb63239b9ecb366e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.021s user 0.001s sys 0.001s
I20251211 20:52:22.450778 594 master_service.cc:954] ReplaceTablet: received request to replace tablet ae3be9d91e5d4551bc706b2043a9edf5 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:22.451022 1166 tablet_bootstrap.cc:492] T d0df24ce65234eeba97f9fdd351b58ad P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:22.452347 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 33e1d8a576cd41b1a34778b1458df396 (DEFAULT_TABLE table=default.Table_5e9e111d44b04f94b728307574520f89 [id=2bf05710731345a786503ee222ff6f55]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.452512 1024 tablet_service.cc:1505] Processing CreateTablet for tablet a5eeb56709f046d9be92ac600ebc330e (DEFAULT_TABLE table=default.Table_a7c6475971744685ace8304ecf702b38 [id=f78e1da479af4d23a552af1c4fc55f81]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.452800 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33e1d8a576cd41b1a34778b1458df396. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.453014 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5eeb56709f046d9be92ac600ebc330e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.459745 1024 tablet_service.cc:1505] Processing CreateTablet for tablet ec7df7b11ce34b8489048e6acdcb4e94 (DEFAULT_TABLE table=default.Table_02014bfadb7b434f960168f2db70b838 [id=14a0dbb8486246b09f3e06fc2120cd28]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.460168 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec7df7b11ce34b8489048e6acdcb4e94. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.460805 894 tablet_service.cc:1505] Processing CreateTablet for tablet e41f052151624df39a3c83744870c94e (DEFAULT_TABLE table=default.Table_bcccedcd97444712a0f74e76db8cd74a [id=3de8f4cc61014e96aa64e7d4dd768096]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.457324 879 tablet_service.cc:1505] Processing CreateTablet for tablet 694518e6c1ae4032b06b99e9bde2e5dd (DEFAULT_TABLE table=default.Table_21f7b3d63a0146a7ab330b1469611bfe [id=d49d5216afba41d68497f42fc72b1bf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.467073 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e41f052151624df39a3c83744870c94e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.467085 1166 tablet_bootstrap.cc:654] T d0df24ce65234eeba97f9fdd351b58ad P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:22.467557 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet ae3be9d91e5d4551bc706b2043a9edf5
I20251211 20:52:22.468036 594 catalog_manager.cc:6724] ReplaceTablet: tablet ae3be9d91e5d4551bc706b2043a9edf5 deleted and replaced by tablet e2e0a1e32bba40e294bf0a2fe10a98cd
I20251211 20:52:22.468833 1023 tablet_service.cc:1505] Processing CreateTablet for tablet f92d26bcde3b410aa56bc23a61b1ac35 (DEFAULT_TABLE table=default.Table_86e66f9a19644d61b002c42103d4fda8 [id=bee5829ed7d44761a9ae57d3134d38dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.469281 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f92d26bcde3b410aa56bc23a61b1ac35. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.461236 881 tablet_service.cc:1505] Processing CreateTablet for tablet 732c76659c0540bc8b6909b430ffb146 (DEFAULT_TABLE table=default.Table_a2a9717ed78046cab7c00077ac646665 [id=65dab4128d204227947dd4651a5aeb6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.463268 877 tablet_service.cc:1505] Processing CreateTablet for tablet 33e1d8a576cd41b1a34778b1458df396 (DEFAULT_TABLE table=default.Table_5e9e111d44b04f94b728307574520f89 [id=2bf05710731345a786503ee222ff6f55]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.472502 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97a519a8ca7c4f5380a230a70ee84052. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.473836 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_7747db5b5c664662bfe1e87d1e206c91" } new_table_name: "default.Table_204572508f5643e6a0c5decb479b3932" modify_external_catalogs: true
I20251211 20:52:22.474763 595 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a7c6475971744685ace8304ecf702b38" } new_table_name: "default.Table_f2ce490e0c5d445ab84989aae77c2f19" modify_external_catalogs: true
I20251211 20:52:22.475274 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 274c10c2aa46479dab5edbb483e5b883 (table default.Table_7a3a07a264fc4bed96455f83ae51fb05 [id=5a91ece54cfa446880f72a42678f4f26]) successfully deleted
I20251211 20:52:22.476344 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 72d7d2f78ab64bbebb63239b9ecb366e with delete_type TABLET_DATA_DELETED (replaced by tablet 8a064789f89c424ca698e52333d73eea) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:22.476667 1163 tablet_replica.cc:333] T 72d7d2f78ab64bbebb63239b9ecb366e P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:22.476727 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e0a4d8f6e8a84693b285f1f8e6de6e42" } new_table_name: "default.Table_5cb009a3a8b24126af4e28df1795d049" modify_external_catalogs: true
I20251211 20:52:22.476876 1163 raft_consensus.cc:2243] T 72d7d2f78ab64bbebb63239b9ecb366e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.477061 1163 raft_consensus.cc:2272] T 72d7d2f78ab64bbebb63239b9ecb366e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.477036 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b5a3abc502a64d6083d78344400b5eeb" } modify_external_catalogs: true
I20251211 20:52:22.477309 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_673da12cd31d4ba789de37d39fd60988" } modify_external_catalogs: true
I20251211 20:52:22.477612 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_02014bfadb7b434f960168f2db70b838" } modify_external_catalogs: true
I20251211 20:52:22.477841 1163 ts_tablet_manager.cc:1916] T 72d7d2f78ab64bbebb63239b9ecb366e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:22.477882 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f439572c5b1d4997af54e0ab3c036023" } modify_external_catalogs: true
I20251211 20:52:22.478471 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_5e9e111d44b04f94b728307574520f89" } new_table_name: "default.Table_ef1bdedfd2fc40f29769fd6245dccf4a" modify_external_catalogs: true
I20251211 20:52:22.480980 1029 tablet_service.cc:1505] Processing CreateTablet for tablet c58241f7e1404a8c8db92c4f9a47d143 (DEFAULT_TABLE table=default.Table_ce0d3130963649ef8ca062313c67f1cc [id=90602e87d8b6425295bb04a5fca84ba7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.483959 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 9662efc97a53461d888a062463e80ee6 (DEFAULT_TABLE table=default.Table_9d6a8e006ed241819eaecb22446277f5 [id=655f70b9f7e0400aa42179b922af2fa2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.484369 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c58241f7e1404a8c8db92c4f9a47d143. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.485646 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 4870da0224e048ed82f227d7ba676a94 (DEFAULT_TABLE table=default.Table_e0a4d8f6e8a84693b285f1f8e6de6e42 [id=28f3ebe9b7a64b5e89404b0407b0e5ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.486017 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4870da0224e048ed82f227d7ba676a94. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.492389 1021 tablet_service.cc:1552] Processing DeleteTablet for tablet ae3be9d91e5d4551bc706b2043a9edf5 with delete_type TABLET_DATA_DELETED (replaced by tablet e2e0a1e32bba40e294bf0a2fe10a98cd) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:22.492842 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9662efc97a53461d888a062463e80ee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.493831 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_02014bfadb7b434f960168f2db70b838" } modify_external_catalogs: true
I20251211 20:52:22.495069 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b5a3abc502a64d6083d78344400b5eeb" } modify_external_catalogs: true
I20251211 20:52:22.497867 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_673da12cd31d4ba789de37d39fd60988" } modify_external_catalogs: true
I20251211 20:52:22.499292 754 tablet_service.cc:1505] Processing CreateTablet for tablet 97a519a8ca7c4f5380a230a70ee84052 (DEFAULT_TABLE table=default.Table_7faeeac71cd74273819ae03fe5272ee4 [id=383392d835da4c50848068d83261cecc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.499758 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97a519a8ca7c4f5380a230a70ee84052. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.499922 1163 ts_tablet_manager.cc:1929] T 72d7d2f78ab64bbebb63239b9ecb366e P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.500059 1163 log.cc:1199] T 72d7d2f78ab64bbebb63239b9ecb366e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/72d7d2f78ab64bbebb63239b9ecb366e
I20251211 20:52:22.500914 1163 ts_tablet_manager.cc:1950] T 72d7d2f78ab64bbebb63239b9ecb366e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:22.501730 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f439572c5b1d4997af54e0ab3c036023" } modify_external_catalogs: true
I20251211 20:52:22.502054 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 72d7d2f78ab64bbebb63239b9ecb366e (table default.Table_f439572c5b1d4997af54e0ab3c036023 [id=cda4c0ad8a0347be88a0518491e6af49]) successfully deleted
I20251211 20:52:22.502358 1163 tablet_replica.cc:333] T ae3be9d91e5d4551bc706b2043a9edf5 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:22.502539 1163 raft_consensus.cc:2243] T ae3be9d91e5d4551bc706b2043a9edf5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.502703 1163 raft_consensus.cc:2272] T ae3be9d91e5d4551bc706b2043a9edf5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.505590 1163 ts_tablet_manager.cc:1916] T ae3be9d91e5d4551bc706b2043a9edf5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:22.505910 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 12313c06b33749589ceb08ade798fc59 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:22.508617 1166 tablet_bootstrap.cc:492] T d0df24ce65234eeba97f9fdd351b58ad P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:22.508796 1166 ts_tablet_manager.cc:1403] T d0df24ce65234eeba97f9fdd351b58ad P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.058s user 0.003s sys 0.000s
I20251211 20:52:22.471307 875 tablet_service.cc:1505] Processing CreateTablet for tablet ec7df7b11ce34b8489048e6acdcb4e94 (DEFAULT_TABLE table=default.Table_02014bfadb7b434f960168f2db70b838 [id=14a0dbb8486246b09f3e06fc2120cd28]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.472661 876 tablet_service.cc:1505] Processing CreateTablet for tablet a5eeb56709f046d9be92ac600ebc330e (DEFAULT_TABLE table=default.Table_a7c6475971744685ace8304ecf702b38 [id=f78e1da479af4d23a552af1c4fc55f81]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.470738 878 tablet_service.cc:1505] Processing CreateTablet for tablet 1951a67964834152afb31f95741d981a (DEFAULT_TABLE table=default.Table_7747db5b5c664662bfe1e87d1e206c91 [id=016a0bf9d2704b74af4e64d2f9a23878]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.509318 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 694518e6c1ae4032b06b99e9bde2e5dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.509518 1166 raft_consensus.cc:359] T d0df24ce65234eeba97f9fdd351b58ad P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:22.509723 1166 raft_consensus.cc:385] T d0df24ce65234eeba97f9fdd351b58ad P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:22.509800 1166 raft_consensus.cc:740] T d0df24ce65234eeba97f9fdd351b58ad P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
W20251211 20:52:22.509965 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1951a67964834152afb31f95741d981a consensus state has no leader
I20251211 20:52:22.510097 1166 consensus_queue.cc:260] T d0df24ce65234eeba97f9fdd351b58ad P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:22.513015 1166 ts_tablet_manager.cc:1434] T d0df24ce65234eeba97f9fdd351b58ad P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:22.514472 1166 tablet_bootstrap.cc:492] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:22.514986 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33e1d8a576cd41b1a34778b1458df396. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.517335 1166 tablet_bootstrap.cc:654] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:22.519943 595 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a5eeb56709f046d9be92ac600ebc330e consensus state has no leader
I20251211 20:52:22.520402 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 034f47d8564f4094a9d76aee2fea3119
I20251211 20:52:22.521370 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet 034f47d8564f4094a9d76aee2fea3119 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:22.521534 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9d6a8e006ed241819eaecb22446277f5" } modify_external_catalogs: true
I20251211 20:52:22.521719 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9d6a8e006ed241819eaecb22446277f5" } modify_external_catalogs: true
I20251211 20:52:22.522856 1163 ts_tablet_manager.cc:1929] T ae3be9d91e5d4551bc706b2043a9edf5 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.522995 1163 log.cc:1199] T ae3be9d91e5d4551bc706b2043a9edf5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/ae3be9d91e5d4551bc706b2043a9edf5
I20251211 20:52:22.522996 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_9b96425fc6f34e15ab398770cb57bc98"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:22.523314 1163 ts_tablet_manager.cc:1950] T ae3be9d91e5d4551bc706b2043a9edf5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
W20251211 20:52:22.523612 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9b96425fc6f34e15ab398770cb57bc98 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.523765 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 732c76659c0540bc8b6909b430ffb146. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.524271 875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec7df7b11ce34b8489048e6acdcb4e94. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.524353 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet ae3be9d91e5d4551bc706b2043a9edf5 (table default.Table_673da12cd31d4ba789de37d39fd60988 [id=b9fb32193afe4e5faa21db3c195030ab]) successfully deleted
I20251211 20:52:22.524926 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet ec7df7b11ce34b8489048e6acdcb4e94
I20251211 20:52:22.525763 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet ec7df7b11ce34b8489048e6acdcb4e94 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:22.526087 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4870da0224e048ed82f227d7ba676a94 consensus state has no leader
W20251211 20:52:22.527874 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 034f47d8564f4094a9d76aee2fea3119 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 034f47d8564f4094a9d76aee2fea3119 already in progress: creating tablet
I20251211 20:52:22.528343 755 tablet_service.cc:1505] Processing CreateTablet for tablet 1951a67964834152afb31f95741d981a (DEFAULT_TABLE table=default.Table_7747db5b5c664662bfe1e87d1e206c91 [id=016a0bf9d2704b74af4e64d2f9a23878]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.528801 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1951a67964834152afb31f95741d981a. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:22.528817 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet ec7df7b11ce34b8489048e6acdcb4e94 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ec7df7b11ce34b8489048e6acdcb4e94 already in progress: creating tablet
I20251211 20:52:22.529728 756 tablet_service.cc:1505] Processing CreateTablet for tablet 732c76659c0540bc8b6909b430ffb146 (DEFAULT_TABLE table=default.Table_a2a9717ed78046cab7c00077ac646665 [id=65dab4128d204227947dd4651a5aeb6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.530514 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet d0df24ce65234eeba97f9fdd351b58ad with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:22.530790 1163 tablet_replica.cc:333] T d0df24ce65234eeba97f9fdd351b58ad P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:22.530963 1163 raft_consensus.cc:2243] T d0df24ce65234eeba97f9fdd351b58ad P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.531093 876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5eeb56709f046d9be92ac600ebc330e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.531154 1163 raft_consensus.cc:2272] T d0df24ce65234eeba97f9fdd351b58ad P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.531352 589 master_service.cc:954] ReplaceTablet: received request to replace tablet edc4300dc5a84173aa95b587d77fc95c from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:22.532613 1163 ts_tablet_manager.cc:1916] T d0df24ce65234eeba97f9fdd351b58ad P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:22.531994 753 tablet_service.cc:1505] Processing CreateTablet for tablet 33e1d8a576cd41b1a34778b1458df396 (DEFAULT_TABLE table=default.Table_5e9e111d44b04f94b728307574520f89 [id=2bf05710731345a786503ee222ff6f55]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.538329 1163 ts_tablet_manager.cc:1929] T d0df24ce65234eeba97f9fdd351b58ad P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.538470 1163 log.cc:1199] T d0df24ce65234eeba97f9fdd351b58ad P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/d0df24ce65234eeba97f9fdd351b58ad
I20251211 20:52:22.538839 1163 ts_tablet_manager.cc:1950] T d0df24ce65234eeba97f9fdd351b58ad P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:22.542026 744 tablet_service.cc:1505] Processing CreateTablet for tablet ec7df7b11ce34b8489048e6acdcb4e94 (DEFAULT_TABLE table=default.Table_02014bfadb7b434f960168f2db70b838 [id=14a0dbb8486246b09f3e06fc2120cd28]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.543445 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1951a67964834152afb31f95741d981a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.544205 588 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet e2e0a1e32bba40e294bf0a2fe10a98cd
I20251211 20:52:22.544602 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 8a064789f89c424ca698e52333d73eea
I20251211 20:52:22.545454 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet 8a064789f89c424ca698e52333d73eea with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:22.546303 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 732c76659c0540bc8b6909b430ffb146. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.548152 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet d0df24ce65234eeba97f9fdd351b58ad (table default.Table_1b829e0ef7ef4b17be671db434f3858c [id=b8c401dfdb5f4f7584dca0989287ce31]) successfully deleted
W20251211 20:52:22.548900 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 8a064789f89c424ca698e52333d73eea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8a064789f89c424ca698e52333d73eea already in progress: creating tablet
I20251211 20:52:22.549453 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33e1d8a576cd41b1a34778b1458df396. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.550060 750 tablet_service.cc:1505] Processing CreateTablet for tablet 4870da0224e048ed82f227d7ba676a94 (DEFAULT_TABLE table=default.Table_e0a4d8f6e8a84693b285f1f8e6de6e42 [id=28f3ebe9b7a64b5e89404b0407b0e5ae]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:22.554442 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 33e1d8a576cd41b1a34778b1458df396 consensus state has no leader
I20251211 20:52:22.556840 738 tablet_service.cc:1552] Processing DeleteTablet for tablet ec7df7b11ce34b8489048e6acdcb4e94 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:22.557080 749 tablet_service.cc:1505] Processing CreateTablet for tablet 9662efc97a53461d888a062463e80ee6 (DEFAULT_TABLE table=default.Table_9d6a8e006ed241819eaecb22446277f5 [id=655f70b9f7e0400aa42179b922af2fa2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.558853 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 034f47d8564f4094a9d76aee2fea3119 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:22.545384 741 tablet_service.cc:1505] Processing CreateTablet for tablet a5eeb56709f046d9be92ac600ebc330e (DEFAULT_TABLE table=default.Table_a7c6475971744685ace8304ecf702b38 [id=f78e1da479af4d23a552af1c4fc55f81]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.559886 739 tablet_service.cc:1505] Processing CreateTablet for tablet f92d26bcde3b410aa56bc23a61b1ac35 (DEFAULT_TABLE table=default.Table_86e66f9a19644d61b002c42103d4fda8 [id=bee5829ed7d44761a9ae57d3134d38dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.563654 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 9662efc97a53461d888a062463e80ee6
I20251211 20:52:22.564666 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet 9662efc97a53461d888a062463e80ee6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:22.565356 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 9662efc97a53461d888a062463e80ee6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9662efc97a53461d888a062463e80ee6 already in progress: creating tablet
I20251211 20:52:22.568522 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_7eabd17b66f44d40aebe078e0404f66f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.569361 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7eabd17b66f44d40aebe078e0404f66f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.575114 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_7faeeac71cd74273819ae03fe5272ee4" } new_table_name: "default.Table_b225141042f247aca03de91c6a346513" modify_external_catalogs: true
I20251211 20:52:22.577208 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_e1406b8c64104f0a9d31ff37e042163e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.577823 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e1406b8c64104f0a9d31ff37e042163e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.579617 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_ebec874d437748c487f8bf9d0ccaa7fa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.580124 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ebec874d437748c487f8bf9d0ccaa7fa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.581874 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_242e7ff74c6742708701ef45a63199f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.582476 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_242e7ff74c6742708701ef45a63199f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.593537 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_26b3b7e5db73457da38332d4e3d943f6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.594178 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_26b3b7e5db73457da38332d4e3d943f6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.596100 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b06a3971c3be404e9787e05466931423"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.596709 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b06a3971c3be404e9787e05466931423 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.598934 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_d8c241666f7840b8bf0cb7672482aa55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.599570 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d8c241666f7840b8bf0cb7672482aa55 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.601915 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_daeaf2f3dd24423e9060e4f09a5f4c99"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.602511 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_daeaf2f3dd24423e9060e4f09a5f4c99 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.613067 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_fd4577173dc44659b95db8e463e0131c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.613809 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fd4577173dc44659b95db8e463e0131c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.629210 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4870da0224e048ed82f227d7ba676a94. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.629387 745 tablet_service.cc:1552] Processing DeleteTablet for tablet ae3be9d91e5d4551bc706b2043a9edf5 with delete_type TABLET_DATA_DELETED (replaced by tablet e2e0a1e32bba40e294bf0a2fe10a98cd) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:22.629639 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5eeb56709f046d9be92ac600ebc330e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.629773 738 tablet_service.cc:1552] Processing DeleteTablet for tablet 8a064789f89c424ca698e52333d73eea with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:22.629824 1168 tablet_bootstrap.cc:654] T b67de345cf44441abe9e1431b64c4ba4 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:22.629596 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b67de345cf44441abe9e1431b64c4ba4" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:22.630645 982 leader_election.cc:343] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:22.632485 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 9662efc97a53461d888a062463e80ee6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:22.635784 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f92d26bcde3b410aa56bc23a61b1ac35. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:22.638029 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 97a519a8ca7c4f5380a230a70ee84052 consensus state has no leader
I20251211 20:52:22.639489 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9662efc97a53461d888a062463e80ee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.644945 1165 ts_tablet_manager.cc:1929] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.648247 1165 log.cc:1199] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/03ebcee3069f44f99761cc13e34512ed
I20251211 20:52:22.648949 1165 ts_tablet_manager.cc:1950] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:22.647089 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec7df7b11ce34b8489048e6acdcb4e94. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.650133 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 03ebcee3069f44f99761cc13e34512ed (table default.Table_ebc736f835124930b0060d37924294af [id=7b11b994218b4802a07f78be9ed38877]) successfully deleted
W20251211 20:52:22.652035 581 catalog_manager.cc:4952] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 997fd184e900424bb5eabfde8c676d0e because the tablet was not found. No further retry: Not found: Tablet not found: 997fd184e900424bb5eabfde8c676d0e
I20251211 20:52:22.655050 1168 tablet_bootstrap.cc:492] T b67de345cf44441abe9e1431b64c4ba4 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:22.655234 1168 ts_tablet_manager.cc:1403] T b67de345cf44441abe9e1431b64c4ba4 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.243s user 0.005s sys 0.009s
I20251211 20:52:22.655941 1165 tablet_replica.cc:333] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:22.656143 1165 raft_consensus.cc:2243] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.656034 1168 raft_consensus.cc:359] T b67de345cf44441abe9e1431b64c4ba4 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:22.656271 1168 raft_consensus.cc:385] T b67de345cf44441abe9e1431b64c4ba4 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:22.656338 1165 raft_consensus.cc:2272] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.656356 1168 raft_consensus.cc:740] T b67de345cf44441abe9e1431b64c4ba4 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:22.656644 1168 consensus_queue.cc:260] T b67de345cf44441abe9e1431b64c4ba4 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:22.657143 1168 ts_tablet_manager.cc:1434] T b67de345cf44441abe9e1431b64c4ba4 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20251211 20:52:22.657276 711 leader_election.cc:343] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Not found: Tablet not found: 03ebcee3069f44f99761cc13e34512ed
I20251211 20:52:22.657477 1168 tablet_bootstrap.cc:492] T 72d7d2f78ab64bbebb63239b9ecb366e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:22.657426 711 leader_election.cc:304] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:22.657847 711 raft_consensus.cc:2686] T 03ebcee3069f44f99761cc13e34512ed P 6f8d784fd1cd492bbe3809403ec34919: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251211 20:52:22.660873 1166 tablet_bootstrap.cc:492] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:22.661042 1166 ts_tablet_manager.cc:1403] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.147s user 0.040s sys 0.052s
I20251211 20:52:22.661831 1166 raft_consensus.cc:359] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:22.662021 1166 raft_consensus.cc:385] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:22.662124 1166 raft_consensus.cc:740] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:22.662348 1166 consensus_queue.cc:260] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:22.664927 711 leader_election.cc:343] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Not found: Tablet not found: f5467114696046fbb2f311856b7a4d73
I20251211 20:52:22.666178 1166 ts_tablet_manager.cc:1434] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251211 20:52:22.666149 711 leader_election.cc:304] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:22.666895 1182 raft_consensus.cc:493] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:22.667047 1182 raft_consensus.cc:515] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:22.668841 1182 leader_election.cc:290] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
W20251211 20:52:22.669112 710 leader_election.cc:343] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: d42e5cce248342f59d67f7c9aa6f66a2
I20251211 20:52:22.669154 749 tablet_service.cc:1552] Processing DeleteTablet for tablet b67de345cf44441abe9e1431b64c4ba4 with delete_type TABLET_DATA_DELETED (replaced by tablet d0df24ce65234eeba97f9fdd351b58ad) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:22.669019 1166 tablet_bootstrap.cc:492] T deacd1f8f3bf4b1f923b61b312492104 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:22.670194 891 tablet_service.cc:1505] Processing CreateTablet for tablet f92d26bcde3b410aa56bc23a61b1ac35 (DEFAULT_TABLE table=default.Table_86e66f9a19644d61b002c42103d4fda8 [id=bee5829ed7d44761a9ae57d3134d38dc]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:22.669303 711 raft_consensus.cc:2686] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251211 20:52:22.671537 1166 tablet_bootstrap.cc:654] T deacd1f8f3bf4b1f923b61b312492104 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:22.672092 754 tablet_service.cc:1505] Processing CreateTablet for tablet c58241f7e1404a8c8db92c4f9a47d143 (DEFAULT_TABLE table=default.Table_ce0d3130963649ef8ca062313c67f1cc [id=90602e87d8b6425295bb04a5fca84ba7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.672484 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c58241f7e1404a8c8db92c4f9a47d143. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.686581 1168 tablet_bootstrap.cc:654] T 72d7d2f78ab64bbebb63239b9ecb366e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:22.687089 1215 raft_consensus.cc:493] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:22.687274 1215 raft_consensus.cc:515] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:22.688046 1215 leader_election.cc:290] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:22.688416 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8e9eded009e44c1a9544d4471c04c5c8" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:22.688962 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 8e9eded009e44c1a9544d4471c04c5c8 with delete_type TABLET_DATA_DELETED (replaced by tablet 97a519a8ca7c4f5380a230a70ee84052) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:22.688964 982 leader_election.cc:343] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:22.689230 1163 tablet_replica.cc:333] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:22.689422 1163 raft_consensus.cc:2243] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.689590 1163 raft_consensus.cc:2272] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.690620 1029 tablet_service.cc:1505] Processing CreateTablet for tablet d19975cc513e414b80f478a6c5eb09de (DEFAULT_TABLE table=default.Table_7eabd17b66f44d40aebe078e0404f66f [id=d770b26c251e41e38f9720b37852cb11]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.691061 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d19975cc513e414b80f478a6c5eb09de. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.691203 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f92d26bcde3b410aa56bc23a61b1ac35. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:22.692219 711 leader_election.cc:343] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Not found: Tablet not found: d42e5cce248342f59d67f7c9aa6f66a2
I20251211 20:52:22.692377 711 leader_election.cc:304] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:22.692905 1182 raft_consensus.cc:2749] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:22.693287 739 tablet_service.cc:1505] Processing CreateTablet for tablet d19975cc513e414b80f478a6c5eb09de (DEFAULT_TABLE table=default.Table_7eabd17b66f44d40aebe078e0404f66f [id=d770b26c251e41e38f9720b37852cb11]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.693794 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d19975cc513e414b80f478a6c5eb09de. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.696147 1165 ts_tablet_manager.cc:1916] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:22.696681 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 297dc26ade384213b364a876c54bdd14 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:22.702162 591 master_service.cc:954] ReplaceTablet: received request to replace tablet 274c10c2aa46479dab5edbb483e5b883 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:22.704476 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 33cce042f33b4be198d11fd39f5f8838 (DEFAULT_TABLE table=default.Table_e1406b8c64104f0a9d31ff37e042163e [id=5a7b1e562b084b109569a8733fd470a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.704452 884 tablet_service.cc:1505] Processing CreateTablet for tablet c58241f7e1404a8c8db92c4f9a47d143 (DEFAULT_TABLE table=default.Table_ce0d3130963649ef8ca062313c67f1cc [id=90602e87d8b6425295bb04a5fca84ba7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.704879 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c58241f7e1404a8c8db92c4f9a47d143. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.704900 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33cce042f33b4be198d11fd39f5f8838. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.706629 1165 ts_tablet_manager.cc:1929] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.706746 1165 log.cc:1199] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/f5467114696046fbb2f311856b7a4d73
I20251211 20:52:22.707073 1165 ts_tablet_manager.cc:1950] T f5467114696046fbb2f311856b7a4d73 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:22.708205 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 297dc26ade384213b364a876c54bdd14
I20251211 20:52:22.708706 595 catalog_manager.cc:6724] ReplaceTablet: tablet 297dc26ade384213b364a876c54bdd14 deleted and replaced by tablet 8dff13e22899452c8d4f2731b8dac7d5
I20251211 20:52:22.709818 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 297dc26ade384213b364a876c54bdd14 with delete_type TABLET_DATA_DELETED (replaced by tablet 8dff13e22899452c8d4f2731b8dac7d5) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:22.710038 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 297dc26ade384213b364a876c54bdd14 with delete_type TABLET_DATA_DELETED (replaced by tablet 8dff13e22899452c8d4f2731b8dac7d5) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:22.711026 1165 tablet_replica.cc:333] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:22.711192 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet f5467114696046fbb2f311856b7a4d73 (table default.Table_b37d05b3e8eb4425bb5cb72d46ce9e91 [id=4e18dbcc527d4b12ba6978ca125d5085]) successfully deleted
I20251211 20:52:22.713526 595 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e1406b8c64104f0a9d31ff37e042163e" } new_table_name: "default.Table_fdd4d94139714e3c954d6daf3c82a649" modify_external_catalogs: true
I20251211 20:52:22.714107 1165 raft_consensus.cc:2243] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.714285 1165 raft_consensus.cc:2272] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.714835 1165 ts_tablet_manager.cc:1916] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:22.715641 1164 ts_tablet_manager.cc:1929] T 274c10c2aa46479dab5edbb483e5b883 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.716269 591 master_service.cc:954] ReplaceTablet: received request to replace tablet b815bca893a8446e92d48973d9a0408e from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:22.722254 591 master_service.cc:954] ReplaceTablet: received request to replace tablet d0df24ce65234eeba97f9fdd351b58ad from {username='slave'} at 127.0.0.1:44772
W20251211 20:52:22.726577 595 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 33cce042f33b4be198d11fd39f5f8838 consensus state has no leader
I20251211 20:52:22.727061 754 tablet_service.cc:1505] Processing CreateTablet for tablet 33cce042f33b4be198d11fd39f5f8838 (DEFAULT_TABLE table=default.Table_e1406b8c64104f0a9d31ff37e042163e [id=5a7b1e562b084b109569a8733fd470a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.727319 889 tablet_service.cc:1505] Processing CreateTablet for tablet 9662efc97a53461d888a062463e80ee6 (DEFAULT_TABLE table=default.Table_9d6a8e006ed241819eaecb22446277f5 [id=655f70b9f7e0400aa42179b922af2fa2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.728416 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33cce042f33b4be198d11fd39f5f8838. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.729533 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e6a96166bf33442989e15129d4d44631" } modify_external_catalogs: true
I20251211 20:52:22.729701 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e6a96166bf33442989e15129d4d44631" } modify_external_catalogs: true
I20251211 20:52:22.731086 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a11e64052cf34ba6a6a31f4edb417c19" } modify_external_catalogs: true
I20251211 20:52:22.731251 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a11e64052cf34ba6a6a31f4edb417c19" } modify_external_catalogs: true
I20251211 20:52:22.732468 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_21f7b3d63a0146a7ab330b1469611bfe" } new_table_name: "default.Table_61835079722245debfab2fb8362f368b" modify_external_catalogs: true
I20251211 20:52:22.734004 1165 ts_tablet_manager.cc:1929] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.729501 595 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ef1bdedfd2fc40f29769fd6245dccf4a" } new_table_name: "default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a" modify_external_catalogs: true
I20251211 20:52:22.736511 1164 log.cc:1199] T 274c10c2aa46479dab5edbb483e5b883 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/274c10c2aa46479dab5edbb483e5b883
I20251211 20:52:22.739183 893 tablet_service.cc:1552] Processing DeleteTablet for tablet ae3be9d91e5d4551bc706b2043a9edf5 with delete_type TABLET_DATA_DELETED (replaced by tablet e2e0a1e32bba40e294bf0a2fe10a98cd) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:22.740365 1165 log.cc:1199] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/d42e5cce248342f59d67f7c9aa6f66a2
I20251211 20:52:22.740558 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9662efc97a53461d888a062463e80ee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.741310 1165 ts_tablet_manager.cc:1950] T d42e5cce248342f59d67f7c9aa6f66a2 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:22.742782 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet d42e5cce248342f59d67f7c9aa6f66a2 (table default.Table_8cbb799adf2747bba3e7edf97a7f1f59 [id=1473b367a32b480f842106bf0f4e41fb]) successfully deleted
I20251211 20:52:22.744028 1165 tablet_replica.cc:333] T 638fee757fc444f8a76b5189c103c40c P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:22.744535 1165 raft_consensus.cc:2243] T 638fee757fc444f8a76b5189c103c40c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.745012 1165 raft_consensus.cc:2272] T 638fee757fc444f8a76b5189c103c40c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.745803 1165 ts_tablet_manager.cc:1916] T 638fee757fc444f8a76b5189c103c40c P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:22.752442 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e5a162d6463e4ff5a4edaaae5a4d03cb" } modify_external_catalogs: true
I20251211 20:52:22.752605 1164 ts_tablet_manager.cc:1950] T 274c10c2aa46479dab5edbb483e5b883 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:22.752674 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e5a162d6463e4ff5a4edaaae5a4d03cb" } modify_external_catalogs: true
I20251211 20:52:22.754455 1164 tablet_replica.cc:333] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:22.754413 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_2645aa306b5c44f6bad5e2e67a91ebec" } modify_external_catalogs: true
I20251211 20:52:22.754606 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_2645aa306b5c44f6bad5e2e67a91ebec" } modify_external_catalogs: true
I20251211 20:52:22.754631 1164 raft_consensus.cc:2243] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.754940 1164 raft_consensus.cc:2272] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.755514 1164 ts_tablet_manager.cc:1916] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:22.756670 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 274c10c2aa46479dab5edbb483e5b883 (table default.Table_7a3a07a264fc4bed96455f83ae51fb05 [id=5a91ece54cfa446880f72a42678f4f26]) successfully deleted
W20251211 20:52:22.756944 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet aa685d1b051a4c93a2d313429d914ead because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aa685d1b051a4c93a2d313429d914ead already in progress: creating tablet
W20251211 20:52:22.757187 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet b815bca893a8446e92d48973d9a0408e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b815bca893a8446e92d48973d9a0408e already in progress: creating tablet
W20251211 20:52:22.757397 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 531a74c1fef34b2dba6d4a732e13ce38 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 531a74c1fef34b2dba6d4a732e13ce38 already in progress: creating tablet
I20251211 20:52:22.758069 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ce0d3130963649ef8ca062313c67f1cc" } new_table_name: "default.Table_4c2b2d7d11a347be9c1152d447467897" modify_external_catalogs: true
I20251211 20:52:22.760703 1023 tablet_service.cc:1505] Processing CreateTablet for tablet ef87e052644c41bbbc54b8dad68ce152 (DEFAULT_TABLE table=default.Table_9b96425fc6f34e15ab398770cb57bc98 [id=f8384ab62c6749b7b9c458e8e90d3c02]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.761333 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef87e052644c41bbbc54b8dad68ce152. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.765275 1165 ts_tablet_manager.cc:1929] T 638fee757fc444f8a76b5189c103c40c P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.766182 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_3fca4e8f1b6a427d8d016e27f88ea1ce" } modify_external_catalogs: true
I20251211 20:52:22.766415 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_3fca4e8f1b6a427d8d016e27f88ea1ce" } modify_external_catalogs: true
I20251211 20:52:22.766649 1165 log.cc:1199] T 638fee757fc444f8a76b5189c103c40c P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/638fee757fc444f8a76b5189c103c40c
I20251211 20:52:22.766994 1165 ts_tablet_manager.cc:1950] T 638fee757fc444f8a76b5189c103c40c P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:22.769574 892 tablet_service.cc:1505] Processing CreateTablet for tablet 4870da0224e048ed82f227d7ba676a94 (DEFAULT_TABLE table=default.Table_e0a4d8f6e8a84693b285f1f8e6de6e42 [id=28f3ebe9b7a64b5e89404b0407b0e5ae]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:22.770323 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 531a74c1fef34b2dba6d4a732e13ce38 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 531a74c1fef34b2dba6d4a732e13ce38 already in progress: creating tablet
W20251211 20:52:22.771186 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet b815bca893a8446e92d48973d9a0408e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b815bca893a8446e92d48973d9a0408e already in progress: creating tablet
W20251211 20:52:22.772006 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet edc4300dc5a84173aa95b587d77fc95c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet edc4300dc5a84173aa95b587d77fc95c already in progress: creating tablet
W20251211 20:52:22.772871 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 8e9eded009e44c1a9544d4471c04c5c8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e9eded009e44c1a9544d4471c04c5c8 already in progress: creating tablet
W20251211 20:52:22.772944 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 694518e6c1ae4032b06b99e9bde2e5dd consensus state has no leader
I20251211 20:52:22.773656 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet af3ca9ac988e458cb7068c10c006d587
W20251211 20:52:22.774045 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet aa685d1b051a4c93a2d313429d914ead because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aa685d1b051a4c93a2d313429d914ead already in progress: creating tablet
W20251211 20:52:22.774514 595 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 33e1d8a576cd41b1a34778b1458df396 consensus state has no leader
I20251211 20:52:22.774685 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet af3ca9ac988e458cb7068c10c006d587 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:22.774955 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 049cdecf12304211adbc1c1008dba989
I20251211 20:52:22.775102 750 tablet_service.cc:1552] Processing DeleteTablet for tablet af3ca9ac988e458cb7068c10c006d587 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:22.775977 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 049cdecf12304211adbc1c1008dba989 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:22.776243 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 049cdecf12304211adbc1c1008dba989 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:22.780068 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 5ed0a0141051462eb45ca3a66d41d009 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5ed0a0141051462eb45ca3a66d41d009 already in progress: creating tablet
I20251211 20:52:22.781162 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 4d98967714184fe6a3e38dd4ca7979dc (DEFAULT_TABLE table=default.Table_ebec874d437748c487f8bf9d0ccaa7fa [id=aea22a96d5b840399f2140486953c1ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.781601 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d98967714184fe6a3e38dd4ca7979dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.783272 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 034f47d8564f4094a9d76aee2fea3119 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:22.783524 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 8a064789f89c424ca698e52333d73eea with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:22.783733 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 9662efc97a53461d888a062463e80ee6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:22.784112 893 tablet_service.cc:1505] Processing CreateTablet for tablet d19975cc513e414b80f478a6c5eb09de (DEFAULT_TABLE table=default.Table_7eabd17b66f44d40aebe078e0404f66f [id=d770b26c251e41e38f9720b37852cb11]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.784586 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d19975cc513e414b80f478a6c5eb09de. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.788329 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_95dbc60f2f8f431d96180162033fd31c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.788982 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_95dbc60f2f8f431d96180162033fd31c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.791450 750 tablet_service.cc:1505] Processing CreateTablet for tablet ef87e052644c41bbbc54b8dad68ce152 (DEFAULT_TABLE table=default.Table_9b96425fc6f34e15ab398770cb57bc98 [id=f8384ab62c6749b7b9c458e8e90d3c02]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.792098 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef87e052644c41bbbc54b8dad68ce152. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.795650 744 tablet_service.cc:1505] Processing CreateTablet for tablet 4d98967714184fe6a3e38dd4ca7979dc (DEFAULT_TABLE table=default.Table_ebec874d437748c487f8bf9d0ccaa7fa [id=aea22a96d5b840399f2140486953c1ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.796067 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d98967714184fe6a3e38dd4ca7979dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.799166 1166 tablet_bootstrap.cc:492] T deacd1f8f3bf4b1f923b61b312492104 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:22.799350 1166 ts_tablet_manager.cc:1403] T deacd1f8f3bf4b1f923b61b312492104 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.130s user 0.030s sys 0.087s
I20251211 20:52:22.800076 1166 raft_consensus.cc:359] T deacd1f8f3bf4b1f923b61b312492104 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:22.800276 1166 raft_consensus.cc:385] T deacd1f8f3bf4b1f923b61b312492104 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:22.800349 1166 raft_consensus.cc:740] T deacd1f8f3bf4b1f923b61b312492104 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:22.800388 890 tablet_service.cc:1552] Processing DeleteTablet for tablet ec7df7b11ce34b8489048e6acdcb4e94 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:22.800573 1166 consensus_queue.cc:260] T deacd1f8f3bf4b1f923b61b312492104 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:22.803081 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 7f086c7c56fe4abb8df7d3c0978cce6e
I20251211 20:52:22.803813 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet e69d53ac90a7463aa36ff15dadad7e0b
W20251211 20:52:22.804433 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c58241f7e1404a8c8db92c4f9a47d143 consensus state has no leader
I20251211 20:52:22.804772 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 73aa2a3fb2f44bbca870fb7f4cf6087c
I20251211 20:52:22.805008 1166 ts_tablet_manager.cc:1434] T deacd1f8f3bf4b1f923b61b312492104 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20251211 20:52:22.806025 749 tablet_service.cc:1552] Processing DeleteTablet for tablet e69d53ac90a7463aa36ff15dadad7e0b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:22.806262 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 73aa2a3fb2f44bbca870fb7f4cf6087c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:22.806787 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 7f086c7c56fe4abb8df7d3c0978cce6e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:22.808089 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 73aa2a3fb2f44bbca870fb7f4cf6087c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:22.808185 1163 ts_tablet_manager.cc:1916] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:22.808527 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet e69d53ac90a7463aa36ff15dadad7e0b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:22.808871 1166 tablet_bootstrap.cc:492] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:22.810106 1165 tablet_replica.cc:333] T 12313c06b33749589ceb08ade798fc59 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:22.810297 1165 raft_consensus.cc:2243] T 12313c06b33749589ceb08ade798fc59 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.810456 1165 raft_consensus.cc:2272] T 12313c06b33749589ceb08ade798fc59 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.810617 1166 tablet_bootstrap.cc:654] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:22.812292 1165 ts_tablet_manager.cc:1916] T 12313c06b33749589ceb08ade798fc59 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:22.812985 1168 tablet_bootstrap.cc:492] T 72d7d2f78ab64bbebb63239b9ecb366e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:22.814253 1168 ts_tablet_manager.cc:1403] T 72d7d2f78ab64bbebb63239b9ecb366e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.157s user 0.015s sys 0.016s
I20251211 20:52:22.814900 1163 ts_tablet_manager.cc:1929] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.815047 1163 log.cc:1199] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/8e9eded009e44c1a9544d4471c04c5c8
I20251211 20:52:22.815282 1168 raft_consensus.cc:359] T 72d7d2f78ab64bbebb63239b9ecb366e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.815410 1163 ts_tablet_manager.cc:1950] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:22.815541 1168 raft_consensus.cc:385] T 72d7d2f78ab64bbebb63239b9ecb366e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:22.815711 1168 raft_consensus.cc:740] T 72d7d2f78ab64bbebb63239b9ecb366e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:22.815793 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet 7f086c7c56fe4abb8df7d3c0978cce6e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:22.815981 1168 consensus_queue.cc:260] T 72d7d2f78ab64bbebb63239b9ecb366e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.816448 1168 ts_tablet_manager.cc:1434] T 72d7d2f78ab64bbebb63239b9ecb366e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:22.821717 739 tablet_service.cc:1552] Processing DeleteTablet for tablet 72d7d2f78ab64bbebb63239b9ecb366e with delete_type TABLET_DATA_DELETED (replaced by tablet 8a064789f89c424ca698e52333d73eea) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:22.822158 1168 tablet_bootstrap.cc:492] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:22.823443 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 8e9eded009e44c1a9544d4471c04c5c8 (table default.Table_b225141042f247aca03de91c6a346513 [id=383392d835da4c50848068d83261cecc]) successfully deleted
W20251211 20:52:22.824074 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 297dc26ade384213b364a876c54bdd14 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 297dc26ade384213b364a876c54bdd14 already in progress: creating tablet
W20251211 20:52:22.824327 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet af3ca9ac988e458cb7068c10c006d587 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet af3ca9ac988e458cb7068c10c006d587 already in progress: creating tablet
W20251211 20:52:22.826936 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 049cdecf12304211adbc1c1008dba989 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 049cdecf12304211adbc1c1008dba989 already in progress: creating tablet
W20251211 20:52:22.827189 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet e69d53ac90a7463aa36ff15dadad7e0b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e69d53ac90a7463aa36ff15dadad7e0b already in progress: creating tablet
W20251211 20:52:22.827445 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 73aa2a3fb2f44bbca870fb7f4cf6087c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 73aa2a3fb2f44bbca870fb7f4cf6087c already in progress: creating tablet
W20251211 20:52:22.827917 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 7f086c7c56fe4abb8df7d3c0978cce6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7f086c7c56fe4abb8df7d3c0978cce6e already in progress: creating tablet
I20251211 20:52:22.829687 882 tablet_service.cc:1552] Processing DeleteTablet for tablet 297dc26ade384213b364a876c54bdd14 with delete_type TABLET_DATA_DELETED (replaced by tablet 8dff13e22899452c8d4f2731b8dac7d5) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:22.830531 1168 tablet_bootstrap.cc:654] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:22.830850 1166 tablet_bootstrap.cc:492] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:22.831001 1166 ts_tablet_manager.cc:1403] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.014s
I20251211 20:52:22.831439 875 tablet_service.cc:1552] Processing DeleteTablet for tablet af3ca9ac988e458cb7068c10c006d587 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:22.831831 1166 raft_consensus.cc:359] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:22.832015 1166 raft_consensus.cc:385] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:22.832103 1166 raft_consensus.cc:740] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:22.832343 1166 consensus_queue.cc:260] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:22.833271 1166 ts_tablet_manager.cc:1434] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251211 20:52:22.833631 1166 tablet_bootstrap.cc:492] T 62fb046960024a2b9d70dc46ae52d53f P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:22.834357 890 tablet_service.cc:1505] Processing CreateTablet for tablet ef87e052644c41bbbc54b8dad68ce152 (DEFAULT_TABLE table=default.Table_9b96425fc6f34e15ab398770cb57bc98 [id=f8384ab62c6749b7b9c458e8e90d3c02]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.835717 1166 tablet_bootstrap.cc:654] T 62fb046960024a2b9d70dc46ae52d53f P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:22.838061 1165 ts_tablet_manager.cc:1929] T 12313c06b33749589ceb08ade798fc59 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.838224 1165 log.cc:1199] T 12313c06b33749589ceb08ade798fc59 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/12313c06b33749589ceb08ade798fc59
I20251211 20:52:22.838573 1165 ts_tablet_manager.cc:1950] T 12313c06b33749589ceb08ade798fc59 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:22.838932 878 tablet_service.cc:1505] Processing CreateTablet for tablet 4d98967714184fe6a3e38dd4ca7979dc (DEFAULT_TABLE table=default.Table_ebec874d437748c487f8bf9d0ccaa7fa [id=aea22a96d5b840399f2140486953c1ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.839918 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 12313c06b33749589ceb08ade798fc59 (table default.Table_b37d05b3e8eb4425bb5cb72d46ce9e91 [id=4e18dbcc527d4b12ba6978ca125d5085]) successfully deleted
I20251211 20:52:22.840358 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4870da0224e048ed82f227d7ba676a94. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:22.840950 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet ec7df7b11ce34b8489048e6acdcb4e94 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ec7df7b11ce34b8489048e6acdcb4e94 already in progress: creating tablet
I20251211 20:52:22.841090 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 049cdecf12304211adbc1c1008dba989 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:22.842520 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 034f47d8564f4094a9d76aee2fea3119 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 034f47d8564f4094a9d76aee2fea3119 already in progress: creating tablet
I20251211 20:52:22.842988 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_5fa0a824d9234eaebcbbd089e47d3f75" } new_table_name: "default.Table_18d19a317a9c4656b4c764df4a64dbea" modify_external_catalogs: true
I20251211 20:52:22.843714 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_d65f65d608a346968cc2f35b9f55912e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.843859 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 8a064789f89c424ca698e52333d73eea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8a064789f89c424ca698e52333d73eea already in progress: creating tablet
W20251211 20:52:22.844323 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d65f65d608a346968cc2f35b9f55912e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:22.844659 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 9662efc97a53461d888a062463e80ee6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9662efc97a53461d888a062463e80ee6 already in progress: creating tablet
I20251211 20:52:22.845150 1165 tablet_replica.cc:333] T b67de345cf44441abe9e1431b64c4ba4 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:22.845170 887 tablet_service.cc:1505] Processing CreateTablet for tablet 33cce042f33b4be198d11fd39f5f8838 (DEFAULT_TABLE table=default.Table_e1406b8c64104f0a9d31ff37e042163e [id=5a7b1e562b084b109569a8733fd470a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.845575 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33cce042f33b4be198d11fd39f5f8838. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.845587 1165 raft_consensus.cc:2243] T b67de345cf44441abe9e1431b64c4ba4 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.845793 1165 raft_consensus.cc:2272] T b67de345cf44441abe9e1431b64c4ba4 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.846936 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_984c6c545e3a4a2ca496d202198a177b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.847561 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_984c6c545e3a4a2ca496d202198a177b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.848937 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_154a7364b43244ecb91070b153a641a6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.849588 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_154a7364b43244ecb91070b153a641a6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.849721 879 tablet_service.cc:1552] Processing DeleteTablet for tablet e69d53ac90a7463aa36ff15dadad7e0b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:22.853037 1164 ts_tablet_manager.cc:1929] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.853258 1164 log.cc:1199] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/cd13d4e7cf2c4abdbedbb5bbac5e05ce
I20251211 20:52:22.853956 876 tablet_service.cc:1552] Processing DeleteTablet for tablet 7f086c7c56fe4abb8df7d3c0978cce6e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:22.854074 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_64c6ddd508344b0ab2412c9872a28068"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:22.854229 879 tablet_service.cc:1552] Processing DeleteTablet for tablet 73aa2a3fb2f44bbca870fb7f4cf6087c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:22.854581 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_64c6ddd508344b0ab2412c9872a28068 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.855329 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef87e052644c41bbbc54b8dad68ce152. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.856509 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d98967714184fe6a3e38dd4ca7979dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.856940 1165 ts_tablet_manager.cc:1916] T b67de345cf44441abe9e1431b64c4ba4 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:22.860569 1164 ts_tablet_manager.cc:1950] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:22.861377 1164 tablet_replica.cc:333] T 49193f7bd8ac4f20a6ca980a9e1eb830 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:22.862622 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_f26ebdd2202a43c58d2e43dcf381d0aa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.863215 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f26ebdd2202a43c58d2e43dcf381d0aa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.864506 1164 raft_consensus.cc:2243] T 49193f7bd8ac4f20a6ca980a9e1eb830 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.864768 1164 raft_consensus.cc:2272] T 49193f7bd8ac4f20a6ca980a9e1eb830 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.865468 1164 ts_tablet_manager.cc:1916] T 49193f7bd8ac4f20a6ca980a9e1eb830 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:22.865667 1160 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.520s user 0.011s sys 0.003s
W20251211 20:52:22.865792 1160 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.533s user 0.011s sys 0.003s
I20251211 20:52:22.866679 1160 tablet_bootstrap.cc:492] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:22.866899 1160 ts_tablet_manager.cc:1403] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.577s user 0.013s sys 0.004s
I20251211 20:52:22.867833 1160 raft_consensus.cc:359] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.868108 1160 raft_consensus.cc:385] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:22.868362 1160 raft_consensus.cc:740] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:22.868685 1160 consensus_queue.cc:260] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:22.869298 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce (table default.Table_7ce0e2ed6ea4463cac10ba89d3718c5c [id=0e7104bb15484108b86dacf1f1f2c359]) successfully deleted
W20251211 20:52:22.869756 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 5ed0a0141051462eb45ca3a66d41d009 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5ed0a0141051462eb45ca3a66d41d009 already in progress: creating tablet
W20251211 20:52:22.870179 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 8e9eded009e44c1a9544d4471c04c5c8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e9eded009e44c1a9544d4471c04c5c8 already in progress: creating tablet
I20251211 20:52:22.871462 1166 tablet_bootstrap.cc:492] T 62fb046960024a2b9d70dc46ae52d53f P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:22.871640 1215 raft_consensus.cc:493] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:22.871707 1166 ts_tablet_manager.cc:1403] T 62fb046960024a2b9d70dc46ae52d53f P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.038s user 0.001s sys 0.009s
I20251211 20:52:22.871824 1215 raft_consensus.cc:515] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:22.872467 1215 leader_election.cc:290] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:22.873035 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dc7e8db4b51c415e86d8c24a76b8cb1e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:22.873680 982 leader_election.cc:343] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:22.874117 1166 raft_consensus.cc:359] T 62fb046960024a2b9d70dc46ae52d53f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:22.874317 1166 raft_consensus.cc:385] T 62fb046960024a2b9d70dc46ae52d53f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:22.874400 1166 raft_consensus.cc:740] T 62fb046960024a2b9d70dc46ae52d53f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:22.874694 1166 consensus_queue.cc:260] T 62fb046960024a2b9d70dc46ae52d53f P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:22.875509 1166 ts_tablet_manager.cc:1434] T 62fb046960024a2b9d70dc46ae52d53f P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:22.877930 1166 tablet_bootstrap.cc:492] T aa685d1b051a4c93a2d313429d914ead P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:22.878230 1165 ts_tablet_manager.cc:1929] T b67de345cf44441abe9e1431b64c4ba4 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.879809 1166 tablet_bootstrap.cc:654] T aa685d1b051a4c93a2d313429d914ead P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:22.880016 1165 log.cc:1199] T b67de345cf44441abe9e1431b64c4ba4 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/b67de345cf44441abe9e1431b64c4ba4
I20251211 20:52:22.880759 1165 ts_tablet_manager.cc:1950] T b67de345cf44441abe9e1431b64c4ba4 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:22.881955 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet b67de345cf44441abe9e1431b64c4ba4 (table default.Table_1b829e0ef7ef4b17be671db434f3858c [id=b8c401dfdb5f4f7584dca0989287ce31]) successfully deleted
W20251211 20:52:22.882335 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 297dc26ade384213b364a876c54bdd14 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 297dc26ade384213b364a876c54bdd14 already in progress: creating tablet
W20251211 20:52:22.884091 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet af3ca9ac988e458cb7068c10c006d587 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet af3ca9ac988e458cb7068c10c006d587 already in progress: creating tablet
W20251211 20:52:22.884779 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 049cdecf12304211adbc1c1008dba989 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 049cdecf12304211adbc1c1008dba989 already in progress: creating tablet
W20251211 20:52:22.885504 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet e69d53ac90a7463aa36ff15dadad7e0b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e69d53ac90a7463aa36ff15dadad7e0b already in progress: creating tablet
W20251211 20:52:22.886354 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 73aa2a3fb2f44bbca870fb7f4cf6087c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 73aa2a3fb2f44bbca870fb7f4cf6087c already in progress: creating tablet
W20251211 20:52:22.889005 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 7f086c7c56fe4abb8df7d3c0978cce6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7f086c7c56fe4abb8df7d3c0978cce6e already in progress: creating tablet
I20251211 20:52:22.889312 1165 tablet_replica.cc:333] T 72d7d2f78ab64bbebb63239b9ecb366e P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:22.889511 1165 raft_consensus.cc:2243] T 72d7d2f78ab64bbebb63239b9ecb366e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:22.889767 1165 raft_consensus.cc:2272] T 72d7d2f78ab64bbebb63239b9ecb366e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:22.890252 1166 tablet_bootstrap.cc:492] T aa685d1b051a4c93a2d313429d914ead P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:22.890437 1166 ts_tablet_manager.cc:1403] T aa685d1b051a4c93a2d313429d914ead P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.008s
I20251211 20:52:22.890714 1165 ts_tablet_manager.cc:1916] T 72d7d2f78ab64bbebb63239b9ecb366e P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:22.891477 1166 raft_consensus.cc:359] T aa685d1b051a4c93a2d313429d914ead P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:22.891711 1166 raft_consensus.cc:385] T aa685d1b051a4c93a2d313429d914ead P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:22.891808 1166 raft_consensus.cc:740] T aa685d1b051a4c93a2d313429d914ead P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:22.892084 1166 consensus_queue.cc:260] T aa685d1b051a4c93a2d313429d914ead P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:22.893006 1166 ts_tablet_manager.cc:1434] T aa685d1b051a4c93a2d313429d914ead P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251211 20:52:22.893453 1166 tablet_bootstrap.cc:492] T f37f598a720d4161b13c03425e020fa5 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:22.895447 1166 tablet_bootstrap.cc:654] T f37f598a720d4161b13c03425e020fa5 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:22.900633 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_afc299e5a13c43878cf6c746698af917"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.901314 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_afc299e5a13c43878cf6c746698af917 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.902722 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_2bdd381fdf6443da869b647ddb92699b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.903218 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2bdd381fdf6443da869b647ddb92699b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.903795 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_f2db815bce7c4d689a5ce01da255b14f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:22.904253 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f2db815bce7c4d689a5ce01da255b14f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.905495 1165 ts_tablet_manager.cc:1929] T 72d7d2f78ab64bbebb63239b9ecb366e P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:22.905726 1165 log.cc:1199] T 72d7d2f78ab64bbebb63239b9ecb366e P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/72d7d2f78ab64bbebb63239b9ecb366e
I20251211 20:52:22.908077 1165 ts_tablet_manager.cc:1950] T 72d7d2f78ab64bbebb63239b9ecb366e P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:22.913604 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 72d7d2f78ab64bbebb63239b9ecb366e (table default.Table_f439572c5b1d4997af54e0ab3c036023 [id=cda4c0ad8a0347be88a0518491e6af49]) successfully deleted
I20251211 20:52:22.915333 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8e9eded009e44c1a9544d4471c04c5c8" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:22.916024 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dc7e8db4b51c415e86d8c24a76b8cb1e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:22.917052 983 leader_election.cc:343] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Not found: Tablet not found: b67de345cf44441abe9e1431b64c4ba4
I20251211 20:52:22.917227 983 leader_election.cc:304] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:22.917763 983 raft_consensus.cc:2686] T b67de345cf44441abe9e1431b64c4ba4 P e715a811a82446ce8d5bb4aafd5b81c7: unable to run election callback: Service unavailable: Thread pool token was shut down
W20251211 20:52:22.918813 983 leader_election.cc:343] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:22.918967 983 leader_election.cc:304] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:22.919185 983 raft_consensus.cc:2686] T 8e9eded009e44c1a9544d4471c04c5c8 P e715a811a82446ce8d5bb4aafd5b81c7: unable to run election callback: Service unavailable: Thread pool token was shut down
W20251211 20:52:22.920855 983 leader_election.cc:343] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:22.921749 983 leader_election.cc:304] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:22.928300 1215 raft_consensus.cc:2749] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:22.968123 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_10dc3bcef378487d8d4ade7100dd7913"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:22.968623 739 tablet_service.cc:1505] Processing CreateTablet for tablet 2260850f32b541aab1128818f144b5a7 (DEFAULT_TABLE table=default.Table_d8c241666f7840b8bf0cb7672482aa55 [id=81bc0640c53f4710895d487f7c6be071]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:22.968737 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_10dc3bcef378487d8d4ade7100dd7913 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:22.969105 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2260850f32b541aab1128818f144b5a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.969955 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 2260850f32b541aab1128818f144b5a7 (DEFAULT_TABLE table=default.Table_d8c241666f7840b8bf0cb7672482aa55 [id=81bc0640c53f4710895d487f7c6be071]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.970329 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2260850f32b541aab1128818f144b5a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.976004 750 tablet_service.cc:1505] Processing CreateTablet for tablet 2433068ec179490dac3e91d046a42b70 (DEFAULT_TABLE table=default.Table_26b3b7e5db73457da38332d4e3d943f6 [id=18edd3deb100406cb5f421ee06216e6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.976466 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2433068ec179490dac3e91d046a42b70. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.976691 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 2433068ec179490dac3e91d046a42b70 (DEFAULT_TABLE table=default.Table_26b3b7e5db73457da38332d4e3d943f6 [id=18edd3deb100406cb5f421ee06216e6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.977137 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2433068ec179490dac3e91d046a42b70. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.979931 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 4ddf1bd1e18f4187bf050d6f7767b722 (DEFAULT_TABLE table=default.Table_242e7ff74c6742708701ef45a63199f3 [id=3ade2e0cb4924ec38113bf929b5f554b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.980090 744 tablet_service.cc:1505] Processing CreateTablet for tablet 4ddf1bd1e18f4187bf050d6f7767b722 (DEFAULT_TABLE table=default.Table_242e7ff74c6742708701ef45a63199f3 [id=3ade2e0cb4924ec38113bf929b5f554b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.980365 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ddf1bd1e18f4187bf050d6f7767b722. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.980463 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ddf1bd1e18f4187bf050d6f7767b722. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.983544 1024 tablet_service.cc:1505] Processing CreateTablet for tablet 4ef506349e5b434db7a78809dc44014e (DEFAULT_TABLE table=default.Table_fd4577173dc44659b95db8e463e0131c [id=143faf67e1204e7bb2015f06392e48a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.983996 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ef506349e5b434db7a78809dc44014e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.984483 754 tablet_service.cc:1505] Processing CreateTablet for tablet 4ef506349e5b434db7a78809dc44014e (DEFAULT_TABLE table=default.Table_fd4577173dc44659b95db8e463e0131c [id=143faf67e1204e7bb2015f06392e48a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.984900 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ef506349e5b434db7a78809dc44014e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.987941 1021 tablet_service.cc:1505] Processing CreateTablet for tablet 0129162969d74f5da3c08a1e6a27ab22 (DEFAULT_TABLE table=default.Table_daeaf2f3dd24423e9060e4f09a5f4c99 [id=755ac98365be4def8589dcfd1f3e2cbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:22.988473 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0129162969d74f5da3c08a1e6a27ab22. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:22.990934 1236 raft_consensus.cc:493] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:23.001995 595 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_104f49b10d1d45749449d0ff3009e1d9" } new_table_name: "default.Table_28b427311c9b4699a47c38ebb4d93e01" modify_external_catalogs: true
I20251211 20:52:23.002692 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet 62fb046960024a2b9d70dc46ae52d53f with delete_type TABLET_DATA_DELETED (replaced by tablet ec7df7b11ce34b8489048e6acdcb4e94) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.003021 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet dc7e8db4b51c415e86d8c24a76b8cb1e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.003257 1163 tablet_replica.cc:333] T 62fb046960024a2b9d70dc46ae52d53f P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:23.003456 1163 raft_consensus.cc:2243] T 62fb046960024a2b9d70dc46ae52d53f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.003535 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet aa685d1b051a4c93a2d313429d914ead with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.003568 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_26b3b7e5db73457da38332d4e3d943f6" } modify_external_catalogs: true
I20251211 20:52:23.003674 1163 raft_consensus.cc:2272] T 62fb046960024a2b9d70dc46ae52d53f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.003803 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_26b3b7e5db73457da38332d4e3d943f6" } modify_external_catalogs: true
I20251211 20:52:23.002487 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_242e7ff74c6742708701ef45a63199f3" } new_table_name: "default.Table_307027e1160747b98c2fb2ea31eb8745" modify_external_catalogs: true
I20251211 20:52:23.005043 1163 ts_tablet_manager.cc:1916] T 62fb046960024a2b9d70dc46ae52d53f P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.005443 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b225141042f247aca03de91c6a346513" } new_table_name: "default.Table_19db6b94abda4124a0e45f11b8ca0990" modify_external_catalogs: true
W20251211 20:52:23.007980 591 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 01ada50100bc48e39d1bc67082da750e consensus state has no leader
I20251211 20:52:23.011366 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a2a9717ed78046cab7c00077ac646665" } modify_external_catalogs: true
I20251211 20:52:23.011374 884 tablet_service.cc:1505] Processing CreateTablet for tablet 0129162969d74f5da3c08a1e6a27ab22 (DEFAULT_TABLE table=default.Table_daeaf2f3dd24423e9060e4f09a5f4c99 [id=755ac98365be4def8589dcfd1f3e2cbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.011603 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a2a9717ed78046cab7c00077ac646665" } modify_external_catalogs: true
I20251211 20:52:23.011883 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0129162969d74f5da3c08a1e6a27ab22. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.017764 1163 ts_tablet_manager.cc:1929] T 62fb046960024a2b9d70dc46ae52d53f P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.019127 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_fd4577173dc44659b95db8e463e0131c" } modify_external_catalogs: true
I20251211 20:52:23.019393 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_fd4577173dc44659b95db8e463e0131c" } modify_external_catalogs: true
I20251211 20:52:23.021004 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9b96425fc6f34e15ab398770cb57bc98" } modify_external_catalogs: true
I20251211 20:52:23.021185 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9b96425fc6f34e15ab398770cb57bc98" } modify_external_catalogs: true
I20251211 20:52:23.022634 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_872ae36ce97749d39238287355dadbe1" } modify_external_catalogs: true
I20251211 20:52:23.022843 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_872ae36ce97749d39238287355dadbe1" } modify_external_catalogs: true
I20251211 20:52:23.024235 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_204572508f5643e6a0c5decb479b3932" } new_table_name: "default.Table_39124c0c70034d00818c18db160416cf" modify_external_catalogs: true
I20251211 20:52:23.027472 754 tablet_service.cc:1505] Processing CreateTablet for tablet 0129162969d74f5da3c08a1e6a27ab22 (DEFAULT_TABLE table=default.Table_daeaf2f3dd24423e9060e4f09a5f4c99 [id=755ac98365be4def8589dcfd1f3e2cbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.028000 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0129162969d74f5da3c08a1e6a27ab22. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.030802 880 tablet_service.cc:1505] Processing CreateTablet for tablet 4ef506349e5b434db7a78809dc44014e (DEFAULT_TABLE table=default.Table_fd4577173dc44659b95db8e463e0131c [id=143faf67e1204e7bb2015f06392e48a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.033588 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ef506349e5b434db7a78809dc44014e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.034258 893 tablet_service.cc:1505] Processing CreateTablet for tablet 4ddf1bd1e18f4187bf050d6f7767b722 (DEFAULT_TABLE table=default.Table_242e7ff74c6742708701ef45a63199f3 [id=3ade2e0cb4924ec38113bf929b5f554b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.044950 894 tablet_service.cc:1505] Processing CreateTablet for tablet 8dff13e22899452c8d4f2731b8dac7d5 (DEFAULT_TABLE table=default.Table_a0c567f392784be39ceb0176cf58cff8 [id=cb97ccf262fd4c41a180267fd0678cec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.036769 878 tablet_service.cc:1505] Processing CreateTablet for tablet 2433068ec179490dac3e91d046a42b70 (DEFAULT_TABLE table=default.Table_26b3b7e5db73457da38332d4e3d943f6 [id=18edd3deb100406cb5f421ee06216e6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.046186 890 tablet_service.cc:1505] Processing CreateTablet for tablet 2260850f32b541aab1128818f144b5a7 (DEFAULT_TABLE table=default.Table_d8c241666f7840b8bf0cb7672482aa55 [id=81bc0640c53f4710895d487f7c6be071]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.049458 1163 log.cc:1199] T 62fb046960024a2b9d70dc46ae52d53f P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/62fb046960024a2b9d70dc46ae52d53f
I20251211 20:52:23.049439 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_4a3a3fe6b0294c939ebc90871cbcb4de"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:23.049906 1163 ts_tablet_manager.cc:1950] T 62fb046960024a2b9d70dc46ae52d53f P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
W20251211 20:52:23.050136 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4a3a3fe6b0294c939ebc90871cbcb4de in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:23.051051 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 62fb046960024a2b9d70dc46ae52d53f (table default.Table_02014bfadb7b434f960168f2db70b838 [id=14a0dbb8486246b09f3e06fc2120cd28]) successfully deleted
I20251211 20:52:23.047860 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 8dff13e22899452c8d4f2731b8dac7d5 (DEFAULT_TABLE table=default.Table_a0c567f392784be39ceb0176cf58cff8 [id=cb97ccf262fd4c41a180267fd0678cec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.051645 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dff13e22899452c8d4f2731b8dac7d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.049207 1021 tablet_service.cc:1505] Processing CreateTablet for tablet c5b164dc0ce1461fae98d950f7c7a726 (DEFAULT_TABLE table=default.Table_b06a3971c3be404e9787e05466931423 [id=9c24845e4efe475fa792a51ce8a66be6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.056981 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5b164dc0ce1461fae98d950f7c7a726. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.061385 754 tablet_service.cc:1505] Processing CreateTablet for tablet 8dff13e22899452c8d4f2731b8dac7d5 (DEFAULT_TABLE table=default.Table_a0c567f392784be39ceb0176cf58cff8 [id=cb97ccf262fd4c41a180267fd0678cec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.061590 750 tablet_service.cc:1505] Processing CreateTablet for tablet c5b164dc0ce1461fae98d950f7c7a726 (DEFAULT_TABLE table=default.Table_b06a3971c3be404e9787e05466931423 [id=9c24845e4efe475fa792a51ce8a66be6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.061879 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dff13e22899452c8d4f2731b8dac7d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.062180 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5b164dc0ce1461fae98d950f7c7a726. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.063441 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 732c76659c0540bc8b6909b430ffb146
W20251211 20:52:23.063874 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4ddf1bd1e18f4187bf050d6f7767b722 consensus state has no leader
I20251211 20:52:23.064906 739 tablet_service.cc:1552] Processing DeleteTablet for tablet 732c76659c0540bc8b6909b430ffb146 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.064926 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 732c76659c0540bc8b6909b430ffb146 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:23.065666 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 732c76659c0540bc8b6909b430ffb146 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 732c76659c0540bc8b6909b430ffb146 already in progress: creating tablet
W20251211 20:52:23.066262 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 97a519a8ca7c4f5380a230a70ee84052 consensus state has no leader
W20251211 20:52:23.067268 595 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet acdb7dddf3e74ce1bc92f232b6180cca consensus state has no leader
I20251211 20:52:23.067888 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 2433068ec179490dac3e91d046a42b70
I20251211 20:52:23.069222 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 2433068ec179490dac3e91d046a42b70 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:23.069458 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1951a67964834152afb31f95741d981a consensus state has no leader
I20251211 20:52:23.070132 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 2433068ec179490dac3e91d046a42b70 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.070717 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet ef87e052644c41bbbc54b8dad68ce152
I20251211 20:52:23.070643 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet deacd1f8f3bf4b1f923b61b312492104
W20251211 20:52:23.070900 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 2433068ec179490dac3e91d046a42b70 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2433068ec179490dac3e91d046a42b70 already in progress: creating tablet
I20251211 20:52:23.072628 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet ef87e052644c41bbbc54b8dad68ce152 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.072048 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 4ef506349e5b434db7a78809dc44014e
I20251211 20:52:23.074908 739 tablet_service.cc:1552] Processing DeleteTablet for tablet ef87e052644c41bbbc54b8dad68ce152 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.075124 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet deacd1f8f3bf4b1f923b61b312492104 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.075876 739 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ef506349e5b434db7a78809dc44014e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:23.075906 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet ef87e052644c41bbbc54b8dad68ce152 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ef87e052644c41bbbc54b8dad68ce152 already in progress: creating tablet
I20251211 20:52:23.076538 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ef506349e5b434db7a78809dc44014e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:23.076545 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 4ef506349e5b434db7a78809dc44014e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ef506349e5b434db7a78809dc44014e already in progress: creating tablet
I20251211 20:52:23.078125 1163 tablet_replica.cc:333] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:23.078354 1163 raft_consensus.cc:2243] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.078570 1163 raft_consensus.cc:2272] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.079990 1163 ts_tablet_manager.cc:1916] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.083817 750 tablet_service.cc:1552] Processing DeleteTablet for tablet deacd1f8f3bf4b1f923b61b312492104 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:23.084637 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet deacd1f8f3bf4b1f923b61b312492104 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet deacd1f8f3bf4b1f923b61b312492104 already in progress: creating tablet
I20251211 20:52:23.087522 592 master_service.cc:954] ReplaceTablet: received request to replace tablet 297dc26ade384213b364a876c54bdd14 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:23.046988 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ddf1bd1e18f4187bf050d6f7767b722. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.060221 891 tablet_service.cc:1505] Processing CreateTablet for tablet c5b164dc0ce1461fae98d950f7c7a726 (DEFAULT_TABLE table=default.Table_b06a3971c3be404e9787e05466931423 [id=9c24845e4efe475fa792a51ce8a66be6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.066344 892 tablet_service.cc:1552] Processing DeleteTablet for tablet 732c76659c0540bc8b6909b430ffb146 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:23.069914 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 2433068ec179490dac3e91d046a42b70 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:23.077067 883 tablet_service.cc:1552] Processing DeleteTablet for tablet deacd1f8f3bf4b1f923b61b312492104 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:23.079103 881 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ef506349e5b434db7a78809dc44014e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:23.084700 889 tablet_service.cc:1552] Processing DeleteTablet for tablet ef87e052644c41bbbc54b8dad68ce152 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:23.094484 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5b164dc0ce1461fae98d950f7c7a726. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.095741 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dff13e22899452c8d4f2731b8dac7d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.084873 1160 ts_tablet_manager.cc:1434] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.218s user 0.002s sys 0.004s
I20251211 20:52:23.105306 750 tablet_service.cc:1505] Processing CreateTablet for tablet 1c04187b8ed146c289a6b17d62b07199 (DEFAULT_TABLE table=default.Table_2bdd381fdf6443da869b647ddb92699b [id=70ed017f68a140ceaa39557fbe86aa0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.105952 739 tablet_service.cc:1505] Processing CreateTablet for tablet 5fc8d485cc234fd490c487a8bc881bf4 (DEFAULT_TABLE table=default.Table_f2db815bce7c4d689a5ce01da255b14f [id=d9453cea23a8430c98a3317856b3cec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.106688 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c04187b8ed146c289a6b17d62b07199. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.107195 881 tablet_service.cc:1505] Processing CreateTablet for tablet f00f7c1f6a4e44ff89341b4a55cc93fb (DEFAULT_TABLE table=default.Table_64c6ddd508344b0ab2412c9872a28068 [id=ea12bdaa894f446da22635f95d97ac2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.107790 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fc8d485cc234fd490c487a8bc881bf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.107913 744 tablet_service.cc:1505] Processing CreateTablet for tablet f00f7c1f6a4e44ff89341b4a55cc93fb (DEFAULT_TABLE table=default.Table_64c6ddd508344b0ab2412c9872a28068 [id=ea12bdaa894f446da22635f95d97ac2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.107908 1236 raft_consensus.cc:515] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:23.108310 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f00f7c1f6a4e44ff89341b4a55cc93fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.108561 889 tablet_service.cc:1505] Processing CreateTablet for tablet 1c04187b8ed146c289a6b17d62b07199 (DEFAULT_TABLE table=default.Table_2bdd381fdf6443da869b647ddb92699b [id=70ed017f68a140ceaa39557fbe86aa0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.108851 1236 leader_election.cc:290] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:23.109030 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2433068ec179490dac3e91d046a42b70. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.109496 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c04187b8ed146c289a6b17d62b07199. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.118899 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f00f7c1f6a4e44ff89341b4a55cc93fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.122007 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 5fc8d485cc234fd490c487a8bc881bf4 (DEFAULT_TABLE table=default.Table_f2db815bce7c4d689a5ce01da255b14f [id=d9453cea23a8430c98a3317856b3cec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.122536 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fc8d485cc234fd490c487a8bc881bf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.124043 1021 tablet_service.cc:1505] Processing CreateTablet for tablet 1c04187b8ed146c289a6b17d62b07199 (DEFAULT_TABLE table=default.Table_2bdd381fdf6443da869b647ddb92699b [id=70ed017f68a140ceaa39557fbe86aa0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.126178 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c04187b8ed146c289a6b17d62b07199. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.126329 1160 tablet_bootstrap.cc:492] T 8e9eded009e44c1a9544d4471c04c5c8 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:23.128095 1160 tablet_bootstrap.cc:654] T 8e9eded009e44c1a9544d4471c04c5c8 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.129271 894 tablet_service.cc:1505] Processing CreateTablet for tablet 5fc8d485cc234fd490c487a8bc881bf4 (DEFAULT_TABLE table=default.Table_f2db815bce7c4d689a5ce01da255b14f [id=d9453cea23a8430c98a3317856b3cec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.130523 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fc8d485cc234fd490c487a8bc881bf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.131803 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2260850f32b541aab1128818f144b5a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.132283 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ae3be9d91e5d4551bc706b2043a9edf5" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:23.136889 592 master_service.cc:954] ReplaceTablet: received request to replace tablet ec7df7b11ce34b8489048e6acdcb4e94 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:23.138960 1164 ts_tablet_manager.cc:1929] T 49193f7bd8ac4f20a6ca980a9e1eb830 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.139119 1164 log.cc:1199] T 49193f7bd8ac4f20a6ca980a9e1eb830 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/49193f7bd8ac4f20a6ca980a9e1eb830
I20251211 20:52:23.139546 1164 ts_tablet_manager.cc:1950] T 49193f7bd8ac4f20a6ca980a9e1eb830 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:23.140661 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 49193f7bd8ac4f20a6ca980a9e1eb830 (table default.Table_40256d3973c74566a09b1d235062d284 [id=1f4650c549d94e2c9ef753ff96670aa6]) successfully deleted
I20251211 20:52:23.145684 1163 ts_tablet_manager.cc:1929] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.145848 1163 log.cc:1199] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/dc7e8db4b51c415e86d8c24a76b8cb1e
I20251211 20:52:23.146203 1168 tablet_bootstrap.cc:492] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:23.146240 1163 ts_tablet_manager.cc:1950] T dc7e8db4b51c415e86d8c24a76b8cb1e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:23.146382 1168 ts_tablet_manager.cc:1403] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.324s user 0.064s sys 0.134s
I20251211 20:52:23.146603 592 master_service.cc:954] ReplaceTablet: received request to replace tablet ec7df7b11ce34b8489048e6acdcb4e94 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:23.146803 596 master_service.cc:954] ReplaceTablet: received request to replace tablet d703a5a42b2f47d0a7ffe1bb534a1b05 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:23.147172 1168 raft_consensus.cc:359] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:23.147367 1168 raft_consensus.cc:385] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.147437 1168 raft_consensus.cc:740] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.147830 1168 consensus_queue.cc:260] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:23.148417 1021 tablet_service.cc:1505] Processing CreateTablet for tablet f00f7c1f6a4e44ff89341b4a55cc93fb (DEFAULT_TABLE table=default.Table_64c6ddd508344b0ab2412c9872a28068 [id=ea12bdaa894f446da22635f95d97ac2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.148905 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f00f7c1f6a4e44ff89341b4a55cc93fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.151844 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet dc7e8db4b51c415e86d8c24a76b8cb1e (table default.Table_5ef3d8d2746a4016ad8efac84ba492b5 [id=ebae22b7d617403c84c023d0628f8ad8]) successfully deleted
I20251211 20:52:23.152968 1163 tablet_replica.cc:333] T aa685d1b051a4c93a2d313429d914ead P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:23.153147 1163 raft_consensus.cc:2243] T aa685d1b051a4c93a2d313429d914ead P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.153326 1163 raft_consensus.cc:2272] T aa685d1b051a4c93a2d313429d914ead P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.153349 1168 ts_tablet_manager.cc:1434] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20251211 20:52:23.154600 1163 ts_tablet_manager.cc:1916] T aa685d1b051a4c93a2d313429d914ead P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.154800 1168 tablet_bootstrap.cc:492] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:23.157019 1168 tablet_bootstrap.cc:654] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.158039 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet d703a5a42b2f47d0a7ffe1bb534a1b05
I20251211 20:52:23.159107 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet d703a5a42b2f47d0a7ffe1bb534a1b05 with delete_type TABLET_DATA_DELETED (replaced by tablet bd8fd6c977204c90b558827fc0b15f43) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.162508 588 master_service.cc:954] ReplaceTablet: received request to replace tablet edc4300dc5a84173aa95b587d77fc95c from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:23.166349 1163 ts_tablet_manager.cc:1929] T aa685d1b051a4c93a2d313429d914ead P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.166503 1163 log.cc:1199] T aa685d1b051a4c93a2d313429d914ead P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/aa685d1b051a4c93a2d313429d914ead
I20251211 20:52:23.166837 1163 ts_tablet_manager.cc:1950] T aa685d1b051a4c93a2d313429d914ead P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:23.168020 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet aa685d1b051a4c93a2d313429d914ead (table default.Table_ce67c2bcef6d4a6baecc11975d852ba8 [id=f81bfe3e109f48ce86e40e77dcbb30e1]) successfully deleted
W20251211 20:52:23.168326 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 732c76659c0540bc8b6909b430ffb146 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 732c76659c0540bc8b6909b430ffb146 already in progress: creating tablet
I20251211 20:52:23.169785 750 tablet_service.cc:1552] Processing DeleteTablet for tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.170018 596 catalog_manager.cc:6724] ReplaceTablet: tablet d703a5a42b2f47d0a7ffe1bb534a1b05 deleted and replaced by tablet bd8fd6c977204c90b558827fc0b15f43
I20251211 20:52:23.170113 1165 tablet_replica.cc:333] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:23.170276 1165 raft_consensus.cc:2243] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.170466 1165 raft_consensus.cc:2272] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.171002 750 tablet_service.cc:1552] Processing DeleteTablet for tablet d703a5a42b2f47d0a7ffe1bb534a1b05 with delete_type TABLET_DATA_DELETED (replaced by tablet bd8fd6c977204c90b558827fc0b15f43) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.171325 1165 ts_tablet_manager.cc:1916] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.172171 750 tablet_service.cc:1505] Processing CreateTablet for tablet 27466459d8e6411ab87ac474fb617735 (DEFAULT_TABLE table=default.Table_95dbc60f2f8f431d96180162033fd31c [id=18f8d722faaa4293a112525ff0c96216]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.172802 1021 tablet_service.cc:1505] Processing CreateTablet for tablet d824bc02b25842d18f28f711eef6669f (DEFAULT_TABLE table=default.Table_984c6c545e3a4a2ca496d202198a177b [id=861123c9df6d4841a078e84897b4ce35]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.173095 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 27466459d8e6411ab87ac474fb617735 (DEFAULT_TABLE table=default.Table_95dbc60f2f8f431d96180162033fd31c [id=18f8d722faaa4293a112525ff0c96216]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.173622 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27466459d8e6411ab87ac474fb617735. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.173738 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27466459d8e6411ab87ac474fb617735. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.174961 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d824bc02b25842d18f28f711eef6669f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.175019 1029 tablet_service.cc:1505] Processing CreateTablet for tablet dd425601d89148eb95525f796a3abf1e (DEFAULT_TABLE table=default.Table_10dc3bcef378487d8d4ade7100dd7913 [id=c842792b01cb4c689fcfd3d796151508]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.175447 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd425601d89148eb95525f796a3abf1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.179600 884 tablet_service.cc:1552] Processing DeleteTablet for tablet d703a5a42b2f47d0a7ffe1bb534a1b05 with delete_type TABLET_DATA_DELETED (replaced by tablet bd8fd6c977204c90b558827fc0b15f43) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:23.180351 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet edc4300dc5a84173aa95b587d77fc95c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet edc4300dc5a84173aa95b587d77fc95c already in progress: creating tablet
I20251211 20:52:23.180601 893 tablet_service.cc:1505] Processing CreateTablet for tablet 8308ad46551447439a248d20ed1e5922 (DEFAULT_TABLE table=default.Table_154a7364b43244ecb91070b153a641a6 [id=17b2359416c04980857565ab4df594bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.181242 878 tablet_service.cc:1505] Processing CreateTablet for tablet 27466459d8e6411ab87ac474fb617735 (DEFAULT_TABLE table=default.Table_95dbc60f2f8f431d96180162033fd31c [id=18f8d722faaa4293a112525ff0c96216]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:23.183704 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 2433068ec179490dac3e91d046a42b70 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2433068ec179490dac3e91d046a42b70 already in progress: creating tablet
I20251211 20:52:23.183684 891 tablet_service.cc:1505] Processing CreateTablet for tablet d824bc02b25842d18f28f711eef6669f (DEFAULT_TABLE table=default.Table_984c6c545e3a4a2ca496d202198a177b [id=861123c9df6d4841a078e84897b4ce35]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.184285 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8308ad46551447439a248d20ed1e5922. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:23.185774 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet ef87e052644c41bbbc54b8dad68ce152 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ef87e052644c41bbbc54b8dad68ce152 already in progress: creating tablet
I20251211 20:52:23.186249 1163 tablet_replica.cc:333] T deacd1f8f3bf4b1f923b61b312492104 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:23.186484 1163 raft_consensus.cc:2243] T deacd1f8f3bf4b1f923b61b312492104 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.186502 1164 tablet_replica.cc:333] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:23.186851 1163 raft_consensus.cc:2272] T deacd1f8f3bf4b1f923b61b312492104 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.186853 1164 raft_consensus.cc:2243] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.187397 1164 raft_consensus.cc:2272] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.187624 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d824bc02b25842d18f28f711eef6669f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.187970 884 tablet_service.cc:1505] Processing CreateTablet for tablet dd425601d89148eb95525f796a3abf1e (DEFAULT_TABLE table=default.Table_10dc3bcef378487d8d4ade7100dd7913 [id=c842792b01cb4c689fcfd3d796151508]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.189453 1163 ts_tablet_manager.cc:1916] T deacd1f8f3bf4b1f923b61b312492104 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.197783 1021 tablet_service.cc:1505] Processing CreateTablet for tablet 8308ad46551447439a248d20ed1e5922 (DEFAULT_TABLE table=default.Table_154a7364b43244ecb91070b153a641a6 [id=17b2359416c04980857565ab4df594bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.198204 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8308ad46551447439a248d20ed1e5922. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.198346 1166 tablet_bootstrap.cc:492] T f37f598a720d4161b13c03425e020fa5 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:23.198582 1166 ts_tablet_manager.cc:1403] T f37f598a720d4161b13c03425e020fa5 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.305s user 0.033s sys 0.079s
I20251211 20:52:23.199977 1166 raft_consensus.cc:359] T f37f598a720d4161b13c03425e020fa5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.200317 1166 raft_consensus.cc:385] T f37f598a720d4161b13c03425e020fa5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.200470 1166 raft_consensus.cc:740] T f37f598a720d4161b13c03425e020fa5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.200819 1166 consensus_queue.cc:260] T f37f598a720d4161b13c03425e020fa5 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.204164 1164 ts_tablet_manager.cc:1916] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.205178 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27466459d8e6411ab87ac474fb617735. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.205309 1166 ts_tablet_manager.cc:1434] T f37f598a720d4161b13c03425e020fa5 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20251211 20:52:23.205845 744 tablet_service.cc:1505] Processing CreateTablet for tablet dd425601d89148eb95525f796a3abf1e (DEFAULT_TABLE table=default.Table_10dc3bcef378487d8d4ade7100dd7913 [id=c842792b01cb4c689fcfd3d796151508]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.206246 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd425601d89148eb95525f796a3abf1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.208843 880 tablet_service.cc:1505] Processing CreateTablet for tablet 7474a08c51cf4346ae9a518963d56b7a (DEFAULT_TABLE table=default.Table_d65f65d608a346968cc2f35b9f55912e [id=88e8b1b4524846e18f5a638dee69a688]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.210098 741 tablet_service.cc:1505] Processing CreateTablet for tablet 7474a08c51cf4346ae9a518963d56b7a (DEFAULT_TABLE table=default.Table_d65f65d608a346968cc2f35b9f55912e [id=88e8b1b4524846e18f5a638dee69a688]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.208804 889 tablet_service.cc:1505] Processing CreateTablet for tablet ed9ee283045a44978422cd080aa2cda5 (DEFAULT_TABLE table=default.Table_f26ebdd2202a43c58d2e43dcf381d0aa [id=33c5cbdd3d004439ab01b4ae69db56d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.210279 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 91602f30b4464e35b16fa3dcac445b49 (DEFAULT_TABLE table=default.Table_afc299e5a13c43878cf6c746698af917 [id=6534b4b4514f4763bcff70c04d93ac7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.210489 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7474a08c51cf4346ae9a518963d56b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.210713 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91602f30b4464e35b16fa3dcac445b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.211866 749 tablet_service.cc:1505] Processing CreateTablet for tablet ed9ee283045a44978422cd080aa2cda5 (DEFAULT_TABLE table=default.Table_f26ebdd2202a43c58d2e43dcf381d0aa [id=33c5cbdd3d004439ab01b4ae69db56d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.211994 1021 tablet_service.cc:1505] Processing CreateTablet for tablet 7474a08c51cf4346ae9a518963d56b7a (DEFAULT_TABLE table=default.Table_d65f65d608a346968cc2f35b9f55912e [id=88e8b1b4524846e18f5a638dee69a688]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.212220 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed9ee283045a44978422cd080aa2cda5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.212411 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7474a08c51cf4346ae9a518963d56b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.215893 752 tablet_service.cc:1505] Processing CreateTablet for tablet 91602f30b4464e35b16fa3dcac445b49 (DEFAULT_TABLE table=default.Table_afc299e5a13c43878cf6c746698af917 [id=6534b4b4514f4763bcff70c04d93ac7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.216296 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91602f30b4464e35b16fa3dcac445b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.217043 1166 tablet_bootstrap.cc:492] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:23.218806 1166 tablet_bootstrap.cc:654] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.220558 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_03fdaef4fcc144f485f238db0e76fab4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:23.221167 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_03fdaef4fcc144f485f238db0e76fab4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:23.221890 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_64c6ddd508344b0ab2412c9872a28068" } new_table_name: "default.Table_b06f9587f78645bb9712c3b080f3c0d4" modify_external_catalogs: true
I20251211 20:52:23.224999 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_880a8408f1fb4d68a3f9d30b26acffa5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:23.225513 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_88975c1ad9604f50b7f69fbbdacabce3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:23.225654 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_880a8408f1fb4d68a3f9d30b26acffa5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:23.226003 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_88975c1ad9604f50b7f69fbbdacabce3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:23.226949 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_8352c9da1e324dd4847fd22078f22e53"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:23.227119 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_736d6c2eb4dd4eb9a72dca6b9ed2073f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:23.227418 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8352c9da1e324dd4847fd22078f22e53 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:23.227569 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_736d6c2eb4dd4eb9a72dca6b9ed2073f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:23.228590 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_9bec3557f8134357af09573f28abb7ad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:23.228662 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_028df988934346debc0f8c62b4740309"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:23.229027 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9bec3557f8134357af09573f28abb7ad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:23.229085 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_028df988934346debc0f8c62b4740309 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:23.230172 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_943bd9d0a2d848e2bb986aee9bc1d718"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:23.230350 754 tablet_service.cc:1505] Processing CreateTablet for tablet d824bc02b25842d18f28f711eef6669f (DEFAULT_TABLE table=default.Table_984c6c545e3a4a2ca496d202198a177b [id=861123c9df6d4841a078e84897b4ce35]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:23.230602 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_943bd9d0a2d848e2bb986aee9bc1d718 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:23.231781 739 tablet_service.cc:1505] Processing CreateTablet for tablet 8308ad46551447439a248d20ed1e5922 (DEFAULT_TABLE table=default.Table_154a7364b43244ecb91070b153a641a6 [id=17b2359416c04980857565ab4df594bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.232272 1163 ts_tablet_manager.cc:1929] T deacd1f8f3bf4b1f923b61b312492104 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.232416 1163 log.cc:1199] T deacd1f8f3bf4b1f923b61b312492104 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/deacd1f8f3bf4b1f923b61b312492104
I20251211 20:52:23.232764 1163 ts_tablet_manager.cc:1950] T deacd1f8f3bf4b1f923b61b312492104 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:23.233982 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet deacd1f8f3bf4b1f923b61b312492104 (table default.Table_872ae36ce97749d39238287355dadbe1 [id=98bd7ff031214ba7a5e974da798e1248]) successfully deleted
W20251211 20:52:23.234755 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 4ef506349e5b434db7a78809dc44014e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ef506349e5b434db7a78809dc44014e already in progress: creating tablet
W20251211 20:52:23.235440 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet d703a5a42b2f47d0a7ffe1bb534a1b05 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d703a5a42b2f47d0a7ffe1bb534a1b05 already in progress: creating tablet
I20251211 20:52:23.239886 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d824bc02b25842d18f28f711eef6669f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.240062 1166 tablet_bootstrap.cc:492] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:23.240288 1166 ts_tablet_manager.cc:1403] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.004s
I20251211 20:52:23.241776 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8308ad46551447439a248d20ed1e5922. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.241505 1166 raft_consensus.cc:359] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.241885 1166 raft_consensus.cc:385] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.241969 1166 raft_consensus.cc:740] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.242256 1166 consensus_queue.cc:260] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.243525 1168 tablet_bootstrap.cc:492] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:23.243721 1168 ts_tablet_manager.cc:1403] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.089s user 0.001s sys 0.003s
I20251211 20:52:23.245513 1168 raft_consensus.cc:359] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:23.245711 1168 raft_consensus.cc:385] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.246244 1166 ts_tablet_manager.cc:1434] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20251211 20:52:23.247494 1166 tablet_bootstrap.cc:492] T d703a5a42b2f47d0a7ffe1bb534a1b05 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:23.248960 1029 tablet_service.cc:1505] Processing CreateTablet for tablet ed9ee283045a44978422cd080aa2cda5 (DEFAULT_TABLE table=default.Table_f26ebdd2202a43c58d2e43dcf381d0aa [id=33c5cbdd3d004439ab01b4ae69db56d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.249454 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed9ee283045a44978422cd080aa2cda5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.249656 1166 tablet_bootstrap.cc:654] T d703a5a42b2f47d0a7ffe1bb534a1b05 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.254061 1168 raft_consensus.cc:740] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.254388 1168 consensus_queue.cc:260] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:23.254945 1165 ts_tablet_manager.cc:1929] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.255654 1165 log.cc:1199] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/cd13d4e7cf2c4abdbedbb5bbac5e05ce
I20251211 20:52:23.256973 1168 ts_tablet_manager.cc:1434] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.011s user 0.001s sys 0.000s
I20251211 20:52:23.257450 1168 tablet_bootstrap.cc:492] T ae3be9d91e5d4551bc706b2043a9edf5 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:23.259928 1166 tablet_bootstrap.cc:492] T d703a5a42b2f47d0a7ffe1bb534a1b05 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:23.260210 1166 ts_tablet_manager.cc:1403] T d703a5a42b2f47d0a7ffe1bb534a1b05 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.002s
I20251211 20:52:23.260831 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_09578b67c6ea4eb180bc57baafec6d07"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:23.261487 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_09578b67c6ea4eb180bc57baafec6d07 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:23.263612 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f00f7c1f6a4e44ff89341b4a55cc93fb consensus state has no leader
W20251211 20:52:23.264521 847 leader_election.cc:343] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: ae3be9d91e5d4551bc706b2043a9edf5
W20251211 20:52:23.264700 847 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 80496.6us
W20251211 20:52:23.265055 847 leader_election.cc:343] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:23.265164 847 leader_election.cc:304] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:23.268800 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9af853b8388442319882d871dc136367" } modify_external_catalogs: true
I20251211 20:52:23.268989 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9af853b8388442319882d871dc136367" } modify_external_catalogs: true
I20251211 20:52:23.269665 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_86e66f9a19644d61b002c42103d4fda8" } new_table_name: "default.Table_7c42321f3e494dc8835cc2db68e653e1" modify_external_catalogs: true
I20251211 20:52:23.271736 738 tablet_service.cc:1552] Processing DeleteTablet for tablet 49193f7bd8ac4f20a6ca980a9e1eb830 with delete_type TABLET_DATA_DELETED (replaced by tablet 5ed0a0141051462eb45ca3a66d41d009) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.274015 1166 raft_consensus.cc:359] T d703a5a42b2f47d0a7ffe1bb534a1b05 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.274348 1166 raft_consensus.cc:385] T d703a5a42b2f47d0a7ffe1bb534a1b05 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.274472 1166 raft_consensus.cc:740] T d703a5a42b2f47d0a7ffe1bb534a1b05 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.274766 1166 consensus_queue.cc:260] T d703a5a42b2f47d0a7ffe1bb534a1b05 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.277371 1166 ts_tablet_manager.cc:1434] T d703a5a42b2f47d0a7ffe1bb534a1b05 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.017s user 0.002s sys 0.000s
I20251211 20:52:23.282467 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_eac21915c0ae4694a106ce80dc011bbd" } new_table_name: "default.Table_0d02d938bc90411d93a49209080220f0" modify_external_catalogs: true
I20251211 20:52:23.282650 1166 tablet_bootstrap.cc:492] T 531a74c1fef34b2dba6d4a732e13ce38 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:23.284281 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_afc299e5a13c43878cf6c746698af917" } modify_external_catalogs: true
I20251211 20:52:23.284487 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_afc299e5a13c43878cf6c746698af917" } modify_external_catalogs: true
I20251211 20:52:23.285172 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_95dbc60f2f8f431d96180162033fd31c" } new_table_name: "default.Table_29806e5c967a42e6b62d8268f10aa458" modify_external_catalogs: true
I20251211 20:52:23.285301 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0c955f27bf344b1cbe27484262636529" } modify_external_catalogs: true
I20251211 20:52:23.285457 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0c955f27bf344b1cbe27484262636529" } modify_external_catalogs: true
I20251211 20:52:23.286572 588 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f2db815bce7c4d689a5ce01da255b14f" } new_table_name: "default.Table_88dcd36b4447481487bc918a05511862" modify_external_catalogs: true
I20251211 20:52:23.287616 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed9ee283045a44978422cd080aa2cda5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.288064 881 tablet_service.cc:1505] Processing CreateTablet for tablet 91602f30b4464e35b16fa3dcac445b49 (DEFAULT_TABLE table=default.Table_afc299e5a13c43878cf6c746698af917 [id=6534b4b4514f4763bcff70c04d93ac7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.288547 1182 raft_consensus.cc:493] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:23.288720 1182 raft_consensus.cc:515] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:23.289395 1182 leader_election.cc:290] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
W20251211 20:52:23.290308 711 leader_election.cc:343] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Not found: Tablet not found: 49193f7bd8ac4f20a6ca980a9e1eb830
W20251211 20:52:23.290465 710 leader_election.cc:343] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 49193f7bd8ac4f20a6ca980a9e1eb830
I20251211 20:52:23.290655 710 leader_election.cc:304] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:23.291257 1182 raft_consensus.cc:2749] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:23.292135 1166 tablet_bootstrap.cc:654] T 531a74c1fef34b2dba6d4a732e13ce38 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:23.292470 847 raft_consensus.cc:2686] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251211 20:52:23.298316 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7474a08c51cf4346ae9a518963d56b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.299111 1215 raft_consensus.cc:493] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:23.299307 1215 raft_consensus.cc:515] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.299448 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91602f30b4464e35b16fa3dcac445b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.299991 1215 leader_election.cc:290] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:23.300863 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_daeaf2f3dd24423e9060e4f09a5f4c99" } modify_external_catalogs: true
I20251211 20:52:23.301067 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_daeaf2f3dd24423e9060e4f09a5f4c99" } modify_external_catalogs: true
I20251211 20:52:23.302281 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01ada50100bc48e39d1bc67082da750e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:23.302713 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01ada50100bc48e39d1bc67082da750e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:23.302974 983 leader_election.cc:343] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:23.303380 982 leader_election.cc:343] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:23.303515 982 leader_election.cc:304] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:23.304046 1215 raft_consensus.cc:2749] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:23.306847 1166 tablet_bootstrap.cc:492] T 531a74c1fef34b2dba6d4a732e13ce38 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:23.307371 1166 ts_tablet_manager.cc:1403] T 531a74c1fef34b2dba6d4a732e13ce38 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.000s
I20251211 20:52:23.309177 1166 raft_consensus.cc:359] T 531a74c1fef34b2dba6d4a732e13ce38 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.309599 1166 raft_consensus.cc:385] T 531a74c1fef34b2dba6d4a732e13ce38 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.309868 1166 raft_consensus.cc:740] T 531a74c1fef34b2dba6d4a732e13ce38 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.310349 1166 consensus_queue.cc:260] T 531a74c1fef34b2dba6d4a732e13ce38 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.312523 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd425601d89148eb95525f796a3abf1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.314764 592 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 01ada50100bc48e39d1bc67082da750e (table default.Table_18d19a317a9c4656b4c764df4a64dbea [id=a24d8c58b9d84cdea2b278ae9c6e0ba5]). Expected version 2 got 0
I20251211 20:52:23.315797 1166 ts_tablet_manager.cc:1434] T 531a74c1fef34b2dba6d4a732e13ce38 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.008s user 0.001s sys 0.002s
I20251211 20:52:23.317811 1165 ts_tablet_manager.cc:1950] T cd13d4e7cf2c4abdbedbb5bbac5e05ce P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:23.318336 1168 tablet_bootstrap.cc:654] T ae3be9d91e5d4551bc706b2043a9edf5 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.318485 1021 tablet_service.cc:1505] Processing CreateTablet for tablet f19a83b557e24f929b172d926996ae64 (DEFAULT_TABLE table=default.Table_4a3a3fe6b0294c939ebc90871cbcb4de [id=a06a0b2452ea4100b09772b53320b537]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.318892 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f19a83b557e24f929b172d926996ae64. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.319375 1166 tablet_bootstrap.cc:492] T 35a49c9f560c47049bdcbeddd55ccb28 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:23.319623 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet cd13d4e7cf2c4abdbedbb5bbac5e05ce (table default.Table_7ce0e2ed6ea4463cac10ba89d3718c5c [id=0e7104bb15484108b86dacf1f1f2c359]) successfully deleted
I20251211 20:52:23.319927 1022 tablet_service.cc:1505] Processing CreateTablet for tablet bd8fd6c977204c90b558827fc0b15f43 (DEFAULT_TABLE table=default.Table_eac21915c0ae4694a106ce80dc011bbd [id=e5caa22bce84422290707816a93bd3f5]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:23.320402 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet d703a5a42b2f47d0a7ffe1bb534a1b05 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d703a5a42b2f47d0a7ffe1bb534a1b05 already in progress: creating tablet
I20251211 20:52:23.320546 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd8fd6c977204c90b558827fc0b15f43. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.320822 1165 tablet_replica.cc:333] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:23.321065 1165 raft_consensus.cc:2243] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.321326 1165 raft_consensus.cc:2272] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.322234 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_984c6c545e3a4a2ca496d202198a177b" } modify_external_catalogs: true
I20251211 20:52:23.322463 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_984c6c545e3a4a2ca496d202198a177b" } modify_external_catalogs: true
I20251211 20:52:23.323985 1165 ts_tablet_manager.cc:1916] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.327366 744 tablet_service.cc:1505] Processing CreateTablet for tablet bd8fd6c977204c90b558827fc0b15f43 (DEFAULT_TABLE table=default.Table_eac21915c0ae4694a106ce80dc011bbd [id=e5caa22bce84422290707816a93bd3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.327765 1166 tablet_bootstrap.cc:654] T 35a49c9f560c47049bdcbeddd55ccb28 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.328555 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd8fd6c977204c90b558827fc0b15f43. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:23.331102 581 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet f00f7c1f6a4e44ff89341b4a55cc93fb (table default.Table_b06f9587f78645bb9712c3b080f3c0d4 [id=ea12bdaa894f446da22635f95d97ac2a], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet f00f7c1f6a4e44ff89341b4a55cc93fb consensus state has no leader
I20251211 20:52:23.331554 591 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 01ada50100bc48e39d1bc67082da750e (table default.Table_18d19a317a9c4656b4c764df4a64dbea [id=a24d8c58b9d84cdea2b278ae9c6e0ba5]). Expected version 2 got 0
I20251211 20:52:23.337023 894 tablet_service.cc:1505] Processing CreateTablet for tablet f19a83b557e24f929b172d926996ae64 (DEFAULT_TABLE table=default.Table_4a3a3fe6b0294c939ebc90871cbcb4de [id=a06a0b2452ea4100b09772b53320b537]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.337167 1166 tablet_bootstrap.cc:492] T 35a49c9f560c47049bdcbeddd55ccb28 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:23.337347 1166 ts_tablet_manager.cc:1403] T 35a49c9f560c47049bdcbeddd55ccb28 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.006s
I20251211 20:52:23.337482 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f19a83b557e24f929b172d926996ae64. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.338107 1166 raft_consensus.cc:359] T 35a49c9f560c47049bdcbeddd55ccb28 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.338306 1166 raft_consensus.cc:385] T 35a49c9f560c47049bdcbeddd55ccb28 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.338378 1166 raft_consensus.cc:740] T 35a49c9f560c47049bdcbeddd55ccb28 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.338637 1166 consensus_queue.cc:260] T 35a49c9f560c47049bdcbeddd55ccb28 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.341569 883 tablet_service.cc:1505] Processing CreateTablet for tablet bd8fd6c977204c90b558827fc0b15f43 (DEFAULT_TABLE table=default.Table_eac21915c0ae4694a106ce80dc011bbd [id=e5caa22bce84422290707816a93bd3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.342326 1166 ts_tablet_manager.cc:1434] T 35a49c9f560c47049bdcbeddd55ccb28 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
W20251211 20:52:23.343185 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f92d26bcde3b410aa56bc23a61b1ac35 consensus state has no leader
I20251211 20:52:23.343672 741 tablet_service.cc:1505] Processing CreateTablet for tablet f19a83b557e24f929b172d926996ae64 (DEFAULT_TABLE table=default.Table_4a3a3fe6b0294c939ebc90871cbcb4de [id=a06a0b2452ea4100b09772b53320b537]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.344108 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f19a83b557e24f929b172d926996ae64. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.345289 1166 tablet_bootstrap.cc:492] T bc51bc0821a8484caee39c534eb4e6af P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:23.347231 1166 tablet_bootstrap.cc:654] T bc51bc0821a8484caee39c534eb4e6af P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.349442 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd8fd6c977204c90b558827fc0b15f43. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.355873 1166 tablet_bootstrap.cc:492] T bc51bc0821a8484caee39c534eb4e6af P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:23.355777 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_9c03b1369d12432fb6789a0bac513af0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:23.356237 1166 ts_tablet_manager.cc:1403] T bc51bc0821a8484caee39c534eb4e6af P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.001s
W20251211 20:52:23.356488 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c03b1369d12432fb6789a0bac513af0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:23.357178 1166 raft_consensus.cc:359] T bc51bc0821a8484caee39c534eb4e6af P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.357424 1166 raft_consensus.cc:385] T bc51bc0821a8484caee39c534eb4e6af P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.357553 1166 raft_consensus.cc:740] T bc51bc0821a8484caee39c534eb4e6af P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.357868 1166 consensus_queue.cc:260] T bc51bc0821a8484caee39c534eb4e6af P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.362099 1166 ts_tablet_manager.cc:1434] T bc51bc0821a8484caee39c534eb4e6af P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.006s user 0.000s sys 0.001s
W20251211 20:52:23.363459 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bd8fd6c977204c90b558827fc0b15f43 consensus state has no leader
I20251211 20:52:23.363814 597 master_service.cc:954] ReplaceTablet: received request to replace tablet 8dff13e22899452c8d4f2731b8dac7d5 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:23.365628 1166 tablet_bootstrap.cc:492] T 7f086c7c56fe4abb8df7d3c0978cce6e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:23.366037 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 531a74c1fef34b2dba6d4a732e13ce38 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.366320 1163 tablet_replica.cc:333] T 531a74c1fef34b2dba6d4a732e13ce38 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:23.366421 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet d703a5a42b2f47d0a7ffe1bb534a1b05 with delete_type TABLET_DATA_DELETED (replaced by tablet bd8fd6c977204c90b558827fc0b15f43) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.366521 1163 raft_consensus.cc:2243] T 531a74c1fef34b2dba6d4a732e13ce38 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.366731 1163 raft_consensus.cc:2272] T 531a74c1fef34b2dba6d4a732e13ce38 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.367568 1163 ts_tablet_manager.cc:1916] T 531a74c1fef34b2dba6d4a732e13ce38 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.374146 1163 ts_tablet_manager.cc:1929] T 531a74c1fef34b2dba6d4a732e13ce38 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.374289 1163 log.cc:1199] T 531a74c1fef34b2dba6d4a732e13ce38 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/531a74c1fef34b2dba6d4a732e13ce38
I20251211 20:52:23.374522 1160 tablet_bootstrap.cc:492] T 8e9eded009e44c1a9544d4471c04c5c8 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:23.374615 1163 ts_tablet_manager.cc:1950] T 531a74c1fef34b2dba6d4a732e13ce38 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:23.375689 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 531a74c1fef34b2dba6d4a732e13ce38 (table default.Table_9f357fec3afd4c58b2930fea17141463 [id=4f44988d148e427589d8ee1b342f0590]) successfully deleted
I20251211 20:52:23.376135 1163 tablet_replica.cc:333] T d703a5a42b2f47d0a7ffe1bb534a1b05 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:23.376401 1163 raft_consensus.cc:2243] T d703a5a42b2f47d0a7ffe1bb534a1b05 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.376670 1163 raft_consensus.cc:2272] T d703a5a42b2f47d0a7ffe1bb534a1b05 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.378167 1166 tablet_bootstrap.cc:654] T 7f086c7c56fe4abb8df7d3c0978cce6e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.378832 1163 ts_tablet_manager.cc:1916] T d703a5a42b2f47d0a7ffe1bb534a1b05 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:23.380173 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 27466459d8e6411ab87ac474fb617735 consensus state has no leader
I20251211 20:52:23.382274 1160 ts_tablet_manager.cc:1403] T 8e9eded009e44c1a9544d4471c04c5c8 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.278s user 0.023s sys 0.028s
I20251211 20:52:23.383759 1160 raft_consensus.cc:359] T 8e9eded009e44c1a9544d4471c04c5c8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.385066 1163 ts_tablet_manager.cc:1929] T d703a5a42b2f47d0a7ffe1bb534a1b05 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.385105 1160 raft_consensus.cc:385] T 8e9eded009e44c1a9544d4471c04c5c8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.385212 1160 raft_consensus.cc:740] T 8e9eded009e44c1a9544d4471c04c5c8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.387674 1163 log.cc:1199] T d703a5a42b2f47d0a7ffe1bb534a1b05 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/d703a5a42b2f47d0a7ffe1bb534a1b05
I20251211 20:52:23.388446 1163 ts_tablet_manager.cc:1950] T d703a5a42b2f47d0a7ffe1bb534a1b05 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:23.388978 1160 consensus_queue.cc:260] T 8e9eded009e44c1a9544d4471c04c5c8 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.389496 1160 ts_tablet_manager.cc:1434] T 8e9eded009e44c1a9544d4471c04c5c8 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20251211 20:52:23.390410 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet d703a5a42b2f47d0a7ffe1bb534a1b05 (table default.Table_0d02d938bc90411d93a49209080220f0 [id=e5caa22bce84422290707816a93bd3f5]) successfully deleted
I20251211 20:52:23.393172 1166 tablet_bootstrap.cc:492] T 7f086c7c56fe4abb8df7d3c0978cce6e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:23.393424 1166 ts_tablet_manager.cc:1403] T 7f086c7c56fe4abb8df7d3c0978cce6e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.028s user 0.005s sys 0.002s
I20251211 20:52:23.394268 1166 raft_consensus.cc:359] T 7f086c7c56fe4abb8df7d3c0978cce6e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.394551 1166 raft_consensus.cc:385] T 7f086c7c56fe4abb8df7d3c0978cce6e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.394666 1166 raft_consensus.cc:740] T 7f086c7c56fe4abb8df7d3c0978cce6e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.395080 1160 tablet_bootstrap.cc:492] T d0df24ce65234eeba97f9fdd351b58ad P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:23.395567 596 master_service.cc:954] ReplaceTablet: received request to replace tablet f00f7c1f6a4e44ff89341b4a55cc93fb from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:23.397117 1166 consensus_queue.cc:260] T 7f086c7c56fe4abb8df7d3c0978cce6e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.398681 1164 ts_tablet_manager.cc:1929] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.398808 1164 log.cc:1199] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/ae3be9d91e5d4551bc706b2043a9edf5
I20251211 20:52:23.399152 1164 ts_tablet_manager.cc:1950] T ae3be9d91e5d4551bc706b2043a9edf5 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:23.403811 1166 ts_tablet_manager.cc:1434] T 7f086c7c56fe4abb8df7d3c0978cce6e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
W20251211 20:52:23.406924 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 034f47d8564f4094a9d76aee2fea3119 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 034f47d8564f4094a9d76aee2fea3119 already in progress: creating tablet
I20251211 20:52:23.407480 1166 tablet_bootstrap.cc:492] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:23.407859 1160 tablet_bootstrap.cc:654] T d0df24ce65234eeba97f9fdd351b58ad P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:23.409116 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 8a064789f89c424ca698e52333d73eea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8a064789f89c424ca698e52333d73eea already in progress: creating tablet
I20251211 20:52:23.409435 1166 tablet_bootstrap.cc:654] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:23.412061 588 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5fc8d485cc234fd490c487a8bc881bf4 consensus state has no leader
W20251211 20:52:23.416062 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 9662efc97a53461d888a062463e80ee6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9662efc97a53461d888a062463e80ee6 already in progress: creating tablet
I20251211 20:52:23.418018 588 master_service.cc:954] ReplaceTablet: received request to replace tablet 97a519a8ca7c4f5380a230a70ee84052 from {username='slave'} at 127.0.0.1:44772
W20251211 20:52:23.425796 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet ec7df7b11ce34b8489048e6acdcb4e94 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ec7df7b11ce34b8489048e6acdcb4e94 already in progress: creating tablet
I20251211 20:52:23.428650 1166 tablet_bootstrap.cc:492] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:23.428900 1166 ts_tablet_manager.cc:1403] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.014s
I20251211 20:52:23.429805 1166 raft_consensus.cc:359] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.430084 1166 raft_consensus.cc:385] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.430210 1166 raft_consensus.cc:740] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.430505 1166 consensus_queue.cc:260] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.430925 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 35a49c9f560c47049bdcbeddd55ccb28
I20251211 20:52:23.432842 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 35a49c9f560c47049bdcbeddd55ccb28 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.433429 591 master_service.cc:954] ReplaceTablet: received request to replace tablet f37f598a720d4161b13c03425e020fa5 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:23.434185 1163 tablet_replica.cc:333] T 35a49c9f560c47049bdcbeddd55ccb28 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:23.434367 1163 raft_consensus.cc:2243] T 35a49c9f560c47049bdcbeddd55ccb28 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.434542 1163 raft_consensus.cc:2272] T 35a49c9f560c47049bdcbeddd55ccb28 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.434727 595 master_service.cc:954] ReplaceTablet: received request to replace tablet c5b164dc0ce1461fae98d950f7c7a726 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:23.435616 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 8e9eded009e44c1a9544d4471c04c5c8 with delete_type TABLET_DATA_DELETED (replaced by tablet 97a519a8ca7c4f5380a230a70ee84052) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:23.435830 1163 ts_tablet_manager.cc:1916] T 35a49c9f560c47049bdcbeddd55ccb28 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.435838 878 tablet_service.cc:1552] Processing DeleteTablet for tablet 35a49c9f560c47049bdcbeddd55ccb28 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:23.436196 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 297dc26ade384213b364a876c54bdd14 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 297dc26ade384213b364a876c54bdd14 already in progress: creating tablet
W20251211 20:52:23.436800 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet af3ca9ac988e458cb7068c10c006d587 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet af3ca9ac988e458cb7068c10c006d587 already in progress: creating tablet
W20251211 20:52:23.437319 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 049cdecf12304211adbc1c1008dba989 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 049cdecf12304211adbc1c1008dba989 already in progress: creating tablet
W20251211 20:52:23.437852 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet e69d53ac90a7463aa36ff15dadad7e0b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e69d53ac90a7463aa36ff15dadad7e0b already in progress: creating tablet
W20251211 20:52:23.438349 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 7f086c7c56fe4abb8df7d3c0978cce6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7f086c7c56fe4abb8df7d3c0978cce6e already in progress: creating tablet
W20251211 20:52:23.438990 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 73aa2a3fb2f44bbca870fb7f4cf6087c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 73aa2a3fb2f44bbca870fb7f4cf6087c already in progress: creating tablet
W20251211 20:52:23.439567 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 732c76659c0540bc8b6909b430ffb146 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 732c76659c0540bc8b6909b430ffb146 already in progress: creating tablet
W20251211 20:52:23.440171 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 2433068ec179490dac3e91d046a42b70 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2433068ec179490dac3e91d046a42b70 already in progress: creating tablet
W20251211 20:52:23.440719 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet deacd1f8f3bf4b1f923b61b312492104 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet deacd1f8f3bf4b1f923b61b312492104 already in progress: creating tablet
W20251211 20:52:23.441438 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 4ef506349e5b434db7a78809dc44014e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ef506349e5b434db7a78809dc44014e already in progress: creating tablet
W20251211 20:52:23.441690 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet ef87e052644c41bbbc54b8dad68ce152 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ef87e052644c41bbbc54b8dad68ce152 already in progress: creating tablet
I20251211 20:52:23.441782 1164 tablet_replica.cc:333] T 8e9eded009e44c1a9544d4471c04c5c8 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
W20251211 20:52:23.441939 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet d703a5a42b2f47d0a7ffe1bb534a1b05 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d703a5a42b2f47d0a7ffe1bb534a1b05 already in progress: creating tablet
I20251211 20:52:23.442108 1164 raft_consensus.cc:2243] T 8e9eded009e44c1a9544d4471c04c5c8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.442327 1164 raft_consensus.cc:2272] T 8e9eded009e44c1a9544d4471c04c5c8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.443154 1164 ts_tablet_manager.cc:1916] T 8e9eded009e44c1a9544d4471c04c5c8 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.443642 1163 ts_tablet_manager.cc:1929] T 35a49c9f560c47049bdcbeddd55ccb28 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.443856 1163 log.cc:1199] T 35a49c9f560c47049bdcbeddd55ccb28 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/35a49c9f560c47049bdcbeddd55ccb28
I20251211 20:52:23.444268 1163 ts_tablet_manager.cc:1950] T 35a49c9f560c47049bdcbeddd55ccb28 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:23.445396 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 35a49c9f560c47049bdcbeddd55ccb28 (table default.Table_0c955f27bf344b1cbe27484262636529 [id=16a2a090abb14c1a996e695ea2798579]) successfully deleted
I20251211 20:52:23.445806 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet f37f598a720d4161b13c03425e020fa5
I20251211 20:52:23.446223 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 35a49c9f560c47049bdcbeddd55ccb28 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.446543 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet f37f598a720d4161b13c03425e020fa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.447527 1163 tablet_replica.cc:333] T f37f598a720d4161b13c03425e020fa5 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:23.447782 1163 raft_consensus.cc:2243] T f37f598a720d4161b13c03425e020fa5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.447997 1163 raft_consensus.cc:2272] T f37f598a720d4161b13c03425e020fa5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.447086 754 tablet_service.cc:1552] Processing DeleteTablet for tablet f37f598a720d4161b13c03425e020fa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.446028 1166 ts_tablet_manager.cc:1434] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.017s user 0.001s sys 0.001s
I20251211 20:52:23.451795 593 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet acdb7dddf3e74ce1bc92f232b6180cca (table default.Table_28b427311c9b4699a47c38ebb4d93e01 [id=b68aac5147654707b677d020a2cc37cd]). Expected version 1 got 0
I20251211 20:52:23.452958 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 7f086c7c56fe4abb8df7d3c0978cce6e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.453240 1166 tablet_bootstrap.cc:492] T af3ca9ac988e458cb7068c10c006d587 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:23.454722 593 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet acdb7dddf3e74ce1bc92f232b6180cca (table default.Table_28b427311c9b4699a47c38ebb4d93e01 [id=b68aac5147654707b677d020a2cc37cd]). Expected version 1 got 0
I20251211 20:52:23.455205 1163 ts_tablet_manager.cc:1916] T f37f598a720d4161b13c03425e020fa5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.445996 1160 tablet_bootstrap.cc:492] T d0df24ce65234eeba97f9fdd351b58ad P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:23.455796 1160 ts_tablet_manager.cc:1403] T d0df24ce65234eeba97f9fdd351b58ad P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.061s user 0.007s sys 0.000s
I20251211 20:52:23.455981 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_86e5b1e5b4214b48a21f398642d97640"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:23.456713 1166 tablet_bootstrap.cc:654] T af3ca9ac988e458cb7068c10c006d587 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:23.456763 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_86e5b1e5b4214b48a21f398642d97640 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:23.456816 1160 raft_consensus.cc:359] T d0df24ce65234eeba97f9fdd351b58ad P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.457069 1160 raft_consensus.cc:385] T d0df24ce65234eeba97f9fdd351b58ad P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.457213 1160 raft_consensus.cc:740] T d0df24ce65234eeba97f9fdd351b58ad P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.446909 884 tablet_service.cc:1552] Processing DeleteTablet for tablet f37f598a720d4161b13c03425e020fa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:23.457664 1160 consensus_queue.cc:260] T d0df24ce65234eeba97f9fdd351b58ad P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.458376 1160 ts_tablet_manager.cc:1434] T d0df24ce65234eeba97f9fdd351b58ad P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:23.459095 1160 tablet_bootstrap.cc:492] T deacd1f8f3bf4b1f923b61b312492104 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:23.461015 884 tablet_service.cc:1552] Processing DeleteTablet for tablet d0df24ce65234eeba97f9fdd351b58ad with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:23.463644 1168 tablet_bootstrap.cc:492] T ae3be9d91e5d4551bc706b2043a9edf5 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:23.463892 1168 ts_tablet_manager.cc:1403] T ae3be9d91e5d4551bc706b2043a9edf5 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.207s user 0.008s sys 0.000s
I20251211 20:52:23.464962 1168 raft_consensus.cc:359] T ae3be9d91e5d4551bc706b2043a9edf5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:23.465243 1168 raft_consensus.cc:385] T ae3be9d91e5d4551bc706b2043a9edf5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.465368 1168 raft_consensus.cc:740] T ae3be9d91e5d4551bc706b2043a9edf5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.465797 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 0129162969d74f5da3c08a1e6a27ab22
I20251211 20:52:23.465735 1168 consensus_queue.cc:260] T ae3be9d91e5d4551bc706b2043a9edf5 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:23.468513 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 0129162969d74f5da3c08a1e6a27ab22 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.469461 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 0129162969d74f5da3c08a1e6a27ab22 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:23.469966 1168 ts_tablet_manager.cc:1434] T ae3be9d91e5d4551bc706b2043a9edf5 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20251211 20:52:23.471140 1168 tablet_bootstrap.cc:492] T 274c10c2aa46479dab5edbb483e5b883 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:23.471537 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9bec3557f8134357af09573f28abb7ad" } new_table_name: "default.Table_f9014306b36e47b98c8e4c6070ea4a37" modify_external_catalogs: true
I20251211 20:52:23.472785 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 0129162969d74f5da3c08a1e6a27ab22 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.473373 1168 tablet_bootstrap.cc:654] T 274c10c2aa46479dab5edbb483e5b883 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.474181 1163 ts_tablet_manager.cc:1929] T f37f598a720d4161b13c03425e020fa5 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.474722 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_40d581fe03ef48d1b5639a95ffc00b7b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:23.475129 1163 log.cc:1199] T f37f598a720d4161b13c03425e020fa5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/f37f598a720d4161b13c03425e020fa5
W20251211 20:52:23.475297 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_40d581fe03ef48d1b5639a95ffc00b7b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:23.475731 754 tablet_service.cc:1552] Processing DeleteTablet for tablet ae3be9d91e5d4551bc706b2043a9edf5 with delete_type TABLET_DATA_DELETED (replaced by tablet e2e0a1e32bba40e294bf0a2fe10a98cd) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.476256 1163 ts_tablet_manager.cc:1950] T f37f598a720d4161b13c03425e020fa5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:23.476333 1165 ts_tablet_manager.cc:1929] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.474340 1160 tablet_bootstrap.cc:654] T deacd1f8f3bf4b1f923b61b312492104 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.477502 1165 log.cc:1199] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/49193f7bd8ac4f20a6ca980a9e1eb830
I20251211 20:52:23.477810 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet f37f598a720d4161b13c03425e020fa5 (table default.Table_9af853b8388442319882d871dc136367 [id=0563c542d899496e8586946ff3a66d96]) successfully deleted
I20251211 20:52:23.477933 1165 ts_tablet_manager.cc:1950] T 49193f7bd8ac4f20a6ca980a9e1eb830 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:23.479024 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 49193f7bd8ac4f20a6ca980a9e1eb830 (table default.Table_40256d3973c74566a09b1d235062d284 [id=1f4650c549d94e2c9ef753ff96670aa6]) successfully deleted
I20251211 20:52:23.479132 1163 tablet_replica.cc:333] T 7f086c7c56fe4abb8df7d3c0978cce6e P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:23.479343 1163 raft_consensus.cc:2243] T 7f086c7c56fe4abb8df7d3c0978cce6e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.479550 1163 raft_consensus.cc:2272] T 7f086c7c56fe4abb8df7d3c0978cce6e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.481244 1166 tablet_bootstrap.cc:492] T af3ca9ac988e458cb7068c10c006d587 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:23.481490 1166 ts_tablet_manager.cc:1403] T af3ca9ac988e458cb7068c10c006d587 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.030s user 0.005s sys 0.001s
I20251211 20:52:23.482402 1166 raft_consensus.cc:359] T af3ca9ac988e458cb7068c10c006d587 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.482694 1166 raft_consensus.cc:385] T af3ca9ac988e458cb7068c10c006d587 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.482816 1166 raft_consensus.cc:740] T af3ca9ac988e458cb7068c10c006d587 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.483145 1166 consensus_queue.cc:260] T af3ca9ac988e458cb7068c10c006d587 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
W20251211 20:52:23.485023 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 35a49c9f560c47049bdcbeddd55ccb28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 35a49c9f560c47049bdcbeddd55ccb28 already in progress: creating tablet
I20251211 20:52:23.485319 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 91602f30b4464e35b16fa3dcac445b49
W20251211 20:52:23.485745 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet f37f598a720d4161b13c03425e020fa5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f37f598a720d4161b13c03425e020fa5 already in progress: creating tablet
I20251211 20:52:23.486620 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 91602f30b4464e35b16fa3dcac445b49 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:23.486666 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 0129162969d74f5da3c08a1e6a27ab22 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0129162969d74f5da3c08a1e6a27ab22 already in progress: creating tablet
I20251211 20:52:23.486939 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 91602f30b4464e35b16fa3dcac445b49 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:23.487264 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 91602f30b4464e35b16fa3dcac445b49 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.489140 1166 ts_tablet_manager.cc:1434] T af3ca9ac988e458cb7068c10c006d587 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20251211 20:52:23.489182 1165 tablet_replica.cc:333] T ae3be9d91e5d4551bc706b2043a9edf5 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:23.489359 1165 raft_consensus.cc:2243] T ae3be9d91e5d4551bc706b2043a9edf5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.491501 1165 raft_consensus.cc:2272] T ae3be9d91e5d4551bc706b2043a9edf5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.492293 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_242ad5ed8a5a48cf9d57630363e9b3c5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:23.492431 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet af3ca9ac988e458cb7068c10c006d587 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.492601 1166 tablet_bootstrap.cc:492] T 049cdecf12304211adbc1c1008dba989 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:23.493291 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_242ad5ed8a5a48cf9d57630363e9b3c5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:23.494124 1165 ts_tablet_manager.cc:1916] T ae3be9d91e5d4551bc706b2043a9edf5 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.494403 1166 tablet_bootstrap.cc:654] T 049cdecf12304211adbc1c1008dba989 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.495919 1168 tablet_bootstrap.cc:492] T 274c10c2aa46479dab5edbb483e5b883 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:23.496156 1168 ts_tablet_manager.cc:1403] T 274c10c2aa46479dab5edbb483e5b883 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.000s
I20251211 20:52:23.495119 1163 ts_tablet_manager.cc:1916] T 7f086c7c56fe4abb8df7d3c0978cce6e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.497162 1168 raft_consensus.cc:359] T 274c10c2aa46479dab5edbb483e5b883 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:23.497439 1168 raft_consensus.cc:385] T 274c10c2aa46479dab5edbb483e5b883 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.497650 1168 raft_consensus.cc:740] T 274c10c2aa46479dab5edbb483e5b883 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.497963 1168 consensus_queue.cc:260] T 274c10c2aa46479dab5edbb483e5b883 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
W20251211 20:52:23.500463 1204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:52:23.500861 1204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:52:23.500963 1204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20251211 20:52:23.502441 1168 ts_tablet_manager.cc:1434] T 274c10c2aa46479dab5edbb483e5b883 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20251211 20:52:23.503520 1163 ts_tablet_manager.cc:1929] T 7f086c7c56fe4abb8df7d3c0978cce6e P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.503716 1163 log.cc:1199] T 7f086c7c56fe4abb8df7d3c0978cce6e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/7f086c7c56fe4abb8df7d3c0978cce6e
I20251211 20:52:23.504050 1163 ts_tablet_manager.cc:1950] T 7f086c7c56fe4abb8df7d3c0978cce6e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:23.504990 1168 tablet_bootstrap.cc:492] T 8e9eded009e44c1a9544d4471c04c5c8 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:23.505045 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 7f086c7c56fe4abb8df7d3c0978cce6e (table default.Table_e5a162d6463e4ff5a4edaaae5a4d03cb [id=872ff521abd740ee924f22e9c0ff8585]) successfully deleted
W20251211 20:52:23.505934 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 0129162969d74f5da3c08a1e6a27ab22 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0129162969d74f5da3c08a1e6a27ab22 already in progress: creating tablet
W20251211 20:52:23.506498 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 91602f30b4464e35b16fa3dcac445b49 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 91602f30b4464e35b16fa3dcac445b49 already in progress: creating tablet
I20251211 20:52:23.506803 1163 tablet_replica.cc:333] T af3ca9ac988e458cb7068c10c006d587 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:23.507005 1163 raft_consensus.cc:2243] T af3ca9ac988e458cb7068c10c006d587 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.507184 1163 raft_consensus.cc:2272] T af3ca9ac988e458cb7068c10c006d587 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.507748 1160 tablet_bootstrap.cc:492] T deacd1f8f3bf4b1f923b61b312492104 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:23.507942 1163 ts_tablet_manager.cc:1916] T af3ca9ac988e458cb7068c10c006d587 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.507969 1160 ts_tablet_manager.cc:1403] T deacd1f8f3bf4b1f923b61b312492104 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.049s user 0.001s sys 0.003s
I20251211 20:52:23.508832 1160 raft_consensus.cc:359] T deacd1f8f3bf4b1f923b61b312492104 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.509094 1160 raft_consensus.cc:385] T deacd1f8f3bf4b1f923b61b312492104 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.509215 1160 raft_consensus.cc:740] T deacd1f8f3bf4b1f923b61b312492104 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.509544 1160 consensus_queue.cc:260] T deacd1f8f3bf4b1f923b61b312492104 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.510280 1160 ts_tablet_manager.cc:1434] T deacd1f8f3bf4b1f923b61b312492104 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:23.511210 1160 tablet_bootstrap.cc:492] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:23.513294 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet d824bc02b25842d18f28f711eef6669f
I20251211 20:52:23.513818 1166 tablet_bootstrap.cc:492] T 049cdecf12304211adbc1c1008dba989 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:23.513957 1166 ts_tablet_manager.cc:1403] T 049cdecf12304211adbc1c1008dba989 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.002s
I20251211 20:52:23.514678 1166 raft_consensus.cc:359] T 049cdecf12304211adbc1c1008dba989 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.514854 1166 raft_consensus.cc:385] T 049cdecf12304211adbc1c1008dba989 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.514925 1166 raft_consensus.cc:740] T 049cdecf12304211adbc1c1008dba989 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
W20251211 20:52:23.515368 1204 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251211 20:52:23.515470 1204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:52:23.515550 1204 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251211 20:52:23.515628 1204 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251211 20:52:23.517793 1160 tablet_bootstrap.cc:654] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.511752 1164 ts_tablet_manager.cc:1929] T 8e9eded009e44c1a9544d4471c04c5c8 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.518504 1166 consensus_queue.cc:260] T 049cdecf12304211adbc1c1008dba989 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.518639 1164 log.cc:1199] T 8e9eded009e44c1a9544d4471c04c5c8 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/8e9eded009e44c1a9544d4471c04c5c8
I20251211 20:52:23.519057 1166 ts_tablet_manager.cc:1434] T 049cdecf12304211adbc1c1008dba989 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251211 20:52:23.519758 1166 tablet_bootstrap.cc:492] T 8a064789f89c424ca698e52333d73eea P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:23.521378 1166 tablet_bootstrap.cc:654] T 8a064789f89c424ca698e52333d73eea P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.523805 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet d824bc02b25842d18f28f711eef6669f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.523923 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 274c10c2aa46479dab5edbb483e5b883 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.524616 1168 tablet_bootstrap.cc:654] T 8e9eded009e44c1a9544d4471c04c5c8 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.530416 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_ef1c4ca915474d569c91296213346105"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:23.530987 1163 ts_tablet_manager.cc:1929] T af3ca9ac988e458cb7068c10c006d587 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.531142 1163 log.cc:1199] T af3ca9ac988e458cb7068c10c006d587 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/af3ca9ac988e458cb7068c10c006d587
I20251211 20:52:23.531481 1163 ts_tablet_manager.cc:1950] T af3ca9ac988e458cb7068c10c006d587 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:23.531610 1166 tablet_bootstrap.cc:492] T 8a064789f89c424ca698e52333d73eea P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:23.531726 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 049cdecf12304211adbc1c1008dba989 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.531778 1166 ts_tablet_manager.cc:1403] T 8a064789f89c424ca698e52333d73eea P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.004s
I20251211 20:52:23.531770 884 tablet_service.cc:1552] Processing DeleteTablet for tablet d824bc02b25842d18f28f711eef6669f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:23.532006 584 consensus_peers.cc:597] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 -> Peer 1830b0c58cfe495989f3d911dda0409a (127.0.104.62:41123): Couldn't send request to peer 1830b0c58cfe495989f3d911dda0409a. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.62:41123: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20251211 20:52:23.532286 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef1c4ca915474d569c91296213346105 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:23.532981 1163 tablet_replica.cc:333] T 049cdecf12304211adbc1c1008dba989 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:23.533174 1163 raft_consensus.cc:2243] T 049cdecf12304211adbc1c1008dba989 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.533123 1166 raft_consensus.cc:359] T 8a064789f89c424ca698e52333d73eea P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:23.533308 1166 raft_consensus.cc:385] T 8a064789f89c424ca698e52333d73eea P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.533360 1163 raft_consensus.cc:2272] T 049cdecf12304211adbc1c1008dba989 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.533385 1166 raft_consensus.cc:740] T 8a064789f89c424ca698e52333d73eea P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.533357 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet af3ca9ac988e458cb7068c10c006d587 (table default.Table_a11e64052cf34ba6a6a31f4edb417c19 [id=0afc1e783888443eb3f3e7848a04c207]) successfully deleted
I20251211 20:52:23.533620 1166 consensus_queue.cc:260] T 8a064789f89c424ca698e52333d73eea P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
W20251211 20:52:23.534080 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet d824bc02b25842d18f28f711eef6669f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d824bc02b25842d18f28f711eef6669f already in progress: creating tablet
I20251211 20:52:23.534130 1166 ts_tablet_manager.cc:1434] T 8a064789f89c424ca698e52333d73eea P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:23.534418 1166 tablet_bootstrap.cc:492] T f4db14d082d34aef857e106992bb8950 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:23.534394 744 tablet_service.cc:1552] Processing DeleteTablet for tablet d824bc02b25842d18f28f711eef6669f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.536073 1166 tablet_bootstrap.cc:654] T f4db14d082d34aef857e106992bb8950 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.537212 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_897d3e48834a42f59e057317699b566d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:23.537750 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_897d3e48834a42f59e057317699b566d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:23.539158 884 tablet_service.cc:1552] Processing DeleteTablet for tablet deacd1f8f3bf4b1f923b61b312492104 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:23.539268 1163 ts_tablet_manager.cc:1916] T 049cdecf12304211adbc1c1008dba989 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.539891 1204 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.20:41333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.62:41123,127.0.104.61:40749,127.0.104.60:42087
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.62:41123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.104.62
--webserver_port=37413
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:52:23.542104 1204 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:52:23.542913 1164 ts_tablet_manager.cc:1950] T 8e9eded009e44c1a9544d4471c04c5c8 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:23.544757 1204 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251211 20:52:23.544927 1164 tablet_replica.cc:333] T d0df24ce65234eeba97f9fdd351b58ad P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:23.545143 1164 raft_consensus.cc:2243] T d0df24ce65234eeba97f9fdd351b58ad P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.545405 1164 raft_consensus.cc:2272] T d0df24ce65234eeba97f9fdd351b58ad P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.546531 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 8e9eded009e44c1a9544d4471c04c5c8 (table default.Table_19db6b94abda4124a0e45f11b8ca0990 [id=383392d835da4c50848068d83261cecc]) successfully deleted
I20251211 20:52:23.546851 1164 ts_tablet_manager.cc:1916] T d0df24ce65234eeba97f9fdd351b58ad P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:23.546834 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 35a49c9f560c47049bdcbeddd55ccb28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 35a49c9f560c47049bdcbeddd55ccb28 already in progress: creating tablet
W20251211 20:52:23.547061 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet f37f598a720d4161b13c03425e020fa5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f37f598a720d4161b13c03425e020fa5 already in progress: creating tablet
I20251211 20:52:23.549556 1163 ts_tablet_manager.cc:1929] T 049cdecf12304211adbc1c1008dba989 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.549690 1163 log.cc:1199] T 049cdecf12304211adbc1c1008dba989 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/049cdecf12304211adbc1c1008dba989
I20251211 20:52:23.550028 1163 ts_tablet_manager.cc:1950] T 049cdecf12304211adbc1c1008dba989 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:23.551028 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 049cdecf12304211adbc1c1008dba989 (table default.Table_e6a96166bf33442989e15129d4d44631 [id=34edf86667d448c0b87e55627c67a704]) successfully deleted
I20251211 20:52:23.553977 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 8dff13e22899452c8d4f2731b8dac7d5
I20251211 20:52:23.554109 1165 ts_tablet_manager.cc:1929] T ae3be9d91e5d4551bc706b2043a9edf5 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.554224 1165 log.cc:1199] T ae3be9d91e5d4551bc706b2043a9edf5 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/ae3be9d91e5d4551bc706b2043a9edf5
I20251211 20:52:23.554540 597 catalog_manager.cc:6724] ReplaceTablet: tablet 8dff13e22899452c8d4f2731b8dac7d5 deleted and replaced by tablet dc5de12701714fcd81b9dba6973a28b5
I20251211 20:52:23.554586 1165 ts_tablet_manager.cc:1950] T ae3be9d91e5d4551bc706b2043a9edf5 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:23.554976 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 8dff13e22899452c8d4f2731b8dac7d5 with delete_type TABLET_DATA_DELETED (replaced by tablet dc5de12701714fcd81b9dba6973a28b5) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:23.555632 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 8dff13e22899452c8d4f2731b8dac7d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8dff13e22899452c8d4f2731b8dac7d5 already in progress: creating tablet
I20251211 20:52:23.556306 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 8a064789f89c424ca698e52333d73eea with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.556540 1163 tablet_replica.cc:333] T 8a064789f89c424ca698e52333d73eea P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:23.556768 1163 raft_consensus.cc:2243] T 8a064789f89c424ca698e52333d73eea P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.557111 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_c2300bb8bfe44c689969a2b233d69353"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:23.557308 1163 raft_consensus.cc:2272] T 8a064789f89c424ca698e52333d73eea P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:23.557682 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c2300bb8bfe44c689969a2b233d69353 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:23.557996 1163 ts_tablet_manager.cc:1916] T 8a064789f89c424ca698e52333d73eea P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.558851 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet f00f7c1f6a4e44ff89341b4a55cc93fb
I20251211 20:52:23.559253 596 catalog_manager.cc:6724] ReplaceTablet: tablet f00f7c1f6a4e44ff89341b4a55cc93fb deleted and replaced by tablet 88066d94d5d34e0fab20da9fb6ef178e
I20251211 20:52:23.559547 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_6bf819d1a1794e62b4cf65ffe5d2c2b5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:23.559763 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet f00f7c1f6a4e44ff89341b4a55cc93fb with delete_type TABLET_DATA_DELETED (replaced by tablet 88066d94d5d34e0fab20da9fb6ef178e) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:23.560077 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6bf819d1a1794e62b4cf65ffe5d2c2b5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:23.562434 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 8dff13e22899452c8d4f2731b8dac7d5 with delete_type TABLET_DATA_DELETED (replaced by tablet dc5de12701714fcd81b9dba6973a28b5) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.563072 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 8dff13e22899452c8d4f2731b8dac7d5 with delete_type TABLET_DATA_DELETED (replaced by tablet dc5de12701714fcd81b9dba6973a28b5) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:23.563095 744 tablet_service.cc:1552] Processing DeleteTablet for tablet f00f7c1f6a4e44ff89341b4a55cc93fb with delete_type TABLET_DATA_DELETED (replaced by tablet 88066d94d5d34e0fab20da9fb6ef178e) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.564167 1168 tablet_bootstrap.cc:492] T 8e9eded009e44c1a9544d4471c04c5c8 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:23.564137 884 tablet_service.cc:1552] Processing DeleteTablet for tablet f00f7c1f6a4e44ff89341b4a55cc93fb with delete_type TABLET_DATA_DELETED (replaced by tablet 88066d94d5d34e0fab20da9fb6ef178e) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:23.564394 1168 ts_tablet_manager.cc:1403] T 8e9eded009e44c1a9544d4471c04c5c8 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.059s user 0.003s sys 0.002s
I20251211 20:52:23.565210 1168 raft_consensus.cc:359] T 8e9eded009e44c1a9544d4471c04c5c8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.565478 1168 raft_consensus.cc:385] T 8e9eded009e44c1a9544d4471c04c5c8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.565598 1168 raft_consensus.cc:740] T 8e9eded009e44c1a9544d4471c04c5c8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.565923 1168 consensus_queue.cc:260] T 8e9eded009e44c1a9544d4471c04c5c8 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.568584 1168 ts_tablet_manager.cc:1434] T 8e9eded009e44c1a9544d4471c04c5c8 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251211 20:52:23.569128 1168 tablet_bootstrap.cc:492] T d0df24ce65234eeba97f9fdd351b58ad P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:23.571614 1163 ts_tablet_manager.cc:1929] T 8a064789f89c424ca698e52333d73eea P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.571796 1163 log.cc:1199] T 8a064789f89c424ca698e52333d73eea P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/8a064789f89c424ca698e52333d73eea
I20251211 20:52:23.572093 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 97a519a8ca7c4f5380a230a70ee84052
I20251211 20:52:23.572147 1163 ts_tablet_manager.cc:1950] T 8a064789f89c424ca698e52333d73eea P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:23.572585 588 catalog_manager.cc:6724] ReplaceTablet: tablet 97a519a8ca7c4f5380a230a70ee84052 deleted and replaced by tablet 27a08a64d92342d08ab4406e8006a97b
I20251211 20:52:23.573626 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 97a519a8ca7c4f5380a230a70ee84052 with delete_type TABLET_DATA_DELETED (replaced by tablet 27a08a64d92342d08ab4406e8006a97b) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.573861 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 97a519a8ca7c4f5380a230a70ee84052 with delete_type TABLET_DATA_DELETED (replaced by tablet 27a08a64d92342d08ab4406e8006a97b) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:23.573978 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 97a519a8ca7c4f5380a230a70ee84052 with delete_type TABLET_DATA_DELETED (replaced by tablet 27a08a64d92342d08ab4406e8006a97b) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.576342 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 8a064789f89c424ca698e52333d73eea (table default.Table_f439572c5b1d4997af54e0ab3c036023 [id=cda4c0ad8a0347be88a0518491e6af49]) successfully deleted
I20251211 20:52:23.577250 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet ae3be9d91e5d4551bc706b2043a9edf5 (table default.Table_673da12cd31d4ba789de37d39fd60988 [id=b9fb32193afe4e5faa21db3c195030ab]) successfully deleted
W20251211 20:52:23.577518 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet f00f7c1f6a4e44ff89341b4a55cc93fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f00f7c1f6a4e44ff89341b4a55cc93fb already in progress: creating tablet
W20251211 20:52:23.577797 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 91602f30b4464e35b16fa3dcac445b49 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 91602f30b4464e35b16fa3dcac445b49 already in progress: creating tablet
I20251211 20:52:23.577984 1165 tablet_replica.cc:333] T 274c10c2aa46479dab5edbb483e5b883 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:23.578069 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_4c974eff9ab442d499f0405abc8a4acd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:23.578329 1165 raft_consensus.cc:2243] T 274c10c2aa46479dab5edbb483e5b883 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.578500 1165 raft_consensus.cc:2272] T 274c10c2aa46479dab5edbb483e5b883 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:23.578684 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4c974eff9ab442d499f0405abc8a4acd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:23.579416 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 8e9eded009e44c1a9544d4471c04c5c8 with delete_type TABLET_DATA_DELETED (replaced by tablet 97a519a8ca7c4f5380a230a70ee84052) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:23.580336 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 97a519a8ca7c4f5380a230a70ee84052 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 97a519a8ca7c4f5380a230a70ee84052 already in progress: creating tablet
I20251211 20:52:23.580469 1168 tablet_bootstrap.cc:654] T d0df24ce65234eeba97f9fdd351b58ad P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.582041 1165 ts_tablet_manager.cc:1916] T 274c10c2aa46479dab5edbb483e5b883 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.587963 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet c5b164dc0ce1461fae98d950f7c7a726
I20251211 20:52:23.588411 595 catalog_manager.cc:6724] ReplaceTablet: tablet c5b164dc0ce1461fae98d950f7c7a726 deleted and replaced by tablet 49a33084c0fc460eb9592224b6dfc279
I20251211 20:52:23.589181 744 tablet_service.cc:1552] Processing DeleteTablet for tablet c5b164dc0ce1461fae98d950f7c7a726 with delete_type TABLET_DATA_DELETED (replaced by tablet 49a33084c0fc460eb9592224b6dfc279) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.589973 1160 tablet_bootstrap.cc:492] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:23.590129 1160 ts_tablet_manager.cc:1403] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.080s user 0.003s sys 0.003s
I20251211 20:52:23.590102 884 tablet_service.cc:1552] Processing DeleteTablet for tablet c5b164dc0ce1461fae98d950f7c7a726 with delete_type TABLET_DATA_DELETED (replaced by tablet 49a33084c0fc460eb9592224b6dfc279) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:23.590987 1160 raft_consensus.cc:359] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.591159 1160 raft_consensus.cc:385] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.591243 1160 raft_consensus.cc:740] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.591516 1160 consensus_queue.cc:260] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.592952 1160 ts_tablet_manager.cc:1434] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:23.593892 1160 tablet_bootstrap.cc:492] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:23.595985 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_6c528dbf0fb149b3894f6afb543ae25b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:23.596176 1160 tablet_bootstrap.cc:654] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:23.596521 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6c528dbf0fb149b3894f6afb543ae25b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:23.598310 1168 tablet_bootstrap.cc:492] T d0df24ce65234eeba97f9fdd351b58ad P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:23.598479 1168 ts_tablet_manager.cc:1403] T d0df24ce65234eeba97f9fdd351b58ad P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.002s
I20251211 20:52:23.598737 884 tablet_service.cc:1552] Processing DeleteTablet for tablet dc7e8db4b51c415e86d8c24a76b8cb1e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:23.597489 1164 ts_tablet_manager.cc:1929] T d0df24ce65234eeba97f9fdd351b58ad P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.599062 1164 log.cc:1199] T d0df24ce65234eeba97f9fdd351b58ad P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/d0df24ce65234eeba97f9fdd351b58ad
I20251211 20:52:23.599486 1164 ts_tablet_manager.cc:1950] T d0df24ce65234eeba97f9fdd351b58ad P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:23.603080 1168 raft_consensus.cc:359] T d0df24ce65234eeba97f9fdd351b58ad P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.604688 1168 raft_consensus.cc:385] T d0df24ce65234eeba97f9fdd351b58ad P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.605134 1168 raft_consensus.cc:740] T d0df24ce65234eeba97f9fdd351b58ad P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.605448 1168 consensus_queue.cc:260] T d0df24ce65234eeba97f9fdd351b58ad P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.606046 1164 tablet_replica.cc:333] T deacd1f8f3bf4b1f923b61b312492104 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:23.606348 1164 raft_consensus.cc:2243] T deacd1f8f3bf4b1f923b61b312492104 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.607990 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet d0df24ce65234eeba97f9fdd351b58ad (table default.Table_1b829e0ef7ef4b17be671db434f3858c [id=b8c401dfdb5f4f7584dca0989287ce31]) successfully deleted
W20251211 20:52:23.608245 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 0129162969d74f5da3c08a1e6a27ab22 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0129162969d74f5da3c08a1e6a27ab22 already in progress: creating tablet
W20251211 20:52:23.608460 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 91602f30b4464e35b16fa3dcac445b49 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 91602f30b4464e35b16fa3dcac445b49 already in progress: creating tablet
W20251211 20:52:23.608654 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet d824bc02b25842d18f28f711eef6669f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d824bc02b25842d18f28f711eef6669f already in progress: creating tablet
I20251211 20:52:23.609608 1168 ts_tablet_manager.cc:1434] T d0df24ce65234eeba97f9fdd351b58ad P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20251211 20:52:23.609844 1164 raft_consensus.cc:2272] T deacd1f8f3bf4b1f923b61b312492104 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.610741 1164 ts_tablet_manager.cc:1916] T deacd1f8f3bf4b1f923b61b312492104 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.611496 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_61835079722245debfab2fb8362f368b" } new_table_name: "default.Table_86450313ee0040bb87db3b340566266b" modify_external_catalogs: true
I20251211 20:52:23.612934 744 tablet_service.cc:1552] Processing DeleteTablet for tablet d0df24ce65234eeba97f9fdd351b58ad with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.614521 1168 tablet_bootstrap.cc:492] T deacd1f8f3bf4b1f923b61b312492104 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:23.616385 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet c5b164dc0ce1461fae98d950f7c7a726 with delete_type TABLET_DATA_DELETED (replaced by tablet 49a33084c0fc460eb9592224b6dfc279) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:23.617234 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet c5b164dc0ce1461fae98d950f7c7a726 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c5b164dc0ce1461fae98d950f7c7a726 already in progress: creating tablet
I20251211 20:52:23.618752 1165 ts_tablet_manager.cc:1929] T 274c10c2aa46479dab5edbb483e5b883 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.618824 1166 tablet_bootstrap.cc:492] T f4db14d082d34aef857e106992bb8950 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:23.618891 1165 log.cc:1199] T 274c10c2aa46479dab5edbb483e5b883 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/274c10c2aa46479dab5edbb483e5b883
I20251211 20:52:23.618984 1166 ts_tablet_manager.cc:1403] T f4db14d082d34aef857e106992bb8950 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.085s user 0.010s sys 0.038s
I20251211 20:52:23.619899 1166 raft_consensus.cc:359] T f4db14d082d34aef857e106992bb8950 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.620103 1166 raft_consensus.cc:385] T f4db14d082d34aef857e106992bb8950 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.620196 1166 raft_consensus.cc:740] T f4db14d082d34aef857e106992bb8950 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.620452 1166 consensus_queue.cc:260] T f4db14d082d34aef857e106992bb8950 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.621413 1168 tablet_bootstrap.cc:654] T deacd1f8f3bf4b1f923b61b312492104 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.622502 1160 tablet_bootstrap.cc:492] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:23.622705 1160 ts_tablet_manager.cc:1403] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.000s
W20251211 20:52:23.622892 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4b736a1b347d44b398980654caa21332 has no consensus state
I20251211 20:52:23.623540 1160 raft_consensus.cc:359] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.623795 1160 raft_consensus.cc:385] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.623873 1160 raft_consensus.cc:740] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.624150 1160 consensus_queue.cc:260] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.625386 1166 ts_tablet_manager.cc:1434] T f4db14d082d34aef857e106992bb8950 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20251211 20:52:23.626658 1160 ts_tablet_manager.cc:1434] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:23.627652 1160 tablet_bootstrap.cc:492] T 531a74c1fef34b2dba6d4a732e13ce38 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:23.629307 1166 tablet_bootstrap.cc:492] T 034f47d8564f4094a9d76aee2fea3119 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:23.629372 1165 ts_tablet_manager.cc:1950] T 274c10c2aa46479dab5edbb483e5b883 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:23.630367 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 274c10c2aa46479dab5edbb483e5b883 (table default.Table_7a3a07a264fc4bed96455f83ae51fb05 [id=5a91ece54cfa446880f72a42678f4f26]) successfully deleted
I20251211 20:52:23.631125 1166 tablet_bootstrap.cc:654] T 034f47d8564f4094a9d76aee2fea3119 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.635497 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4c2b2d7d11a347be9c1152d447467897" } new_table_name: "default.Table_f53b7da399c043038b4c6d59e93bc943" modify_external_catalogs: true
I20251211 20:52:23.637562 1160 tablet_bootstrap.cc:654] T 531a74c1fef34b2dba6d4a732e13ce38 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.638628 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 62fb046960024a2b9d70dc46ae52d53f with delete_type TABLET_DATA_DELETED (replaced by tablet ec7df7b11ce34b8489048e6acdcb4e94) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:23.640954 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet d824bc02b25842d18f28f711eef6669f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d824bc02b25842d18f28f711eef6669f already in progress: creating tablet
W20251211 20:52:23.641573 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 8dff13e22899452c8d4f2731b8dac7d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8dff13e22899452c8d4f2731b8dac7d5 already in progress: creating tablet
I20251211 20:52:23.641701 1166 tablet_bootstrap.cc:492] T 034f47d8564f4094a9d76aee2fea3119 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:23.641880 1166 ts_tablet_manager.cc:1403] T 034f47d8564f4094a9d76aee2fea3119 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.008s
W20251211 20:52:23.642136 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet f00f7c1f6a4e44ff89341b4a55cc93fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f00f7c1f6a4e44ff89341b4a55cc93fb already in progress: creating tablet
W20251211 20:52:23.642699 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 97a519a8ca7c4f5380a230a70ee84052 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 97a519a8ca7c4f5380a230a70ee84052 already in progress: creating tablet
I20251211 20:52:23.642720 1166 raft_consensus.cc:359] T 034f47d8564f4094a9d76aee2fea3119 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.642926 1166 raft_consensus.cc:385] T 034f47d8564f4094a9d76aee2fea3119 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.643007 1165 tablet_replica.cc:333] T 8e9eded009e44c1a9544d4471c04c5c8 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:23.643006 1166 raft_consensus.cc:740] T 034f47d8564f4094a9d76aee2fea3119 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.643635 1166 consensus_queue.cc:260] T 034f47d8564f4094a9d76aee2fea3119 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.644361 1165 raft_consensus.cc:2243] T 8e9eded009e44c1a9544d4471c04c5c8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.644582 1165 raft_consensus.cc:2272] T 8e9eded009e44c1a9544d4471c04c5c8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.646822 1166 ts_tablet_manager.cc:1434] T 034f47d8564f4094a9d76aee2fea3119 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20251211 20:52:23.648186 1166 tablet_bootstrap.cc:492] T 73aa2a3fb2f44bbca870fb7f4cf6087c P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:23.648451 1165 ts_tablet_manager.cc:1916] T 8e9eded009e44c1a9544d4471c04c5c8 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.650148 1166 tablet_bootstrap.cc:654] T 73aa2a3fb2f44bbca870fb7f4cf6087c P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.650799 1236 raft_consensus.cc:493] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:23.650990 1236 raft_consensus.cc:515] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.652197 1236 leader_election.cc:290] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:23.653162 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dc7e8db4b51c415e86d8c24a76b8cb1e" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:23.655663 847 leader_election.cc:343] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: dc7e8db4b51c415e86d8c24a76b8cb1e
W20251211 20:52:23.655908 847 leader_election.cc:343] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:23.656010 847 leader_election.cc:304] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:23.658179 1166 tablet_bootstrap.cc:492] T 73aa2a3fb2f44bbca870fb7f4cf6087c P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:23.658341 1166 ts_tablet_manager.cc:1403] T 73aa2a3fb2f44bbca870fb7f4cf6087c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.009s
I20251211 20:52:23.658970 1165 ts_tablet_manager.cc:1929] T 8e9eded009e44c1a9544d4471c04c5c8 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.659083 1165 log.cc:1199] T 8e9eded009e44c1a9544d4471c04c5c8 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/8e9eded009e44c1a9544d4471c04c5c8
I20251211 20:52:23.659179 1166 raft_consensus.cc:359] T 73aa2a3fb2f44bbca870fb7f4cf6087c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.659391 1166 raft_consensus.cc:385] T 73aa2a3fb2f44bbca870fb7f4cf6087c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.659447 1165 ts_tablet_manager.cc:1950] T 8e9eded009e44c1a9544d4471c04c5c8 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:23.659473 1166 raft_consensus.cc:740] T 73aa2a3fb2f44bbca870fb7f4cf6087c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.659839 1166 consensus_queue.cc:260] T 73aa2a3fb2f44bbca870fb7f4cf6087c P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.660339 1236 raft_consensus.cc:2749] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:23.661180 1164 ts_tablet_manager.cc:1929] T deacd1f8f3bf4b1f923b61b312492104 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.664162 1166 ts_tablet_manager.cc:1434] T 73aa2a3fb2f44bbca870fb7f4cf6087c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251211 20:52:23.665150 1168 tablet_bootstrap.cc:492] T deacd1f8f3bf4b1f923b61b312492104 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:23.665376 1168 ts_tablet_manager.cc:1403] T deacd1f8f3bf4b1f923b61b312492104 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.051s user 0.003s sys 0.004s
I20251211 20:52:23.665472 1166 tablet_bootstrap.cc:492] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:23.666247 1168 raft_consensus.cc:359] T deacd1f8f3bf4b1f923b61b312492104 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.666570 1164 log.cc:1199] T deacd1f8f3bf4b1f923b61b312492104 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/deacd1f8f3bf4b1f923b61b312492104
I20251211 20:52:23.666985 1164 ts_tablet_manager.cc:1950] T deacd1f8f3bf4b1f923b61b312492104 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:23.667131 1168 raft_consensus.cc:385] T deacd1f8f3bf4b1f923b61b312492104 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.669798 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0d02d938bc90411d93a49209080220f0" } new_table_name: "default.Table_5b8aa62e6fdf46089c4c8ae541856fbe" modify_external_catalogs: true
I20251211 20:52:23.669945 1168 raft_consensus.cc:740] T deacd1f8f3bf4b1f923b61b312492104 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.670650 1166 tablet_bootstrap.cc:654] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.671181 1168 consensus_queue.cc:260] T deacd1f8f3bf4b1f923b61b312492104 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.672591 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_154a7364b43244ecb91070b153a641a6" } new_table_name: "default.Table_8ab2da28cfc84df5b1f6efe450cde2a3" modify_external_catalogs: true
I20251211 20:52:23.674086 1168 ts_tablet_manager.cc:1434] T deacd1f8f3bf4b1f923b61b312492104 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20251211 20:52:23.677088 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet deacd1f8f3bf4b1f923b61b312492104 (table default.Table_872ae36ce97749d39238287355dadbe1 [id=98bd7ff031214ba7a5e974da798e1248]) successfully deleted
W20251211 20:52:23.677821 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 8dff13e22899452c8d4f2731b8dac7d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8dff13e22899452c8d4f2731b8dac7d5 already in progress: creating tablet
I20251211 20:52:23.680454 1166 tablet_bootstrap.cc:492] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:23.680661 1166 ts_tablet_manager.cc:1403] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.006s
I20251211 20:52:23.681586 1166 raft_consensus.cc:359] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.681824 1166 raft_consensus.cc:385] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.681902 1166 raft_consensus.cc:740] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.682204 1166 consensus_queue.cc:260] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.686595 1166 ts_tablet_manager.cc:1434] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.006s user 0.001s sys 0.002s
I20251211 20:52:23.690644 1166 tablet_bootstrap.cc:492] T e69d53ac90a7463aa36ff15dadad7e0b P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:23.692498 1168 tablet_bootstrap.cc:492] T 62fb046960024a2b9d70dc46ae52d53f P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:23.694069 1166 tablet_bootstrap.cc:654] T e69d53ac90a7463aa36ff15dadad7e0b P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.694862 1168 tablet_bootstrap.cc:654] T 62fb046960024a2b9d70dc46ae52d53f P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.696188 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_2bdd381fdf6443da869b647ddb92699b" } modify_external_catalogs: true
I20251211 20:52:23.696362 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_2bdd381fdf6443da869b647ddb92699b" } modify_external_catalogs: true
I20251211 20:52:23.696595 1160 tablet_bootstrap.cc:492] T 531a74c1fef34b2dba6d4a732e13ce38 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:23.696739 1160 ts_tablet_manager.cc:1403] T 531a74c1fef34b2dba6d4a732e13ce38 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.070s user 0.005s sys 0.000s
I20251211 20:52:23.697070 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8352c9da1e324dd4847fd22078f22e53" } modify_external_catalogs: true
I20251211 20:52:23.697206 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8352c9da1e324dd4847fd22078f22e53" } modify_external_catalogs: true
I20251211 20:52:23.697575 1160 raft_consensus.cc:359] T 531a74c1fef34b2dba6d4a732e13ce38 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.697741 1160 raft_consensus.cc:385] T 531a74c1fef34b2dba6d4a732e13ce38 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.697820 1160 raft_consensus.cc:740] T 531a74c1fef34b2dba6d4a732e13ce38 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.698033 1160 consensus_queue.cc:260] T 531a74c1fef34b2dba6d4a732e13ce38 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.698848 592 master_service.cc:954] ReplaceTablet: received request to replace tablet bc51bc0821a8484caee39c534eb4e6af from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:23.699518 744 tablet_service.cc:1552] Processing DeleteTablet for tablet deacd1f8f3bf4b1f923b61b312492104 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.700870 1160 ts_tablet_manager.cc:1434] T 531a74c1fef34b2dba6d4a732e13ce38 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251211 20:52:23.701218 1160 tablet_bootstrap.cc:492] T 35a49c9f560c47049bdcbeddd55ccb28 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:23.702884 1160 tablet_bootstrap.cc:654] T 35a49c9f560c47049bdcbeddd55ccb28 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.704684 1164 tablet_replica.cc:333] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:23.704856 1164 raft_consensus.cc:2243] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.705040 1164 raft_consensus.cc:2272] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.705739 1164 ts_tablet_manager.cc:1916] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:23.704891 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet f00f7c1f6a4e44ff89341b4a55cc93fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f00f7c1f6a4e44ff89341b4a55cc93fb already in progress: creating tablet
W20251211 20:52:23.706306 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 97a519a8ca7c4f5380a230a70ee84052 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 97a519a8ca7c4f5380a230a70ee84052 already in progress: creating tablet
I20251211 20:52:23.706722 1166 tablet_bootstrap.cc:492] T e69d53ac90a7463aa36ff15dadad7e0b P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:23.706897 1166 ts_tablet_manager.cc:1403] T e69d53ac90a7463aa36ff15dadad7e0b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.006s
W20251211 20:52:23.706939 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet c5b164dc0ce1461fae98d950f7c7a726 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c5b164dc0ce1461fae98d950f7c7a726 already in progress: creating tablet
I20251211 20:52:23.707852 1166 raft_consensus.cc:359] T e69d53ac90a7463aa36ff15dadad7e0b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.708069 1166 raft_consensus.cc:385] T e69d53ac90a7463aa36ff15dadad7e0b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.708866 1166 raft_consensus.cc:740] T e69d53ac90a7463aa36ff15dadad7e0b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.709129 1166 consensus_queue.cc:260] T e69d53ac90a7463aa36ff15dadad7e0b P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.711522 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_307027e1160747b98c2fb2ea31eb8745" } modify_external_catalogs: true
I20251211 20:52:23.712003 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_307027e1160747b98c2fb2ea31eb8745" } modify_external_catalogs: true
I20251211 20:52:23.713475 1166 ts_tablet_manager.cc:1434] T e69d53ac90a7463aa36ff15dadad7e0b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20251211 20:52:23.713579 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 531a74c1fef34b2dba6d4a732e13ce38 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:23.717367 1166 tablet_bootstrap.cc:492] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:23.719353 1166 tablet_bootstrap.cc:654] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:23.722002 1274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:52:23.724826 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 8e9eded009e44c1a9544d4471c04c5c8 (table default.Table_19db6b94abda4124a0e45f11b8ca0990 [id=383392d835da4c50848068d83261cecc]) successfully deleted
W20251211 20:52:23.725561 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet c5b164dc0ce1461fae98d950f7c7a726 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c5b164dc0ce1461fae98d950f7c7a726 already in progress: creating tablet
I20251211 20:52:23.725992 1165 tablet_replica.cc:333] T d0df24ce65234eeba97f9fdd351b58ad P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
W20251211 20:52:23.726011 1276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:52:23.726287 1165 raft_consensus.cc:2243] T d0df24ce65234eeba97f9fdd351b58ad P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.726548 1165 raft_consensus.cc:2272] T d0df24ce65234eeba97f9fdd351b58ad P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.727353 1165 ts_tablet_manager.cc:1916] T d0df24ce65234eeba97f9fdd351b58ad P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.727480 1166 tablet_bootstrap.cc:492] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:23.727674 1166 ts_tablet_manager.cc:1403] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.005s
I20251211 20:52:23.728469 1166 raft_consensus.cc:359] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.728682 1166 raft_consensus.cc:385] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.728768 1166 raft_consensus.cc:740] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.729048 1166 consensus_queue.cc:260] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.729571 1168 tablet_bootstrap.cc:492] T 62fb046960024a2b9d70dc46ae52d53f P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:23.729758 1168 ts_tablet_manager.cc:1403] T 62fb046960024a2b9d70dc46ae52d53f P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.037s user 0.003s sys 0.001s
I20251211 20:52:23.731123 1168 raft_consensus.cc:359] T 62fb046960024a2b9d70dc46ae52d53f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.731370 1168 raft_consensus.cc:385] T 62fb046960024a2b9d70dc46ae52d53f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.731527 1168 raft_consensus.cc:740] T 62fb046960024a2b9d70dc46ae52d53f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.731832 1168 consensus_queue.cc:260] T 62fb046960024a2b9d70dc46ae52d53f P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.732195 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_242ad5ed8a5a48cf9d57630363e9b3c5" } modify_external_catalogs: true
I20251211 20:52:23.732362 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_242ad5ed8a5a48cf9d57630363e9b3c5" } modify_external_catalogs: true
I20251211 20:52:23.732801 1029 tablet_service.cc:1505] Processing CreateTablet for tablet c4e048ca28824329b924845ccf5067cf (DEFAULT_TABLE table=default.Table_943bd9d0a2d848e2bb986aee9bc1d718 [id=6b55ad645bec4a7daefc996978420295]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.733223 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4e048ca28824329b924845ccf5067cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.733652 1024 tablet_service.cc:1505] Processing CreateTablet for tablet 78922f26cedb4bb989aa1c1cb40bfa3a (DEFAULT_TABLE table=default.Table_736d6c2eb4dd4eb9a72dca6b9ed2073f [id=289e09e5a445461a9d0949c41ddcbd18]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.733891 741 tablet_service.cc:1505] Processing CreateTablet for tablet dc40b9826ea84728910b1fd61ce62a25 (DEFAULT_TABLE table=default.Table_880a8408f1fb4d68a3f9d30b26acffa5 [id=db38368266194271a02ad265fd991439]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.734001 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78922f26cedb4bb989aa1c1cb40bfa3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.734269 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc40b9826ea84728910b1fd61ce62a25. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.735877 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 6a206b37e43247b5babe3113eab2c9fb (DEFAULT_TABLE table=default.Table_028df988934346debc0f8c62b4740309 [id=e1949419f3924474bf195ac17a794497]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.736130 1021 tablet_service.cc:1505] Processing CreateTablet for tablet 0246f1df99684d0da43d97acbf1182a0 (DEFAULT_TABLE table=default.Table_03fdaef4fcc144f485f238db0e76fab4 [id=4264180c89854054b14e98d5a49984fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.736280 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a206b37e43247b5babe3113eab2c9fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.736524 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0246f1df99684d0da43d97acbf1182a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.736792 744 tablet_service.cc:1505] Processing CreateTablet for tablet c4e048ca28824329b924845ccf5067cf (DEFAULT_TABLE table=default.Table_943bd9d0a2d848e2bb986aee9bc1d718 [id=6b55ad645bec4a7daefc996978420295]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.735882 1022 tablet_service.cc:1505] Processing CreateTablet for tablet dc40b9826ea84728910b1fd61ce62a25 (DEFAULT_TABLE table=default.Table_880a8408f1fb4d68a3f9d30b26acffa5 [id=db38368266194271a02ad265fd991439]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.737805 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc40b9826ea84728910b1fd61ce62a25. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.742095 1027 tablet_service.cc:1505] Processing CreateTablet for tablet b8828385cec34d70b717ae7f3813b9c3 (DEFAULT_TABLE table=default.Table_8352c9da1e324dd4847fd22078f22e53 [id=9dc54153dda4495684c2b07732795bfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.742585 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8828385cec34d70b717ae7f3813b9c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.743283 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4e048ca28824329b924845ccf5067cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.745963 738 tablet_service.cc:1505] Processing CreateTablet for tablet 4b736a1b347d44b398980654caa21332 (DEFAULT_TABLE table=default.Table_f9014306b36e47b98c8e4c6070ea4a37 [id=4e64e54fb4fc42d48679371e4bc1efe5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.746372 1023 tablet_service.cc:1505] Processing CreateTablet for tablet c38c1a4c0e574eb0957de127d41c26ad (DEFAULT_TABLE table=default.Table_88975c1ad9604f50b7f69fbbdacabce3 [id=52f34b88d367454286943a27138b20f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.746606 754 tablet_service.cc:1505] Processing CreateTablet for tablet 0246f1df99684d0da43d97acbf1182a0 (DEFAULT_TABLE table=default.Table_03fdaef4fcc144f485f238db0e76fab4 [id=4264180c89854054b14e98d5a49984fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.746793 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c38c1a4c0e574eb0957de127d41c26ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.746944 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0246f1df99684d0da43d97acbf1182a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.747325 749 tablet_service.cc:1505] Processing CreateTablet for tablet 6a206b37e43247b5babe3113eab2c9fb (DEFAULT_TABLE table=default.Table_028df988934346debc0f8c62b4740309 [id=e1949419f3924474bf195ac17a794497]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.747643 739 tablet_service.cc:1505] Processing CreateTablet for tablet c38c1a4c0e574eb0957de127d41c26ad (DEFAULT_TABLE table=default.Table_88975c1ad9604f50b7f69fbbdacabce3 [id=52f34b88d367454286943a27138b20f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.747817 752 tablet_service.cc:1505] Processing CreateTablet for tablet 78922f26cedb4bb989aa1c1cb40bfa3a (DEFAULT_TABLE table=default.Table_736d6c2eb4dd4eb9a72dca6b9ed2073f [id=289e09e5a445461a9d0949c41ddcbd18]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.747934 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 4b736a1b347d44b398980654caa21332 (DEFAULT_TABLE table=default.Table_f9014306b36e47b98c8e4c6070ea4a37 [id=4e64e54fb4fc42d48679371e4bc1efe5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.748046 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 62fb046960024a2b9d70dc46ae52d53f with delete_type TABLET_DATA_DELETED (replaced by tablet ec7df7b11ce34b8489048e6acdcb4e94) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.748298 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a206b37e43247b5babe3113eab2c9fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.748453 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b736a1b347d44b398980654caa21332. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.747915 750 tablet_service.cc:1505] Processing CreateTablet for tablet b8828385cec34d70b717ae7f3813b9c3 (DEFAULT_TABLE table=default.Table_8352c9da1e324dd4847fd22078f22e53 [id=9dc54153dda4495684c2b07732795bfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.750510 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78922f26cedb4bb989aa1c1cb40bfa3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.751150 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8828385cec34d70b717ae7f3813b9c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.751813 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c38c1a4c0e574eb0957de127d41c26ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.752746 738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b736a1b347d44b398980654caa21332. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.755189 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet e69d53ac90a7463aa36ff15dadad7e0b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.755509 1028 tablet_service.cc:1552] Processing DeleteTablet for tablet 034f47d8564f4094a9d76aee2fea3119 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.755591 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 73aa2a3fb2f44bbca870fb7f4cf6087c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.755800 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet b815bca893a8446e92d48973d9a0408e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.755908 1163 tablet_replica.cc:333] T e69d53ac90a7463aa36ff15dadad7e0b P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:23.754936 1168 ts_tablet_manager.cc:1434] T 62fb046960024a2b9d70dc46ae52d53f P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.025s user 0.003s sys 0.000s
I20251211 20:52:23.756101 1163 raft_consensus.cc:2243] T e69d53ac90a7463aa36ff15dadad7e0b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.756294 1163 raft_consensus.cc:2272] T e69d53ac90a7463aa36ff15dadad7e0b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.757290 1163 ts_tablet_manager.cc:1916] T e69d53ac90a7463aa36ff15dadad7e0b P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.758225 1168 tablet_bootstrap.cc:492] T dc7e8db4b51c415e86d8c24a76b8cb1e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:23.763768 1215 raft_consensus.cc:493] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:23.763985 1215 raft_consensus.cc:515] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.764667 1215 leader_election.cc:290] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:23.768055 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b815bca893a8446e92d48973d9a0408e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:23.768841 982 leader_election.cc:343] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:23.782907 591 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 694518e6c1ae4032b06b99e9bde2e5dd consensus state has no leader
I20251211 20:52:23.783330 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_9540f76e3f984e92b632bc45a25f4b8c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:23.783874 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c58241f7e1404a8c8db92c4f9a47d143 consensus state has no leader
W20251211 20:52:23.784432 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9540f76e3f984e92b632bc45a25f4b8c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:23.785956 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9ac73058eb564b63871197be450065f0" } modify_external_catalogs: true
I20251211 20:52:23.786201 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9ac73058eb564b63871197be450065f0" } modify_external_catalogs: true
I20251211 20:52:23.792840 1163 ts_tablet_manager.cc:1929] T e69d53ac90a7463aa36ff15dadad7e0b P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.793035 1163 log.cc:1199] T e69d53ac90a7463aa36ff15dadad7e0b P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/e69d53ac90a7463aa36ff15dadad7e0b
I20251211 20:52:23.793406 1163 ts_tablet_manager.cc:1950] T e69d53ac90a7463aa36ff15dadad7e0b P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:23.794570 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet e69d53ac90a7463aa36ff15dadad7e0b (table default.Table_2645aa306b5c44f6bad5e2e67a91ebec [id=0ec7cf10eb8a47aca499b141913b38cf]) successfully deleted
I20251211 20:52:23.796026 1166 ts_tablet_manager.cc:1434] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.068s user 0.002s sys 0.000s
I20251211 20:52:23.799054 1166 tablet_bootstrap.cc:492] T 5ed0a0141051462eb45ca3a66d41d009 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:23.801000 1166 tablet_bootstrap.cc:654] T 5ed0a0141051462eb45ca3a66d41d009 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.802109 1163 tablet_replica.cc:333] T 034f47d8564f4094a9d76aee2fea3119 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
W20251211 20:52:23.802162 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bd8fd6c977204c90b558827fc0b15f43 consensus state has no leader
I20251211 20:52:23.802330 1163 raft_consensus.cc:2243] T 034f47d8564f4094a9d76aee2fea3119 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.802510 1163 raft_consensus.cc:2272] T 034f47d8564f4094a9d76aee2fea3119 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.803373 1163 ts_tablet_manager.cc:1916] T 034f47d8564f4094a9d76aee2fea3119 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.809270 1168 tablet_bootstrap.cc:654] T dc7e8db4b51c415e86d8c24a76b8cb1e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.811064 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_01bf09a9b23a450493cea3c5bf9bdfe2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:23.811769 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_01bf09a9b23a450493cea3c5bf9bdfe2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:23.815817 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8308ad46551447439a248d20ed1e5922 consensus state has no leader
I20251211 20:52:23.815877 1166 tablet_bootstrap.cc:492] T 5ed0a0141051462eb45ca3a66d41d009 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:23.816040 1166 ts_tablet_manager.cc:1403] T 5ed0a0141051462eb45ca3a66d41d009 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.017s user 0.014s sys 0.000s
I20251211 20:52:23.816887 1166 raft_consensus.cc:359] T 5ed0a0141051462eb45ca3a66d41d009 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.817085 1166 raft_consensus.cc:385] T 5ed0a0141051462eb45ca3a66d41d009 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.817178 1166 raft_consensus.cc:740] T 5ed0a0141051462eb45ca3a66d41d009 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.817387 1166 consensus_queue.cc:260] T 5ed0a0141051462eb45ca3a66d41d009 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.819090 1165 ts_tablet_manager.cc:1929] T d0df24ce65234eeba97f9fdd351b58ad P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.819253 1165 log.cc:1199] T d0df24ce65234eeba97f9fdd351b58ad P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/d0df24ce65234eeba97f9fdd351b58ad
I20251211 20:52:23.819337 1163 ts_tablet_manager.cc:1929] T 034f47d8564f4094a9d76aee2fea3119 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.819504 1163 log.cc:1199] T 034f47d8564f4094a9d76aee2fea3119 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/034f47d8564f4094a9d76aee2fea3119
I20251211 20:52:23.820547 1168 tablet_bootstrap.cc:492] T dc7e8db4b51c415e86d8c24a76b8cb1e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:23.820583 1163 ts_tablet_manager.cc:1950] T 034f47d8564f4094a9d76aee2fea3119 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:23.820704 1168 ts_tablet_manager.cc:1403] T dc7e8db4b51c415e86d8c24a76b8cb1e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.063s user 0.003s sys 0.000s
I20251211 20:52:23.821291 1166 ts_tablet_manager.cc:1434] T 5ed0a0141051462eb45ca3a66d41d009 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251211 20:52:23.821571 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 034f47d8564f4094a9d76aee2fea3119 (table default.Table_b5a3abc502a64d6083d78344400b5eeb [id=c58ec33f45c348329be4acee3d5f4f6f]) successfully deleted
I20251211 20:52:23.822049 1168 raft_consensus.cc:359] T dc7e8db4b51c415e86d8c24a76b8cb1e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.822238 1168 raft_consensus.cc:385] T dc7e8db4b51c415e86d8c24a76b8cb1e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.822300 1168 raft_consensus.cc:740] T dc7e8db4b51c415e86d8c24a76b8cb1e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.822537 1168 consensus_queue.cc:260] T dc7e8db4b51c415e86d8c24a76b8cb1e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.823984 1163 tablet_replica.cc:333] T 73aa2a3fb2f44bbca870fb7f4cf6087c P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:23.824188 1163 raft_consensus.cc:2243] T 73aa2a3fb2f44bbca870fb7f4cf6087c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.823940 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 5ed0a0141051462eb45ca3a66d41d009 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.824378 1163 raft_consensus.cc:2272] T 73aa2a3fb2f44bbca870fb7f4cf6087c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.824653 1166 tablet_bootstrap.cc:492] T edc4300dc5a84173aa95b587d77fc95c P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:23.825201 1163 ts_tablet_manager.cc:1916] T 73aa2a3fb2f44bbca870fb7f4cf6087c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.825378 1204 server_base.cc:1047] running on GCE node
I20251211 20:52:23.826339 1166 tablet_bootstrap.cc:654] T edc4300dc5a84173aa95b587d77fc95c P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:23.826341 1280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:52:23.827549 1204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:52:23.827620 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_29806e5c967a42e6b62d8268f10aa458" } new_table_name: "default.Table_265379fa17b74d0f954a2a2c4de113c8" modify_external_catalogs: true
I20251211 20:52:23.830506 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 1c04187b8ed146c289a6b17d62b07199
I20251211 20:52:23.831629 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c04187b8ed146c289a6b17d62b07199 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.831900 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c04187b8ed146c289a6b17d62b07199 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.840957 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_40b5190e58a440e9a13d6c9221572914"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:23.840142 741 tablet_service.cc:1552] Processing DeleteTablet for tablet dc7e8db4b51c415e86d8c24a76b8cb1e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:21 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.840168 1165 ts_tablet_manager.cc:1950] T d0df24ce65234eeba97f9fdd351b58ad P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
W20251211 20:52:23.841588 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_40b5190e58a440e9a13d6c9221572914 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:23.842392 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet d0df24ce65234eeba97f9fdd351b58ad (table default.Table_1b829e0ef7ef4b17be671db434f3858c [id=b8c401dfdb5f4f7584dca0989287ce31]) successfully deleted
I20251211 20:52:23.839618 1168 ts_tablet_manager.cc:1434] T dc7e8db4b51c415e86d8c24a76b8cb1e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.019s user 0.001s sys 0.002s
I20251211 20:52:23.842908 1165 tablet_replica.cc:333] T deacd1f8f3bf4b1f923b61b312492104 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:23.843068 1165 raft_consensus.cc:2243] T deacd1f8f3bf4b1f923b61b312492104 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.843114 1163 ts_tablet_manager.cc:1929] T 73aa2a3fb2f44bbca870fb7f4cf6087c P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.843250 1165 raft_consensus.cc:2272] T deacd1f8f3bf4b1f923b61b312492104 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.843276 1163 log.cc:1199] T 73aa2a3fb2f44bbca870fb7f4cf6087c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/73aa2a3fb2f44bbca870fb7f4cf6087c
I20251211 20:52:23.845240 1204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:52:23.845448 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet b8828385cec34d70b717ae7f3813b9c3
I20251211 20:52:23.845769 1163 ts_tablet_manager.cc:1950] T 73aa2a3fb2f44bbca870fb7f4cf6087c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:23.846411 741 tablet_service.cc:1552] Processing DeleteTablet for tablet b8828385cec34d70b717ae7f3813b9c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.846417 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet b8828385cec34d70b717ae7f3813b9c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.847282 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 73aa2a3fb2f44bbca870fb7f4cf6087c (table default.Table_3fca4e8f1b6a427d8d016e27f88ea1ce [id=4f93b23edd4548aeb6bcb8f92878907d]) successfully deleted
I20251211 20:52:23.847890 1163 tablet_replica.cc:333] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:23.848124 1163 raft_consensus.cc:2243] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.848301 1163 raft_consensus.cc:2272] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.849107 1168 tablet_bootstrap.cc:492] T f37f598a720d4161b13c03425e020fa5 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:23.850294 1163 ts_tablet_manager.cc:1916] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.851223 1165 ts_tablet_manager.cc:1916] T deacd1f8f3bf4b1f923b61b312492104 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.852734 1204 hybrid_clock.cc:648] HybridClock initialized: now 1765486343852619 us; error 93 us; skew 500 ppm
I20251211 20:52:23.853315 1204 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:52:23.856113 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet bc51bc0821a8484caee39c534eb4e6af
I20251211 20:52:23.856302 1168 tablet_bootstrap.cc:654] T f37f598a720d4161b13c03425e020fa5 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.856638 592 catalog_manager.cc:6724] ReplaceTablet: tablet bc51bc0821a8484caee39c534eb4e6af deleted and replaced by tablet 6a2442a4ebc940a281f9ea1a394f6621
I20251211 20:52:23.857107 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet bc51bc0821a8484caee39c534eb4e6af with delete_type TABLET_DATA_DELETED (replaced by tablet 6a2442a4ebc940a281f9ea1a394f6621) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:23.857331 741 tablet_service.cc:1552] Processing DeleteTablet for tablet bc51bc0821a8484caee39c534eb4e6af with delete_type TABLET_DATA_DELETED (replaced by tablet 6a2442a4ebc940a281f9ea1a394f6621) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.864919 1168 tablet_bootstrap.cc:492] T f37f598a720d4161b13c03425e020fa5 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:23.865097 1168 ts_tablet_manager.cc:1403] T f37f598a720d4161b13c03425e020fa5 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.000s
I20251211 20:52:23.865954 1168 raft_consensus.cc:359] T f37f598a720d4161b13c03425e020fa5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.866171 1168 raft_consensus.cc:385] T f37f598a720d4161b13c03425e020fa5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.866273 1168 raft_consensus.cc:740] T f37f598a720d4161b13c03425e020fa5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.866544 1168 consensus_queue.cc:260] T f37f598a720d4161b13c03425e020fa5 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.867106 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b815bca893a8446e92d48973d9a0408e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:23.867913 983 leader_election.cc:343] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:23.868090 983 leader_election.cc:304] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:23.868275 983 raft_consensus.cc:2686] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251211 20:52:23.868597 890 tablet_service.cc:1505] Processing CreateTablet for tablet 6a206b37e43247b5babe3113eab2c9fb (DEFAULT_TABLE table=default.Table_028df988934346debc0f8c62b4740309 [id=e1949419f3924474bf195ac17a794497]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.869257 1163 ts_tablet_manager.cc:1929] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.869427 1163 log.cc:1199] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/b815bca893a8446e92d48973d9a0408e
I20251211 20:52:23.869807 1163 ts_tablet_manager.cc:1950] T b815bca893a8446e92d48973d9a0408e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:23.870187 1165 ts_tablet_manager.cc:1929] T deacd1f8f3bf4b1f923b61b312492104 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.870307 1165 log.cc:1199] T deacd1f8f3bf4b1f923b61b312492104 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/deacd1f8f3bf4b1f923b61b312492104
I20251211 20:52:23.870694 891 tablet_service.cc:1505] Processing CreateTablet for tablet c38c1a4c0e574eb0957de127d41c26ad (DEFAULT_TABLE table=default.Table_88975c1ad9604f50b7f69fbbdacabce3 [id=52f34b88d367454286943a27138b20f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.871023 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c04187b8ed146c289a6b17d62b07199 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:23.871228 887 tablet_service.cc:1552] Processing DeleteTablet for tablet bc51bc0821a8484caee39c534eb4e6af with delete_type TABLET_DATA_DELETED (replaced by tablet 6a2442a4ebc940a281f9ea1a394f6621) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:23.871172 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet b815bca893a8446e92d48973d9a0408e (table default.Table_b411ab6a2d934ad8aa183b348dc24d85 [id=fd269c697f9d4c5eb4a0d646f459c954]) successfully deleted
I20251211 20:52:23.871362 881 tablet_service.cc:1552] Processing DeleteTablet for tablet b8828385cec34d70b717ae7f3813b9c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:23.872015 894 tablet_service.cc:1505] Processing CreateTablet for tablet 0246f1df99684d0da43d97acbf1182a0 (DEFAULT_TABLE table=default.Table_03fdaef4fcc144f485f238db0e76fab4 [id=4264180c89854054b14e98d5a49984fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.875891 1215 raft_consensus.cc:493] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:23.876077 1215 raft_consensus.cc:515] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.876155 1168 ts_tablet_manager.cc:1434] T f37f598a720d4161b13c03425e020fa5 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20251211 20:52:23.876546 1163 tablet_replica.cc:333] T 5ed0a0141051462eb45ca3a66d41d009 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:23.876739 1163 raft_consensus.cc:2243] T 5ed0a0141051462eb45ca3a66d41d009 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.876850 1215 leader_election.cc:290] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:23.876935 1163 raft_consensus.cc:2272] T 5ed0a0141051462eb45ca3a66d41d009 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.878217 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "353da9106578450987a4f6544df8d7e6" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:23.878605 1163 ts_tablet_manager.cc:1916] T 5ed0a0141051462eb45ca3a66d41d009 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:23.878814 982 leader_election.cc:343] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:23.879465 1168 tablet_bootstrap.cc:492] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:23.879804 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "353da9106578450987a4f6544df8d7e6" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:23.879918 883 tablet_service.cc:1505] Processing CreateTablet for tablet 78922f26cedb4bb989aa1c1cb40bfa3a (DEFAULT_TABLE table=default.Table_736d6c2eb4dd4eb9a72dca6b9ed2073f [id=289e09e5a445461a9d0949c41ddcbd18]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.881515 1168 tablet_bootstrap.cc:654] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:23.882104 983 leader_election.cc:343] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:23.882238 983 leader_election.cc:304] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:23.882781 1215 raft_consensus.cc:2749] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:23.884052 1165 ts_tablet_manager.cc:1950] T deacd1f8f3bf4b1f923b61b312492104 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:23.884193 884 tablet_service.cc:1505] Processing CreateTablet for tablet c4e048ca28824329b924845ccf5067cf (DEFAULT_TABLE table=default.Table_943bd9d0a2d848e2bb986aee9bc1d718 [id=6b55ad645bec4a7daefc996978420295]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.884842 889 tablet_service.cc:1505] Processing CreateTablet for tablet b8828385cec34d70b717ae7f3813b9c3 (DEFAULT_TABLE table=default.Table_8352c9da1e324dd4847fd22078f22e53 [id=9dc54153dda4495684c2b07732795bfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.885175 878 tablet_service.cc:1505] Processing CreateTablet for tablet dc40b9826ea84728910b1fd61ce62a25 (DEFAULT_TABLE table=default.Table_880a8408f1fb4d68a3f9d30b26acffa5 [id=db38368266194271a02ad265fd991439]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.885473 880 tablet_service.cc:1505] Processing CreateTablet for tablet 4b736a1b347d44b398980654caa21332 (DEFAULT_TABLE table=default.Table_f9014306b36e47b98c8e4c6070ea4a37 [id=4e64e54fb4fc42d48679371e4bc1efe5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:23.885987 1163 ts_tablet_manager.cc:1929] T 5ed0a0141051462eb45ca3a66d41d009 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.886051 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0246f1df99684d0da43d97acbf1182a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.886132 1163 log.cc:1199] T 5ed0a0141051462eb45ca3a66d41d009 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/5ed0a0141051462eb45ca3a66d41d009
I20251211 20:52:23.886543 1163 ts_tablet_manager.cc:1950] T 5ed0a0141051462eb45ca3a66d41d009 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:23.887769 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 5ed0a0141051462eb45ca3a66d41d009 (table default.Table_40256d3973c74566a09b1d235062d284 [id=1f4650c549d94e2c9ef753ff96670aa6]) successfully deleted
I20251211 20:52:23.888092 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet deacd1f8f3bf4b1f923b61b312492104 (table default.Table_872ae36ce97749d39238287355dadbe1 [id=98bd7ff031214ba7a5e974da798e1248]) successfully deleted
I20251211 20:52:23.888505 1204 webserver.cc:492] Webserver started at http://127.0.104.62:37413/ using document root <none> and password file <none>
I20251211 20:52:23.888835 1165 tablet_replica.cc:333] T 62fb046960024a2b9d70dc46ae52d53f P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:23.889027 1165 raft_consensus.cc:2243] T 62fb046960024a2b9d70dc46ae52d53f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.889178 1165 raft_consensus.cc:2272] T 62fb046960024a2b9d70dc46ae52d53f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.890034 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8828385cec34d70b717ae7f3813b9c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.889693 1204 fs_manager.cc:362] Metadata directory not provided
I20251211 20:52:23.890209 1204 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:52:23.890743 1165 ts_tablet_manager.cc:1916] T 62fb046960024a2b9d70dc46ae52d53f P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.892598 1168 tablet_bootstrap.cc:492] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:23.892743 1168 ts_tablet_manager.cc:1403] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20251211 20:52:23.894042 1168 raft_consensus.cc:359] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.894240 1168 raft_consensus.cc:385] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.894296 1168 raft_consensus.cc:740] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.896068 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a206b37e43247b5babe3113eab2c9fb. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:23.898631 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 1c04187b8ed146c289a6b17d62b07199 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c04187b8ed146c289a6b17d62b07199 already in progress: creating tablet
W20251211 20:52:23.899986 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet b8828385cec34d70b717ae7f3813b9c3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b8828385cec34d70b717ae7f3813b9c3 already in progress: creating tablet
I20251211 20:52:23.900744 1163 tablet_replica.cc:333] T bc51bc0821a8484caee39c534eb4e6af P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:23.900949 1163 raft_consensus.cc:2243] T bc51bc0821a8484caee39c534eb4e6af P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.900897 1168 consensus_queue.cc:260] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.901124 1163 raft_consensus.cc:2272] T bc51bc0821a8484caee39c534eb4e6af P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.902621 1163 ts_tablet_manager.cc:1916] T bc51bc0821a8484caee39c534eb4e6af P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.904150 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78922f26cedb4bb989aa1c1cb40bfa3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.905354 1168 ts_tablet_manager.cc:1434] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.013s user 0.002s sys 0.001s
I20251211 20:52:23.907786 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc40b9826ea84728910b1fd61ce62a25. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.909086 1168 tablet_bootstrap.cc:492] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:23.910458 1204 fs_manager.cc:714] Time spent opening directory manager: real 0.017s user 0.008s sys 0.000s
I20251211 20:52:23.910689 1168 tablet_bootstrap.cc:654] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.911687 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4e048ca28824329b924845ccf5067cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.916515 1165 ts_tablet_manager.cc:1929] T 62fb046960024a2b9d70dc46ae52d53f P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.917141 1163 ts_tablet_manager.cc:1929] T bc51bc0821a8484caee39c534eb4e6af P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.917325 1163 log.cc:1199] T bc51bc0821a8484caee39c534eb4e6af P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/bc51bc0821a8484caee39c534eb4e6af
I20251211 20:52:23.917757 1163 ts_tablet_manager.cc:1950] T bc51bc0821a8484caee39c534eb4e6af P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:23.918938 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet bc51bc0821a8484caee39c534eb4e6af (table default.Table_cbe68d84efef4745822f6aef5a723f11 [id=f9c1230a7d71435cac705d4675a9908d]) successfully deleted
I20251211 20:52:23.919169 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b736a1b347d44b398980654caa21332. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.920673 1165 log.cc:1199] T 62fb046960024a2b9d70dc46ae52d53f P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/62fb046960024a2b9d70dc46ae52d53f
I20251211 20:52:23.921033 1165 ts_tablet_manager.cc:1950] T 62fb046960024a2b9d70dc46ae52d53f P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:23.922049 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 62fb046960024a2b9d70dc46ae52d53f (table default.Table_02014bfadb7b434f960168f2db70b838 [id=14a0dbb8486246b09f3e06fc2120cd28]) successfully deleted
I20251211 20:52:23.919972 1168 tablet_bootstrap.cc:492] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:23.922194 1291 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:52:23.922365 1168 ts_tablet_manager.cc:1403] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.002s
I20251211 20:52:23.923839 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c38c1a4c0e574eb0957de127d41c26ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:23.923763 1168 raft_consensus.cc:359] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.923966 1168 raft_consensus.cc:385] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.924039 1168 raft_consensus.cc:740] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.924548 1204 fs_manager.cc:730] Time spent opening block manager: real 0.012s user 0.002s sys 0.004s
I20251211 20:52:23.924753 1204 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/data,/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/wal
uuid: "1830b0c58cfe495989f3d911dda0409a"
format_stamp: "Formatted at 2025-12-11 20:52:18 on dist-test-slave-fp9b"
W20251211 20:52:23.925979 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 1c04187b8ed146c289a6b17d62b07199 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c04187b8ed146c289a6b17d62b07199 already in progress: creating tablet
I20251211 20:52:23.925987 1204 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:52:23.926678 1165 tablet_replica.cc:333] T dc7e8db4b51c415e86d8c24a76b8cb1e P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:23.927351 1168 consensus_queue.cc:260] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.926851 1165 raft_consensus.cc:2243] T dc7e8db4b51c415e86d8c24a76b8cb1e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.931416 1165 raft_consensus.cc:2272] T dc7e8db4b51c415e86d8c24a76b8cb1e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.932895 1165 ts_tablet_manager.cc:1916] T dc7e8db4b51c415e86d8c24a76b8cb1e P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.939491 1165 ts_tablet_manager.cc:1929] T dc7e8db4b51c415e86d8c24a76b8cb1e P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.931965 1168 ts_tablet_manager.cc:1434] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20251211 20:52:23.941474 1168 tablet_bootstrap.cc:492] T 35a49c9f560c47049bdcbeddd55ccb28 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:23.941987 1160 tablet_bootstrap.cc:492] T 35a49c9f560c47049bdcbeddd55ccb28 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:23.942154 1160 ts_tablet_manager.cc:1403] T 35a49c9f560c47049bdcbeddd55ccb28 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.241s user 0.002s sys 0.027s
I20251211 20:52:23.942924 1160 raft_consensus.cc:359] T 35a49c9f560c47049bdcbeddd55ccb28 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.943140 1168 tablet_bootstrap.cc:654] T 35a49c9f560c47049bdcbeddd55ccb28 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.943146 1160 raft_consensus.cc:385] T 35a49c9f560c47049bdcbeddd55ccb28 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.943210 1160 raft_consensus.cc:740] T 35a49c9f560c47049bdcbeddd55ccb28 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.943446 1160 consensus_queue.cc:260] T 35a49c9f560c47049bdcbeddd55ccb28 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.943936 1160 ts_tablet_manager.cc:1434] T 35a49c9f560c47049bdcbeddd55ccb28 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:23.945935 1160 tablet_bootstrap.cc:492] T f37f598a720d4161b13c03425e020fa5 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:23.947974 1160 tablet_bootstrap.cc:654] T f37f598a720d4161b13c03425e020fa5 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.950758 1165 log.cc:1199] T dc7e8db4b51c415e86d8c24a76b8cb1e P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/dc7e8db4b51c415e86d8c24a76b8cb1e
I20251211 20:52:23.951341 1165 ts_tablet_manager.cc:1950] T dc7e8db4b51c415e86d8c24a76b8cb1e P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:23.952983 1168 tablet_bootstrap.cc:492] T 35a49c9f560c47049bdcbeddd55ccb28 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:23.953235 1168 ts_tablet_manager.cc:1403] T 35a49c9f560c47049bdcbeddd55ccb28 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.001s
I20251211 20:52:23.954052 1168 raft_consensus.cc:359] T 35a49c9f560c47049bdcbeddd55ccb28 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.954520 1168 raft_consensus.cc:385] T 35a49c9f560c47049bdcbeddd55ccb28 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.954650 1168 raft_consensus.cc:740] T 35a49c9f560c47049bdcbeddd55ccb28 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.954963 1168 consensus_queue.cc:260] T 35a49c9f560c47049bdcbeddd55ccb28 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.958731 1168 ts_tablet_manager.cc:1434] T 35a49c9f560c47049bdcbeddd55ccb28 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251211 20:52:23.961954 1168 tablet_bootstrap.cc:492] T aa685d1b051a4c93a2d313429d914ead P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:23.963776 1168 tablet_bootstrap.cc:654] T aa685d1b051a4c93a2d313429d914ead P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:23.964607 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet dc7e8db4b51c415e86d8c24a76b8cb1e (table default.Table_5ef3d8d2746a4016ad8efac84ba492b5 [id=ebae22b7d617403c84c023d0628f8ad8]) successfully deleted
W20251211 20:52:23.965348 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet b8828385cec34d70b717ae7f3813b9c3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b8828385cec34d70b717ae7f3813b9c3 already in progress: creating tablet
W20251211 20:52:23.966048 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet bc51bc0821a8484caee39c534eb4e6af because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc51bc0821a8484caee39c534eb4e6af already in progress: creating tablet
I20251211 20:52:23.970930 1236 raft_consensus.cc:493] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:23.971112 1236 raft_consensus.cc:515] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:23.971796 1236 leader_election.cc:290] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:23.972981 847 leader_election.cc:343] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 62fb046960024a2b9d70dc46ae52d53f
W20251211 20:52:23.973240 582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.61:40749, user_credentials={real_user=slave}} blocked reactor thread for 107719us
W20251211 20:52:23.973345 847 leader_election.cc:343] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 62fb046960024a2b9d70dc46ae52d53f
I20251211 20:52:23.973469 847 leader_election.cc:304] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:23.973920 1236 raft_consensus.cc:2749] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:23.978320 1168 tablet_bootstrap.cc:492] T aa685d1b051a4c93a2d313429d914ead P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:23.978560 1168 ts_tablet_manager.cc:1403] T aa685d1b051a4c93a2d313429d914ead P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.003s
I20251211 20:52:23.979418 1168 raft_consensus.cc:359] T aa685d1b051a4c93a2d313429d914ead P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.979715 1168 raft_consensus.cc:385] T aa685d1b051a4c93a2d313429d914ead P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.979856 1168 raft_consensus.cc:740] T aa685d1b051a4c93a2d313429d914ead P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.980141 1168 consensus_queue.cc:260] T aa685d1b051a4c93a2d313429d914ead P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.982342 1204 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:52:23.983439 1204 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:52:23.983911 590 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 29c5a2d130a042e59fcb915f01886316
I20251211 20:52:23.984136 741 tablet_service.cc:1552] Processing DeleteTablet for tablet f37f598a720d4161b13c03425e020fa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.984356 1165 tablet_replica.cc:333] T f37f598a720d4161b13c03425e020fa5 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:23.984546 1165 raft_consensus.cc:2243] T f37f598a720d4161b13c03425e020fa5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:23.984699 1165 raft_consensus.cc:2272] T f37f598a720d4161b13c03425e020fa5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:23.985188 1204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:52:23.986114 1165 ts_tablet_manager.cc:1916] T f37f598a720d4161b13c03425e020fa5 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:23.991118 1164 ts_tablet_manager.cc:1929] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.991256 1164 log.cc:1199] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/dc7e8db4b51c415e86d8c24a76b8cb1e
I20251211 20:52:23.991638 738 tablet_service.cc:1552] Processing DeleteTablet for tablet aa685d1b051a4c93a2d313429d914ead with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.991689 1164 ts_tablet_manager.cc:1950] T dc7e8db4b51c415e86d8c24a76b8cb1e P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:23.991870 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 35a49c9f560c47049bdcbeddd55ccb28 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.992043 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 531a74c1fef34b2dba6d4a732e13ce38 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.992234 741 tablet_service.cc:1552] Processing DeleteTablet for tablet d703a5a42b2f47d0a7ffe1bb534a1b05 with delete_type TABLET_DATA_DELETED (replaced by tablet bd8fd6c977204c90b558827fc0b15f43) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:23.993448 592 master_service.cc:954] ReplaceTablet: received request to replace tablet aa685d1b051a4c93a2d313429d914ead from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:23.994020 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_1e40c473960c4fbebfc86c9680481ec1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:23.994642 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e40c473960c4fbebfc86c9680481ec1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:23.995201 1165 ts_tablet_manager.cc:1929] T f37f598a720d4161b13c03425e020fa5 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:23.995328 1165 log.cc:1199] T f37f598a720d4161b13c03425e020fa5 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/f37f598a720d4161b13c03425e020fa5
I20251211 20:52:23.995728 1165 ts_tablet_manager.cc:1950] T f37f598a720d4161b13c03425e020fa5 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:23.995968 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_36ad93a3bb49488b87ef36eb462443b4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:23.996493 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_36ad93a3bb49488b87ef36eb462443b4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:23.996757 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet f37f598a720d4161b13c03425e020fa5 (table default.Table_9af853b8388442319882d871dc136367 [id=0563c542d899496e8586946ff3a66d96]) successfully deleted
I20251211 20:52:23.997677 1160 tablet_bootstrap.cc:492] T f37f598a720d4161b13c03425e020fa5 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:23.997819 1160 ts_tablet_manager.cc:1403] T f37f598a720d4161b13c03425e020fa5 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.053s user 0.003s sys 0.000s
I20251211 20:52:23.998636 1160 raft_consensus.cc:359] T f37f598a720d4161b13c03425e020fa5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.998833 1160 raft_consensus.cc:385] T f37f598a720d4161b13c03425e020fa5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:23.998925 1160 raft_consensus.cc:740] T f37f598a720d4161b13c03425e020fa5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:23.999141 1160 consensus_queue.cc:260] T f37f598a720d4161b13c03425e020fa5 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:23.999119 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_4cd9b395cfa94302b298a3df4971d631"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:23.999651 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4cd9b395cfa94302b298a3df4971d631 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:23.999714 1168 ts_tablet_manager.cc:1434] T aa685d1b051a4c93a2d313429d914ead P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.021s user 0.001s sys 0.001s
I20251211 20:52:24.002343 1160 ts_tablet_manager.cc:1434] T f37f598a720d4161b13c03425e020fa5 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:24.007149 1168 tablet_bootstrap.cc:492] T 5ed0a0141051462eb45ca3a66d41d009 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:24.007534 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_d73d156809724a399ad27846bfcc460b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:24.008445 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d73d156809724a399ad27846bfcc460b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.008930 1168 tablet_bootstrap.cc:654] T 5ed0a0141051462eb45ca3a66d41d009 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.011411 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 35a49c9f560c47049bdcbeddd55ccb28 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:24.012053 1160 tablet_bootstrap.cc:492] T aa685d1b051a4c93a2d313429d914ead P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:24.012765 1165 tablet_replica.cc:333] T aa685d1b051a4c93a2d313429d914ead P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:24.012982 1165 raft_consensus.cc:2243] T aa685d1b051a4c93a2d313429d914ead P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.013293 1165 raft_consensus.cc:2272] T aa685d1b051a4c93a2d313429d914ead P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.014930 1160 tablet_bootstrap.cc:654] T aa685d1b051a4c93a2d313429d914ead P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.015491 1165 ts_tablet_manager.cc:1916] T aa685d1b051a4c93a2d313429d914ead P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.022570 1168 tablet_bootstrap.cc:492] T 5ed0a0141051462eb45ca3a66d41d009 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:24.022816 1168 ts_tablet_manager.cc:1403] T 5ed0a0141051462eb45ca3a66d41d009 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.000s
I20251211 20:52:24.016125 1164 tablet_replica.cc:333] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:24.017946 880 tablet_service.cc:1552] Processing DeleteTablet for tablet f37f598a720d4161b13c03425e020fa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:24.023713 1168 raft_consensus.cc:359] T 5ed0a0141051462eb45ca3a66d41d009 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:24.023886 1164 raft_consensus.cc:2243] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.024004 1168 raft_consensus.cc:385] T 5ed0a0141051462eb45ca3a66d41d009 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.024065 1164 raft_consensus.cc:2272] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.024163 1168 raft_consensus.cc:740] T 5ed0a0141051462eb45ca3a66d41d009 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.024662 1168 consensus_queue.cc:260] T 5ed0a0141051462eb45ca3a66d41d009 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:24.025883 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 4ddf1bd1e18f4187bf050d6f7767b722
I20251211 20:52:24.026796 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ddf1bd1e18f4187bf050d6f7767b722 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.026726 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_349fa502a7c44fe79baed28f7da89ff9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:24.027376 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_349fa502a7c44fe79baed28f7da89ff9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.027493 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ddf1bd1e18f4187bf050d6f7767b722 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.029078 1164 ts_tablet_manager.cc:1916] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.029975 1168 ts_tablet_manager.cc:1434] T 5ed0a0141051462eb45ca3a66d41d009 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
W20251211 20:52:24.031072 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 4ddf1bd1e18f4187bf050d6f7767b722 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ddf1bd1e18f4187bf050d6f7767b722 already in progress: creating tablet
I20251211 20:52:24.026073 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet dc7e8db4b51c415e86d8c24a76b8cb1e (table default.Table_5ef3d8d2746a4016ad8efac84ba492b5 [id=ebae22b7d617403c84c023d0628f8ad8]) successfully deleted
I20251211 20:52:24.041045 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ddf1bd1e18f4187bf050d6f7767b722 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:24.043012 1160 tablet_bootstrap.cc:492] T aa685d1b051a4c93a2d313429d914ead P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:24.043208 1160 ts_tablet_manager.cc:1403] T aa685d1b051a4c93a2d313429d914ead P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.002s
I20251211 20:52:24.044087 1160 raft_consensus.cc:359] T aa685d1b051a4c93a2d313429d914ead P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.044360 1160 raft_consensus.cc:385] T aa685d1b051a4c93a2d313429d914ead P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.044452 1160 raft_consensus.cc:740] T aa685d1b051a4c93a2d313429d914ead P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.044703 1160 consensus_queue.cc:260] T aa685d1b051a4c93a2d313429d914ead P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.047693 1160 ts_tablet_manager.cc:1434] T aa685d1b051a4c93a2d313429d914ead P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20251211 20:52:24.049052 1168 tablet_bootstrap.cc:492] T edc4300dc5a84173aa95b587d77fc95c P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:24.053710 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet f4db14d082d34aef857e106992bb8950
W20251211 20:52:24.055230 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 27466459d8e6411ab87ac474fb617735 consensus state has no leader
I20251211 20:52:24.055335 1165 ts_tablet_manager.cc:1929] T aa685d1b051a4c93a2d313429d914ead P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.055471 1165 log.cc:1199] T aa685d1b051a4c93a2d313429d914ead P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/aa685d1b051a4c93a2d313429d914ead
I20251211 20:52:24.055521 589 master_service.cc:954] ReplaceTablet: received request to replace tablet edc4300dc5a84173aa95b587d77fc95c from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:24.056344 1160 tablet_bootstrap.cc:492] T b815bca893a8446e92d48973d9a0408e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:24.056432 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet f4db14d082d34aef857e106992bb8950 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.056953 741 tablet_service.cc:1552] Processing DeleteTablet for tablet f4db14d082d34aef857e106992bb8950 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.057739 1163 tablet_replica.cc:333] T f4db14d082d34aef857e106992bb8950 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:24.057951 1163 raft_consensus.cc:2243] T f4db14d082d34aef857e106992bb8950 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.058063 1160 tablet_bootstrap.cc:654] T b815bca893a8446e92d48973d9a0408e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.058156 1163 raft_consensus.cc:2272] T f4db14d082d34aef857e106992bb8950 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.058513 591 master_service.cc:954] ReplaceTablet: received request to replace tablet 5fc8d485cc234fd490c487a8bc881bf4 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:24.058938 1163 ts_tablet_manager.cc:1916] T f4db14d082d34aef857e106992bb8950 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.059635 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 353da9106578450987a4f6544df8d7e6 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:24.060072 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_0209a7c0449b42e9ac582543b779f5c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:24.060658 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0209a7c0449b42e9ac582543b779f5c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.063131 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_ca483456010c49c6a291fbaeb6f864ef"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:24.063843 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ca483456010c49c6a291fbaeb6f864ef in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.064270 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 5ed0a0141051462eb45ca3a66d41d009 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.064855 1163 ts_tablet_manager.cc:1929] T f4db14d082d34aef857e106992bb8950 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.064971 1163 log.cc:1199] T f4db14d082d34aef857e106992bb8950 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/f4db14d082d34aef857e106992bb8950
I20251211 20:52:24.065310 1163 ts_tablet_manager.cc:1950] T f4db14d082d34aef857e106992bb8950 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:24.067548 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet f4db14d082d34aef857e106992bb8950 (table default.Table_9ac73058eb564b63871197be450065f0 [id=ffb336d92b05442b80d70cd1f419431c]) successfully deleted
I20251211 20:52:24.070946 880 tablet_service.cc:1552] Processing DeleteTablet for tablet f4db14d082d34aef857e106992bb8950 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:24.071842 1182 raft_consensus.cc:493] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:24.072000 1182 raft_consensus.cc:515] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.072680 1182 leader_election.cc:290] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:24.073289 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "531a74c1fef34b2dba6d4a732e13ce38" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:24.073527 912 raft_consensus.cc:2468] T 531a74c1fef34b2dba6d4a732e13ce38 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
W20251211 20:52:24.073825 710 leader_election.cc:343] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 531a74c1fef34b2dba6d4a732e13ce38
I20251211 20:52:24.074265 711 leader_election.cc:304] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters: e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:24.074757 1182 raft_consensus.cc:2804] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:24.074867 1182 raft_consensus.cc:493] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:24.074936 1182 raft_consensus.cc:3060] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:24.076503 1168 tablet_bootstrap.cc:654] T edc4300dc5a84173aa95b587d77fc95c P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.077811 1182 raft_consensus.cc:515] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.079222 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "531a74c1fef34b2dba6d4a732e13ce38" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:24.079442 912 raft_consensus.cc:3060] T 531a74c1fef34b2dba6d4a732e13ce38 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:24.080880 1165 ts_tablet_manager.cc:1950] T aa685d1b051a4c93a2d313429d914ead P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:24.081887 1165 tablet_replica.cc:333] T 35a49c9f560c47049bdcbeddd55ccb28 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:24.082063 1165 raft_consensus.cc:2243] T 35a49c9f560c47049bdcbeddd55ccb28 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.082209 1165 raft_consensus.cc:2272] T 35a49c9f560c47049bdcbeddd55ccb28 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.082304 912 raft_consensus.cc:2468] T 531a74c1fef34b2dba6d4a732e13ce38 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:24.083048 711 leader_election.cc:304] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters:
I20251211 20:52:24.083775 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet aa685d1b051a4c93a2d313429d914ead (table default.Table_ce67c2bcef6d4a6baecc11975d852ba8 [id=f81bfe3e109f48ce86e40e77dcbb30e1]) successfully deleted
I20251211 20:52:24.084623 1165 ts_tablet_manager.cc:1916] T 35a49c9f560c47049bdcbeddd55ccb28 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:24.086047 710 leader_election.cc:343] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 531a74c1fef34b2dba6d4a732e13ce38
I20251211 20:52:24.087977 1182 leader_election.cc:290] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:24.088521 1182 raft_consensus.cc:2804] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:24.089210 1182 raft_consensus.cc:697] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:24.089591 1160 tablet_bootstrap.cc:492] T b815bca893a8446e92d48973d9a0408e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:24.089747 1160 ts_tablet_manager.cc:1403] T b815bca893a8446e92d48973d9a0408e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.042s user 0.003s sys 0.000s
I20251211 20:52:24.090579 1160 raft_consensus.cc:359] T b815bca893a8446e92d48973d9a0408e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.090778 1160 raft_consensus.cc:385] T b815bca893a8446e92d48973d9a0408e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.090862 1160 raft_consensus.cc:740] T b815bca893a8446e92d48973d9a0408e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.091343 1165 ts_tablet_manager.cc:1929] T 35a49c9f560c47049bdcbeddd55ccb28 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.091491 1165 log.cc:1199] T 35a49c9f560c47049bdcbeddd55ccb28 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/35a49c9f560c47049bdcbeddd55ccb28
I20251211 20:52:24.090301 1182 consensus_queue.cc:237] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.091900 1165 ts_tablet_manager.cc:1950] T 35a49c9f560c47049bdcbeddd55ccb28 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:24.093389 1165 tablet_replica.cc:333] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:24.095031 1160 consensus_queue.cc:260] T b815bca893a8446e92d48973d9a0408e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.098107 1308 raft_consensus.cc:493] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:24.098297 1308 raft_consensus.cc:515] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.098906 1160 ts_tablet_manager.cc:1434] T b815bca893a8446e92d48973d9a0408e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20251211 20:52:24.099256 1160 tablet_bootstrap.cc:492] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:24.099941 710 leader_election.cc:343] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: d703a5a42b2f47d0a7ffe1bb534a1b05
I20251211 20:52:24.100371 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 35a49c9f560c47049bdcbeddd55ccb28 (table default.Table_0c955f27bf344b1cbe27484262636529 [id=16a2a090abb14c1a996e695ea2798579]) successfully deleted
I20251211 20:52:24.102382 1308 leader_election.cc:290] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:24.104699 880 tablet_service.cc:1552] Processing DeleteTablet for tablet aa685d1b051a4c93a2d313429d914ead with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:24.105012 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d703a5a42b2f47d0a7ffe1bb534a1b05" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:24.106273 1160 tablet_bootstrap.cc:654] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:24.108117 711 leader_election.cc:343] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:24.109131 711 leader_election.cc:304] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:24.109911 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9c03b1369d12432fb6789a0bac513af0" } new_table_name: "default.Table_afd3c9ee35ad41caa1717a299443409e" modify_external_catalogs: true
I20251211 20:52:24.113267 1308 raft_consensus.cc:2749] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:24.113790 1160 tablet_bootstrap.cc:492] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:24.116894 1166 tablet_bootstrap.cc:492] T edc4300dc5a84173aa95b587d77fc95c P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:24.117081 1166 ts_tablet_manager.cc:1403] T edc4300dc5a84173aa95b587d77fc95c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.292s user 0.094s sys 0.140s
I20251211 20:52:24.117914 1166 raft_consensus.cc:359] T edc4300dc5a84173aa95b587d77fc95c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.118116 1166 raft_consensus.cc:385] T edc4300dc5a84173aa95b587d77fc95c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.118189 1166 raft_consensus.cc:740] T edc4300dc5a84173aa95b587d77fc95c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.118484 1166 consensus_queue.cc:260] T edc4300dc5a84173aa95b587d77fc95c P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.121951 1166 ts_tablet_manager.cc:1434] T edc4300dc5a84173aa95b587d77fc95c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251211 20:52:24.123917 1166 tablet_bootstrap.cc:492] T 297dc26ade384213b364a876c54bdd14 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:24.125674 1166 tablet_bootstrap.cc:654] T 297dc26ade384213b364a876c54bdd14 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.127365 880 tablet_service.cc:1552] Processing DeleteTablet for tablet b815bca893a8446e92d48973d9a0408e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:24.127924 1168 tablet_bootstrap.cc:492] T edc4300dc5a84173aa95b587d77fc95c P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:24.128089 1168 ts_tablet_manager.cc:1403] T edc4300dc5a84173aa95b587d77fc95c P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.079s user 0.007s sys 0.002s
I20251211 20:52:24.128881 1168 raft_consensus.cc:359] T edc4300dc5a84173aa95b587d77fc95c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.129062 1168 raft_consensus.cc:385] T edc4300dc5a84173aa95b587d77fc95c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.129143 1168 raft_consensus.cc:740] T edc4300dc5a84173aa95b587d77fc95c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.129355 1168 consensus_queue.cc:260] T edc4300dc5a84173aa95b587d77fc95c P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.130069 1168 ts_tablet_manager.cc:1434] T edc4300dc5a84173aa95b587d77fc95c P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251211 20:52:24.131245 1168 tablet_bootstrap.cc:492] T b815bca893a8446e92d48973d9a0408e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:24.134778 1166 tablet_bootstrap.cc:492] T 297dc26ade384213b364a876c54bdd14 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:24.134954 1166 ts_tablet_manager.cc:1403] T 297dc26ade384213b364a876c54bdd14 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.002s
I20251211 20:52:24.135811 1166 raft_consensus.cc:359] T 297dc26ade384213b364a876c54bdd14 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.136003 1166 raft_consensus.cc:385] T 297dc26ade384213b364a876c54bdd14 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.136104 1166 raft_consensus.cc:740] T 297dc26ade384213b364a876c54bdd14 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.143146 1160 ts_tablet_manager.cc:1403] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.044s user 0.003s sys 0.003s
I20251211 20:52:24.144268 1168 tablet_bootstrap.cc:654] T b815bca893a8446e92d48973d9a0408e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.144377 1165 raft_consensus.cc:2243] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:24.144613 1166 consensus_queue.cc:260] T 297dc26ade384213b364a876c54bdd14 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.145187 1166 ts_tablet_manager.cc:1434] T 297dc26ade384213b364a876c54bdd14 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20251211 20:52:24.145766 1160 raft_consensus.cc:359] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.145965 1160 raft_consensus.cc:385] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.146039 1160 raft_consensus.cc:740] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.146274 1160 consensus_queue.cc:260] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.146780 1160 ts_tablet_manager.cc:1434] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251211 20:52:24.147101 1160 tablet_bootstrap.cc:492] T d703a5a42b2f47d0a7ffe1bb534a1b05 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:24.149309 1160 tablet_bootstrap.cc:654] T d703a5a42b2f47d0a7ffe1bb534a1b05 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.149749 1165 pending_rounds.cc:70] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919: Trying to abort 1 pending ops.
I20251211 20:52:24.149947 1165 pending_rounds.cc:77] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432065410838528 op_type: NO_OP noop_request { }
I20251211 20:52:24.150121 1165 raft_consensus.cc:2889] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:24.150236 1165 raft_consensus.cc:2272] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.153828 1166 tablet_bootstrap.cc:492] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:24.158089 1165 ts_tablet_manager.cc:1916] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.164758 1165 ts_tablet_manager.cc:1929] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:24.164919 1165 log.cc:1199] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/531a74c1fef34b2dba6d4a732e13ce38
I20251211 20:52:24.165277 1165 ts_tablet_manager.cc:1950] T 531a74c1fef34b2dba6d4a732e13ce38 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:24.166442 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 531a74c1fef34b2dba6d4a732e13ce38 (table default.Table_9f357fec3afd4c58b2930fea17141463 [id=4f44988d148e427589d8ee1b342f0590]) successfully deleted
I20251211 20:52:24.171132 1165 tablet_replica.cc:333] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:24.171353 1165 raft_consensus.cc:2243] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.171604 1165 raft_consensus.cc:2272] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.172468 1165 ts_tablet_manager.cc:1916] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.179355 1166 tablet_bootstrap.cc:654] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.184844 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a0c567f392784be39ceb0176cf58cff8" } new_table_name: "default.Table_f572100667d148e8a69e7930596f0f52" modify_external_catalogs: true
I20251211 20:52:24.185120 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet edc4300dc5a84173aa95b587d77fc95c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.185190 1165 ts_tablet_manager.cc:1929] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.185324 1165 log.cc:1199] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/d703a5a42b2f47d0a7ffe1bb534a1b05
I20251211 20:52:24.185397 1163 tablet_replica.cc:333] T edc4300dc5a84173aa95b587d77fc95c P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:24.185593 1163 raft_consensus.cc:2243] T edc4300dc5a84173aa95b587d77fc95c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.185680 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_6c528dbf0fb149b3894f6afb543ae25b" } new_table_name: "default.Table_a81b3b9368cc4a118087cd5131a50534" modify_external_catalogs: true
I20251211 20:52:24.185777 1163 raft_consensus.cc:2272] T edc4300dc5a84173aa95b587d77fc95c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.185976 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 531a74c1fef34b2dba6d4a732e13ce38 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.186406 1163 ts_tablet_manager.cc:1916] T edc4300dc5a84173aa95b587d77fc95c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.186445 1165 ts_tablet_manager.cc:1950] T d703a5a42b2f47d0a7ffe1bb534a1b05 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:24.187039 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f53b7da399c043038b4c6d59e93bc943" } new_table_name: "default.Table_3570af5d8bff402ab3140ae51bae174c" modify_external_catalogs: true
I20251211 20:52:24.188418 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_88975c1ad9604f50b7f69fbbdacabce3" } modify_external_catalogs: true
I20251211 20:52:24.188578 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_88975c1ad9604f50b7f69fbbdacabce3" } modify_external_catalogs: true
I20251211 20:52:24.189414 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_736d6c2eb4dd4eb9a72dca6b9ed2073f" } modify_external_catalogs: true
I20251211 20:52:24.189606 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_736d6c2eb4dd4eb9a72dca6b9ed2073f" } modify_external_catalogs: true
I20251211 20:52:24.191370 1168 tablet_bootstrap.cc:492] T b815bca893a8446e92d48973d9a0408e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:24.191645 1168 ts_tablet_manager.cc:1403] T b815bca893a8446e92d48973d9a0408e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.061s user 0.004s sys 0.000s
I20251211 20:52:24.192520 1168 raft_consensus.cc:359] T b815bca893a8446e92d48973d9a0408e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.192796 1168 raft_consensus.cc:385] T b815bca893a8446e92d48973d9a0408e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.192914 1168 raft_consensus.cc:740] T b815bca893a8446e92d48973d9a0408e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.193542 1163 ts_tablet_manager.cc:1929] T edc4300dc5a84173aa95b587d77fc95c P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.193706 1163 log.cc:1199] T edc4300dc5a84173aa95b587d77fc95c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/edc4300dc5a84173aa95b587d77fc95c
I20251211 20:52:24.194316 1160 tablet_bootstrap.cc:492] T d703a5a42b2f47d0a7ffe1bb534a1b05 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:24.194314 1163 ts_tablet_manager.cc:1950] T edc4300dc5a84173aa95b587d77fc95c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:24.194481 1168 consensus_queue.cc:260] T b815bca893a8446e92d48973d9a0408e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.195052 1168 ts_tablet_manager.cc:1434] T b815bca893a8446e92d48973d9a0408e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:24.195266 1160 ts_tablet_manager.cc:1403] T d703a5a42b2f47d0a7ffe1bb534a1b05 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.048s user 0.004s sys 0.000s
I20251211 20:52:24.196161 1160 raft_consensus.cc:359] T d703a5a42b2f47d0a7ffe1bb534a1b05 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.196660 1168 tablet_bootstrap.cc:492] T 01ada50100bc48e39d1bc67082da750e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:24.197698 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet edc4300dc5a84173aa95b587d77fc95c (table default.Table_4f9b0d5d6ec647df87f89a25344ca9f8 [id=32fff752ffaa46e6af5221a3ecf2adc9]) successfully deleted
I20251211 20:52:24.199081 1168 tablet_bootstrap.cc:654] T 01ada50100bc48e39d1bc67082da750e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.199700 1160 raft_consensus.cc:385] T d703a5a42b2f47d0a7ffe1bb534a1b05 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.199820 1160 raft_consensus.cc:740] T d703a5a42b2f47d0a7ffe1bb534a1b05 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.200067 1160 consensus_queue.cc:260] T d703a5a42b2f47d0a7ffe1bb534a1b05 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.201905 1160 ts_tablet_manager.cc:1434] T d703a5a42b2f47d0a7ffe1bb534a1b05 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20251211 20:52:24.202308 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet d703a5a42b2f47d0a7ffe1bb534a1b05 (table default.Table_5b8aa62e6fdf46089c4c8ae541856fbe [id=e5caa22bce84422290707816a93bd3f5]) successfully deleted
I20251211 20:52:24.202888 1160 tablet_bootstrap.cc:492] T bc51bc0821a8484caee39c534eb4e6af P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:24.203161 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 4ddf1bd1e18f4187bf050d6f7767b722 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ddf1bd1e18f4187bf050d6f7767b722 already in progress: creating tablet
W20251211 20:52:24.203994 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet f4db14d082d34aef857e106992bb8950 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f4db14d082d34aef857e106992bb8950 already in progress: creating tablet
I20251211 20:52:24.204355 1165 tablet_replica.cc:333] T 5ed0a0141051462eb45ca3a66d41d009 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:24.204612 1165 raft_consensus.cc:2243] T 5ed0a0141051462eb45ca3a66d41d009 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.204839 1165 raft_consensus.cc:2272] T 5ed0a0141051462eb45ca3a66d41d009 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.208547 1165 ts_tablet_manager.cc:1916] T 5ed0a0141051462eb45ca3a66d41d009 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.211012 741 tablet_service.cc:1552] Processing DeleteTablet for tablet edc4300dc5a84173aa95b587d77fc95c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.216192 1166 tablet_bootstrap.cc:492] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:24.216388 1166 ts_tablet_manager.cc:1403] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.063s user 0.003s sys 0.000s
I20251211 20:52:24.220000 1165 ts_tablet_manager.cc:1929] T 5ed0a0141051462eb45ca3a66d41d009 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.226118 1165 log.cc:1199] T 5ed0a0141051462eb45ca3a66d41d009 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/5ed0a0141051462eb45ca3a66d41d009
I20251211 20:52:24.226684 1165 ts_tablet_manager.cc:1950] T 5ed0a0141051462eb45ca3a66d41d009 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:24.227944 1166 raft_consensus.cc:359] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.228180 1166 raft_consensus.cc:385] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.228248 1166 raft_consensus.cc:740] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.228291 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 5ed0a0141051462eb45ca3a66d41d009 (table default.Table_40256d3973c74566a09b1d235062d284 [id=1f4650c549d94e2c9ef753ff96670aa6]) successfully deleted
I20251211 20:52:24.228781 1164 ts_tablet_manager.cc:1929] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.228945 1164 log.cc:1199] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/62fb046960024a2b9d70dc46ae52d53f
I20251211 20:52:24.229295 1164 ts_tablet_manager.cc:1950] T 62fb046960024a2b9d70dc46ae52d53f P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:24.229422 1168 tablet_bootstrap.cc:492] T 01ada50100bc48e39d1bc67082da750e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:24.229614 1168 ts_tablet_manager.cc:1403] T 01ada50100bc48e39d1bc67082da750e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.033s user 0.013s sys 0.017s
W20251211 20:52:24.230196 581 catalog_manager.cc:4952] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 531a74c1fef34b2dba6d4a732e13ce38 because the tablet was not found. No further retry: Not found: Tablet not found: 531a74c1fef34b2dba6d4a732e13ce38
I20251211 20:52:24.230486 1166 consensus_queue.cc:260] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.230901 1165 tablet_replica.cc:333] T edc4300dc5a84173aa95b587d77fc95c P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:24.231863 1165 raft_consensus.cc:2243] T edc4300dc5a84173aa95b587d77fc95c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.232074 1165 raft_consensus.cc:2272] T edc4300dc5a84173aa95b587d77fc95c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.231575 1168 raft_consensus.cc:359] T 01ada50100bc48e39d1bc67082da750e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.232339 1168 raft_consensus.cc:385] T 01ada50100bc48e39d1bc67082da750e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.232479 1168 raft_consensus.cc:740] T 01ada50100bc48e39d1bc67082da750e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.232761 1168 consensus_queue.cc:260] T 01ada50100bc48e39d1bc67082da750e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.233273 1168 ts_tablet_manager.cc:1434] T 01ada50100bc48e39d1bc67082da750e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251211 20:52:24.234310 1165 ts_tablet_manager.cc:1916] T edc4300dc5a84173aa95b587d77fc95c P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.236621 1168 tablet_bootstrap.cc:492] T bc51bc0821a8484caee39c534eb4e6af P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:24.236670 1166 ts_tablet_manager.cc:1434] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.020s user 0.003s sys 0.000s
I20251211 20:52:24.241079 1160 tablet_bootstrap.cc:654] T bc51bc0821a8484caee39c534eb4e6af P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.241288 1164 tablet_replica.cc:333] T 531a74c1fef34b2dba6d4a732e13ce38 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:24.241521 1164 raft_consensus.cc:2243] T 531a74c1fef34b2dba6d4a732e13ce38 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.241719 1164 raft_consensus.cc:2272] T 531a74c1fef34b2dba6d4a732e13ce38 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.242244 1164 ts_tablet_manager.cc:1916] T 531a74c1fef34b2dba6d4a732e13ce38 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.244647 1166 tablet_bootstrap.cc:492] T 694518e6c1ae4032b06b99e9bde2e5dd P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:24.246483 1166 tablet_bootstrap.cc:654] T 694518e6c1ae4032b06b99e9bde2e5dd P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.247184 1168 tablet_bootstrap.cc:654] T bc51bc0821a8484caee39c534eb4e6af P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:24.255921 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c4b76a7e84d344268567c551236ce772 consensus state has no leader
I20251211 20:52:24.257709 597 master_service.cc:954] ReplaceTablet: received request to replace tablet 8a064789f89c424ca698e52333d73eea from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:24.258459 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4c974eff9ab442d499f0405abc8a4acd" } modify_external_catalogs: true
I20251211 20:52:24.258656 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4c974eff9ab442d499f0405abc8a4acd" } modify_external_catalogs: true
I20251211 20:52:24.258920 1165 ts_tablet_manager.cc:1929] T edc4300dc5a84173aa95b587d77fc95c P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.259047 1165 log.cc:1199] T edc4300dc5a84173aa95b587d77fc95c P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/edc4300dc5a84173aa95b587d77fc95c
I20251211 20:52:24.259469 1165 ts_tablet_manager.cc:1950] T edc4300dc5a84173aa95b587d77fc95c P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
W20251211 20:52:24.259945 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dc5de12701714fcd81b9dba6973a28b5 consensus state has no leader
I20251211 20:52:24.260342 589 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 01ada50100bc48e39d1bc67082da750e (table default.Table_18d19a317a9c4656b4c764df4a64dbea [id=a24d8c58b9d84cdea2b278ae9c6e0ba5]). Expected version 2 got 0
I20251211 20:52:24.260661 592 master_service.cc:954] ReplaceTablet: received request to replace tablet 27466459d8e6411ab87ac474fb617735 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:24.262099 1164 ts_tablet_manager.cc:1929] T 531a74c1fef34b2dba6d4a732e13ce38 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.262219 1164 log.cc:1199] T 531a74c1fef34b2dba6d4a732e13ce38 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/531a74c1fef34b2dba6d4a732e13ce38
I20251211 20:52:24.262562 1164 ts_tablet_manager.cc:1950] T 531a74c1fef34b2dba6d4a732e13ce38 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:24.264156 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 2630ad57845a4c31b20c8a104dd27335 (DEFAULT_TABLE table=default.Table_6c528dbf0fb149b3894f6afb543ae25b [id=61b2de16fee74487823292913de1b685]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.264564 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2630ad57845a4c31b20c8a104dd27335. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.264932 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 297dc26ade384213b364a876c54bdd14 with delete_type TABLET_DATA_DELETED (replaced by tablet 8dff13e22899452c8d4f2731b8dac7d5) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.266057 564 master_runner.cc:186] Error getting master registration for 127.0.104.62:41123: Network error: Client connection negotiation failed: client connection to 127.0.104.62:41123: connect: Connection refused (error 111),
I20251211 20:52:24.266218 564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20251211 20:52:24.266877 741 tablet_service.cc:1505] Processing CreateTablet for tablet 2630ad57845a4c31b20c8a104dd27335 (DEFAULT_TABLE table=default.Table_6c528dbf0fb149b3894f6afb543ae25b [id=61b2de16fee74487823292913de1b685]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.267288 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2630ad57845a4c31b20c8a104dd27335. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.267863 744 tablet_service.cc:1505] Processing CreateTablet for tablet 9adaee28edd74df68b44b25d505fdfcb (DEFAULT_TABLE table=default.Table_4c974eff9ab442d499f0405abc8a4acd [id=35caba098ab545f7859c004af8568ef7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.267856 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 9adaee28edd74df68b44b25d505fdfcb (DEFAULT_TABLE table=default.Table_4c974eff9ab442d499f0405abc8a4acd [id=35caba098ab545f7859c004af8568ef7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.268052 1164 tablet_replica.cc:333] T 35a49c9f560c47049bdcbeddd55ccb28 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:24.268198 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9adaee28edd74df68b44b25d505fdfcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.268249 1164 raft_consensus.cc:2243] T 35a49c9f560c47049bdcbeddd55ccb28 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.268414 1164 raft_consensus.cc:2272] T 35a49c9f560c47049bdcbeddd55ccb28 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.268365 750 tablet_service.cc:1505] Processing CreateTablet for tablet 8b89ae724b7f41c98741a41d75592da7 (DEFAULT_TABLE table=default.Table_6bf819d1a1794e62b4cf65ffe5d2c2b5 [id=9dae0e60f978448a840aa497da35966a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.268846 738 tablet_service.cc:1505] Processing CreateTablet for tablet c910246b3d5d4e7e8a71d9f18e94d24c (DEFAULT_TABLE table=default.Table_c2300bb8bfe44c689969a2b233d69353 [id=996745d9420c4589a52eca6bad10d6a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.269220 739 tablet_service.cc:1552] Processing DeleteTablet for tablet b815bca893a8446e92d48973d9a0408e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.269650 1165 tablet_replica.cc:333] T b815bca893a8446e92d48973d9a0408e P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:24.269847 1165 raft_consensus.cc:2243] T b815bca893a8446e92d48973d9a0408e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.270016 1165 raft_consensus.cc:2272] T b815bca893a8446e92d48973d9a0408e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.270951 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 8b89ae724b7f41c98741a41d75592da7 (DEFAULT_TABLE table=default.Table_6bf819d1a1794e62b4cf65ffe5d2c2b5 [id=9dae0e60f978448a840aa497da35966a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.271191 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet edc4300dc5a84173aa95b587d77fc95c (table default.Table_4f9b0d5d6ec647df87f89a25344ca9f8 [id=32fff752ffaa46e6af5221a3ecf2adc9]) successfully deleted
I20251211 20:52:24.271309 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b89ae724b7f41c98741a41d75592da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.273485 590 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 01ada50100bc48e39d1bc67082da750e (table default.Table_18d19a317a9c4656b4c764df4a64dbea [id=a24d8c58b9d84cdea2b278ae9c6e0ba5]). Expected version 2 got 0
I20251211 20:52:24.275763 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9adaee28edd74df68b44b25d505fdfcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.275909 1020 tablet_service.cc:1505] Processing CreateTablet for tablet c910246b3d5d4e7e8a71d9f18e94d24c (DEFAULT_TABLE table=default.Table_c2300bb8bfe44c689969a2b233d69353 [id=996745d9420c4589a52eca6bad10d6a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.276286 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c910246b3d5d4e7e8a71d9f18e94d24c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.279727 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b89ae724b7f41c98741a41d75592da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.279747 1019 tablet_service.cc:1552] Processing DeleteTablet for tablet edc4300dc5a84173aa95b587d77fc95c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.280005 1163 tablet_replica.cc:333] T 297dc26ade384213b364a876c54bdd14 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:24.280177 1163 raft_consensus.cc:2243] T 297dc26ade384213b364a876c54bdd14 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.280366 1163 raft_consensus.cc:2272] T 297dc26ade384213b364a876c54bdd14 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.282080 1164 ts_tablet_manager.cc:1916] T 35a49c9f560c47049bdcbeddd55ccb28 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.283789 738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c910246b3d5d4e7e8a71d9f18e94d24c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.284583 1163 ts_tablet_manager.cc:1916] T 297dc26ade384213b364a876c54bdd14 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.289196 1164 ts_tablet_manager.cc:1929] T 35a49c9f560c47049bdcbeddd55ccb28 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.289355 1164 log.cc:1199] T 35a49c9f560c47049bdcbeddd55ccb28 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/35a49c9f560c47049bdcbeddd55ccb28
I20251211 20:52:24.289727 1164 ts_tablet_manager.cc:1950] T 35a49c9f560c47049bdcbeddd55ccb28 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:24.291836 1164 tablet_replica.cc:333] T f37f598a720d4161b13c03425e020fa5 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:24.292066 1164 raft_consensus.cc:2243] T f37f598a720d4161b13c03425e020fa5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.292254 1164 raft_consensus.cc:2272] T f37f598a720d4161b13c03425e020fa5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.293033 1164 ts_tablet_manager.cc:1916] T f37f598a720d4161b13c03425e020fa5 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.296376 752 tablet_service.cc:1505] Processing CreateTablet for tablet 3eac21a07ba34e1fa275e25ee9dcabb1 (DEFAULT_TABLE table=default.Table_86e5b1e5b4214b48a21f398642d97640 [id=9700f9fe6e3044748e34954bef8a0038]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.296854 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3eac21a07ba34e1fa275e25ee9dcabb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.299226 1163 ts_tablet_manager.cc:1929] T 297dc26ade384213b364a876c54bdd14 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.299381 1163 log.cc:1199] T 297dc26ade384213b364a876c54bdd14 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/297dc26ade384213b364a876c54bdd14
I20251211 20:52:24.299782 1163 ts_tablet_manager.cc:1950] T 297dc26ade384213b364a876c54bdd14 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:24.299813 1164 ts_tablet_manager.cc:1929] T f37f598a720d4161b13c03425e020fa5 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.299944 1164 log.cc:1199] T f37f598a720d4161b13c03425e020fa5 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/f37f598a720d4161b13c03425e020fa5
I20251211 20:52:24.299926 749 tablet_service.cc:1505] Processing CreateTablet for tablet c4b76a7e84d344268567c551236ce772 (DEFAULT_TABLE table=default.Table_afd3c9ee35ad41caa1717a299443409e [id=b0ebaab8cd5a4deb9a5ea1e7bd3ccec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.300307 1164 ts_tablet_manager.cc:1950] T f37f598a720d4161b13c03425e020fa5 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:24.300351 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4b76a7e84d344268567c551236ce772. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.301913 1164 tablet_replica.cc:333] T aa685d1b051a4c93a2d313429d914ead P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:24.302131 1164 raft_consensus.cc:2243] T aa685d1b051a4c93a2d313429d914ead P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.302289 1164 raft_consensus.cc:2272] T aa685d1b051a4c93a2d313429d914ead P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.304212 739 tablet_service.cc:1505] Processing CreateTablet for tablet 19d0aa259ac94f36b7f6a24d11a3d420 (DEFAULT_TABLE table=default.Table_ef1c4ca915474d569c91296213346105 [id=9dd03042ff0948b99b39b899255848fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.304656 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19d0aa259ac94f36b7f6a24d11a3d420. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.308537 1023 tablet_service.cc:1505] Processing CreateTablet for tablet ee893e33255745debf73c486d8c70267 (DEFAULT_TABLE table=default.Table_897d3e48834a42f59e057317699b566d [id=3940282e8e794b21b982be1f171b677d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.308956 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee893e33255745debf73c486d8c70267. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.308897 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 297dc26ade384213b364a876c54bdd14 (table default.Table_f572100667d148e8a69e7930596f0f52 [id=cb97ccf262fd4c41a180267fd0678cec]) successfully deleted
W20251211 20:52:24.309158 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet edc4300dc5a84173aa95b587d77fc95c because the tablet was not found. No further retry: Not found: Tablet not found: edc4300dc5a84173aa95b587d77fc95c
I20251211 20:52:24.311887 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 19d0aa259ac94f36b7f6a24d11a3d420 (DEFAULT_TABLE table=default.Table_ef1c4ca915474d569c91296213346105 [id=9dd03042ff0948b99b39b899255848fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.312798 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19d0aa259ac94f36b7f6a24d11a3d420. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.312958 1164 ts_tablet_manager.cc:1916] T aa685d1b051a4c93a2d313429d914ead P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.315933 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 3eac21a07ba34e1fa275e25ee9dcabb1 (DEFAULT_TABLE table=default.Table_86e5b1e5b4214b48a21f398642d97640 [id=9700f9fe6e3044748e34954bef8a0038]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.316311 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3eac21a07ba34e1fa275e25ee9dcabb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.319649 1164 ts_tablet_manager.cc:1929] T aa685d1b051a4c93a2d313429d914ead P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.319809 1164 log.cc:1199] T aa685d1b051a4c93a2d313429d914ead P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/aa685d1b051a4c93a2d313429d914ead
I20251211 20:52:24.319954 1027 tablet_service.cc:1505] Processing CreateTablet for tablet c4b76a7e84d344268567c551236ce772 (DEFAULT_TABLE table=default.Table_afd3c9ee35ad41caa1717a299443409e [id=b0ebaab8cd5a4deb9a5ea1e7bd3ccec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.320174 1164 ts_tablet_manager.cc:1950] T aa685d1b051a4c93a2d313429d914ead P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:24.320299 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4b76a7e84d344268567c551236ce772. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.321333 1164 tablet_replica.cc:333] T b815bca893a8446e92d48973d9a0408e P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:24.321524 1164 raft_consensus.cc:2243] T b815bca893a8446e92d48973d9a0408e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.321699 1164 raft_consensus.cc:2272] T b815bca893a8446e92d48973d9a0408e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.322330 1164 ts_tablet_manager.cc:1916] T b815bca893a8446e92d48973d9a0408e P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.327981 750 tablet_service.cc:1505] Processing CreateTablet for tablet ee893e33255745debf73c486d8c70267 (DEFAULT_TABLE table=default.Table_897d3e48834a42f59e057317699b566d [id=3940282e8e794b21b982be1f171b677d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.328510 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee893e33255745debf73c486d8c70267. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.329480 1164 ts_tablet_manager.cc:1929] T b815bca893a8446e92d48973d9a0408e P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.329658 1164 log.cc:1199] T b815bca893a8446e92d48973d9a0408e P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/b815bca893a8446e92d48973d9a0408e
I20251211 20:52:24.330091 1164 ts_tablet_manager.cc:1950] T b815bca893a8446e92d48973d9a0408e P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
W20251211 20:52:24.334842 581 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet dc5de12701714fcd81b9dba6973a28b5 (table default.Table_f572100667d148e8a69e7930596f0f52 [id=cb97ccf262fd4c41a180267fd0678cec], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet dc5de12701714fcd81b9dba6973a28b5 consensus state has no leader
I20251211 20:52:24.337014 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 27a08a64d92342d08ab4406e8006a97b (DEFAULT_TABLE table=default.Table_19db6b94abda4124a0e45f11b8ca0990 [id=383392d835da4c50848068d83261cecc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.337628 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27a08a64d92342d08ab4406e8006a97b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.340201 739 tablet_service.cc:1505] Processing CreateTablet for tablet 27a08a64d92342d08ab4406e8006a97b (DEFAULT_TABLE table=default.Table_19db6b94abda4124a0e45f11b8ca0990 [id=383392d835da4c50848068d83261cecc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.340487 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 49a33084c0fc460eb9592224b6dfc279 (DEFAULT_TABLE table=default.Table_b06a3971c3be404e9787e05466931423 [id=9c24845e4efe475fa792a51ce8a66be6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.341176 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27a08a64d92342d08ab4406e8006a97b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.341543 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49a33084c0fc460eb9592224b6dfc279. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.340996 1020 tablet_service.cc:1505] Processing CreateTablet for tablet ae1abef61473443b9922eb13f2ae92fa (DEFAULT_TABLE table=default.Table_40d581fe03ef48d1b5639a95ffc00b7b [id=66810eaf071c4613a4d30dfa016788a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.342468 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 88066d94d5d34e0fab20da9fb6ef178e (DEFAULT_TABLE table=default.Table_b06f9587f78645bb9712c3b080f3c0d4 [id=ea12bdaa894f446da22635f95d97ac2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.342931 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88066d94d5d34e0fab20da9fb6ef178e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.343133 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 5fd2a6cf80a04f20aef8ff9c84bc88db (DEFAULT_TABLE table=default.Table_09578b67c6ea4eb180bc57baafec6d07 [id=10fd0368198c447c95b692b1f3e6a023]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.343488 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fd2a6cf80a04f20aef8ff9c84bc88db. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.341547 1029 tablet_service.cc:1505] Processing CreateTablet for tablet dc5de12701714fcd81b9dba6973a28b5 (DEFAULT_TABLE table=default.Table_f572100667d148e8a69e7930596f0f52 [id=cb97ccf262fd4c41a180267fd0678cec]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:24.346829 582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.61:40749, user_credentials={real_user=slave}} blocked reactor thread for 105967us
I20251211 20:52:24.347100 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 62fb046960024a2b9d70dc46ae52d53f (table default.Table_02014bfadb7b434f960168f2db70b838 [id=14a0dbb8486246b09f3e06fc2120cd28]) successfully deleted
I20251211 20:52:24.347316 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 531a74c1fef34b2dba6d4a732e13ce38 (table default.Table_9f357fec3afd4c58b2930fea17141463 [id=4f44988d148e427589d8ee1b342f0590]) successfully deleted
W20251211 20:52:24.347548 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 1c04187b8ed146c289a6b17d62b07199 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c04187b8ed146c289a6b17d62b07199 already in progress: creating tablet
W20251211 20:52:24.347795 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet bc51bc0821a8484caee39c534eb4e6af because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc51bc0821a8484caee39c534eb4e6af already in progress: creating tablet
W20251211 20:52:24.347998 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet b8828385cec34d70b717ae7f3813b9c3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b8828385cec34d70b717ae7f3813b9c3 already in progress: creating tablet
I20251211 20:52:24.348166 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 35a49c9f560c47049bdcbeddd55ccb28 (table default.Table_0c955f27bf344b1cbe27484262636529 [id=16a2a090abb14c1a996e695ea2798579]) successfully deleted
I20251211 20:52:24.348338 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet f37f598a720d4161b13c03425e020fa5 (table default.Table_9af853b8388442319882d871dc136367 [id=0563c542d899496e8586946ff3a66d96]) successfully deleted
W20251211 20:52:24.348529 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 4ddf1bd1e18f4187bf050d6f7767b722 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ddf1bd1e18f4187bf050d6f7767b722 already in progress: creating tablet
W20251211 20:52:24.348717 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet f4db14d082d34aef857e106992bb8950 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f4db14d082d34aef857e106992bb8950 already in progress: creating tablet
I20251211 20:52:24.348870 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet aa685d1b051a4c93a2d313429d914ead (table default.Table_ce67c2bcef6d4a6baecc11975d852ba8 [id=f81bfe3e109f48ce86e40e77dcbb30e1]) successfully deleted
I20251211 20:52:24.349033 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet b815bca893a8446e92d48973d9a0408e (table default.Table_b411ab6a2d934ad8aa183b348dc24d85 [id=fd269c697f9d4c5eb4a0d646f459c954]) successfully deleted
I20251211 20:52:24.349892 880 tablet_service.cc:1505] Processing CreateTablet for tablet 2630ad57845a4c31b20c8a104dd27335 (DEFAULT_TABLE table=default.Table_6c528dbf0fb149b3894f6afb543ae25b [id=61b2de16fee74487823292913de1b685]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.350317 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2630ad57845a4c31b20c8a104dd27335. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.345053 738 tablet_service.cc:1505] Processing CreateTablet for tablet 5fd2a6cf80a04f20aef8ff9c84bc88db (DEFAULT_TABLE table=default.Table_09578b67c6ea4eb180bc57baafec6d07 [id=10fd0368198c447c95b692b1f3e6a023]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.351938 738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fd2a6cf80a04f20aef8ff9c84bc88db. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.352970 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae1abef61473443b9922eb13f2ae92fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.345415 752 tablet_service.cc:1505] Processing CreateTablet for tablet ae1abef61473443b9922eb13f2ae92fa (DEFAULT_TABLE table=default.Table_40d581fe03ef48d1b5639a95ffc00b7b [id=66810eaf071c4613a4d30dfa016788a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.355918 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae1abef61473443b9922eb13f2ae92fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.345738 744 tablet_service.cc:1505] Processing CreateTablet for tablet 88066d94d5d34e0fab20da9fb6ef178e (DEFAULT_TABLE table=default.Table_b06f9587f78645bb9712c3b080f3c0d4 [id=ea12bdaa894f446da22635f95d97ac2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.356643 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88066d94d5d34e0fab20da9fb6ef178e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.357036 880 tablet_service.cc:1505] Processing CreateTablet for tablet 9adaee28edd74df68b44b25d505fdfcb (DEFAULT_TABLE table=default.Table_4c974eff9ab442d499f0405abc8a4acd [id=35caba098ab545f7859c004af8568ef7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.357470 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9adaee28edd74df68b44b25d505fdfcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.346069 741 tablet_service.cc:1505] Processing CreateTablet for tablet dc5de12701714fcd81b9dba6973a28b5 (DEFAULT_TABLE table=default.Table_f572100667d148e8a69e7930596f0f52 [id=cb97ccf262fd4c41a180267fd0678cec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.362977 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc5de12701714fcd81b9dba6973a28b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.363651 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc5de12701714fcd81b9dba6973a28b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.367376 890 tablet_service.cc:1505] Processing CreateTablet for tablet 8b89ae724b7f41c98741a41d75592da7 (DEFAULT_TABLE table=default.Table_6bf819d1a1794e62b4cf65ffe5d2c2b5 [id=9dae0e60f978448a840aa497da35966a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.367861 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b89ae724b7f41c98741a41d75592da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.368747 1165 ts_tablet_manager.cc:1916] T b815bca893a8446e92d48973d9a0408e P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.375262 890 tablet_service.cc:1505] Processing CreateTablet for tablet c910246b3d5d4e7e8a71d9f18e94d24c (DEFAULT_TABLE table=default.Table_c2300bb8bfe44c689969a2b233d69353 [id=996745d9420c4589a52eca6bad10d6a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.375859 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c910246b3d5d4e7e8a71d9f18e94d24c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.383769 889 tablet_service.cc:1552] Processing DeleteTablet for tablet d703a5a42b2f47d0a7ffe1bb534a1b05 with delete_type TABLET_DATA_DELETED (replaced by tablet bd8fd6c977204c90b558827fc0b15f43) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:24.384064 1164 tablet_replica.cc:333] T d703a5a42b2f47d0a7ffe1bb534a1b05 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:24.384299 1164 raft_consensus.cc:2243] T d703a5a42b2f47d0a7ffe1bb534a1b05 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.384492 1164 raft_consensus.cc:2272] T d703a5a42b2f47d0a7ffe1bb534a1b05 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.384816 889 tablet_service.cc:1505] Processing CreateTablet for tablet ee893e33255745debf73c486d8c70267 (DEFAULT_TABLE table=default.Table_897d3e48834a42f59e057317699b566d [id=3940282e8e794b21b982be1f171b677d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.385202 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee893e33255745debf73c486d8c70267. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.385260 1168 tablet_bootstrap.cc:492] T bc51bc0821a8484caee39c534eb4e6af P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:24.385375 1166 tablet_bootstrap.cc:492] T 694518e6c1ae4032b06b99e9bde2e5dd P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:24.385445 1168 ts_tablet_manager.cc:1403] T bc51bc0821a8484caee39c534eb4e6af P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.149s user 0.047s sys 0.056s
I20251211 20:52:24.385526 1166 ts_tablet_manager.cc:1403] T 694518e6c1ae4032b06b99e9bde2e5dd P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.149s user 0.003s sys 0.000s
I20251211 20:52:24.386305 1166 raft_consensus.cc:359] T 694518e6c1ae4032b06b99e9bde2e5dd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.386476 1166 raft_consensus.cc:385] T 694518e6c1ae4032b06b99e9bde2e5dd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.386389 1168 raft_consensus.cc:359] T bc51bc0821a8484caee39c534eb4e6af P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.386552 1166 raft_consensus.cc:740] T 694518e6c1ae4032b06b99e9bde2e5dd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.386608 1168 raft_consensus.cc:385] T bc51bc0821a8484caee39c534eb4e6af P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.386767 1166 consensus_queue.cc:260] T 694518e6c1ae4032b06b99e9bde2e5dd P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.386848 1168 raft_consensus.cc:740] T bc51bc0821a8484caee39c534eb4e6af P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.387233 1166 ts_tablet_manager.cc:1434] T 694518e6c1ae4032b06b99e9bde2e5dd P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:24.387148 1168 consensus_queue.cc:260] T bc51bc0821a8484caee39c534eb4e6af P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.388324 1160 tablet_bootstrap.cc:492] T bc51bc0821a8484caee39c534eb4e6af P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:24.389170 1160 ts_tablet_manager.cc:1403] T bc51bc0821a8484caee39c534eb4e6af P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.187s user 0.005s sys 0.009s
I20251211 20:52:24.387538 1166 tablet_bootstrap.cc:492] T 97a519a8ca7c4f5380a230a70ee84052 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:24.391343 590 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 694518e6c1ae4032b06b99e9bde2e5dd (table default.Table_86450313ee0040bb87db3b340566266b [id=d49d5216afba41d68497f42fc72b1bf0]). Expected version 2 got 0
I20251211 20:52:24.392063 1164 ts_tablet_manager.cc:1916] T d703a5a42b2f47d0a7ffe1bb534a1b05 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.393489 1166 tablet_bootstrap.cc:654] T 97a519a8ca7c4f5380a230a70ee84052 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.393918 1160 raft_consensus.cc:359] T bc51bc0821a8484caee39c534eb4e6af P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.394191 1160 raft_consensus.cc:385] T bc51bc0821a8484caee39c534eb4e6af P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.394294 1160 raft_consensus.cc:740] T bc51bc0821a8484caee39c534eb4e6af P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.394582 1215 raft_consensus.cc:493] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:24.394723 1215 raft_consensus.cc:515] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.394734 1160 consensus_queue.cc:260] T bc51bc0821a8484caee39c534eb4e6af P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.395398 1215 leader_election.cc:290] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:24.395728 590 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 694518e6c1ae4032b06b99e9bde2e5dd (table default.Table_86450313ee0040bb87db3b340566266b [id=d49d5216afba41d68497f42fc72b1bf0]). Expected version 2 got 0
I20251211 20:52:24.396073 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "acdb7dddf3e74ce1bc92f232b6180cca" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:24.396237 1165 ts_tablet_manager.cc:1929] T b815bca893a8446e92d48973d9a0408e P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.396401 1165 log.cc:1199] T b815bca893a8446e92d48973d9a0408e P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/b815bca893a8446e92d48973d9a0408e
I20251211 20:52:24.396827 1165 ts_tablet_manager.cc:1950] T b815bca893a8446e92d48973d9a0408e P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:24.396934 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "acdb7dddf3e74ce1bc92f232b6180cca" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:24.397341 1168 ts_tablet_manager.cc:1434] T bc51bc0821a8484caee39c534eb4e6af P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.012s user 0.001s sys 0.002s
W20251211 20:52:24.397542 983 leader_election.cc:343] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:24.398525 1160 ts_tablet_manager.cc:1434] T bc51bc0821a8484caee39c534eb4e6af P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
W20251211 20:52:24.398664 982 leader_election.cc:343] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:24.398799 982 leader_election.cc:304] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:24.398933 1160 tablet_bootstrap.cc:492] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:24.399484 1168 tablet_bootstrap.cc:492] T 297dc26ade384213b364a876c54bdd14 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:24.401062 1215 raft_consensus.cc:2749] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:24.401328 1168 tablet_bootstrap.cc:654] T 297dc26ade384213b364a876c54bdd14 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.402454 750 tablet_service.cc:1505] Processing CreateTablet for tablet 49a33084c0fc460eb9592224b6dfc279 (DEFAULT_TABLE table=default.Table_b06a3971c3be404e9787e05466931423 [id=9c24845e4efe475fa792a51ce8a66be6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.402848 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49a33084c0fc460eb9592224b6dfc279. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.405166 1160 tablet_bootstrap.cc:654] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.407225 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet b815bca893a8446e92d48973d9a0408e (table default.Table_b411ab6a2d934ad8aa183b348dc24d85 [id=fd269c697f9d4c5eb4a0d646f459c954]) successfully deleted
I20251211 20:52:24.410234 1164 ts_tablet_manager.cc:1929] T d703a5a42b2f47d0a7ffe1bb534a1b05 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.410516 1164 log.cc:1199] T d703a5a42b2f47d0a7ffe1bb534a1b05 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/d703a5a42b2f47d0a7ffe1bb534a1b05
I20251211 20:52:24.410969 1164 ts_tablet_manager.cc:1950] T d703a5a42b2f47d0a7ffe1bb534a1b05 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:24.415782 739 tablet_service.cc:1552] Processing DeleteTablet for tablet bc51bc0821a8484caee39c534eb4e6af with delete_type TABLET_DATA_DELETED (replaced by tablet 6a2442a4ebc940a281f9ea1a394f6621) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.416265 1165 tablet_replica.cc:333] T bc51bc0821a8484caee39c534eb4e6af P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:24.416471 1165 raft_consensus.cc:2243] T bc51bc0821a8484caee39c534eb4e6af P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.416694 1165 raft_consensus.cc:2272] T bc51bc0821a8484caee39c534eb4e6af P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.417470 1165 ts_tablet_manager.cc:1916] T bc51bc0821a8484caee39c534eb4e6af P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.417726 878 tablet_service.cc:1505] Processing CreateTablet for tablet 3eac21a07ba34e1fa275e25ee9dcabb1 (DEFAULT_TABLE table=default.Table_86e5b1e5b4214b48a21f398642d97640 [id=9700f9fe6e3044748e34954bef8a0038]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.418169 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3eac21a07ba34e1fa275e25ee9dcabb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.424497 891 tablet_service.cc:1505] Processing CreateTablet for tablet c4b76a7e84d344268567c551236ce772 (DEFAULT_TABLE table=default.Table_afd3c9ee35ad41caa1717a299443409e [id=b0ebaab8cd5a4deb9a5ea1e7bd3ccec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.424975 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4b76a7e84d344268567c551236ce772. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.429193 1165 ts_tablet_manager.cc:1929] T bc51bc0821a8484caee39c534eb4e6af P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.429366 1165 log.cc:1199] T bc51bc0821a8484caee39c534eb4e6af P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/bc51bc0821a8484caee39c534eb4e6af
I20251211 20:52:24.429771 1165 ts_tablet_manager.cc:1950] T bc51bc0821a8484caee39c534eb4e6af P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:24.434403 889 tablet_service.cc:1505] Processing CreateTablet for tablet 19d0aa259ac94f36b7f6a24d11a3d420 (DEFAULT_TABLE table=default.Table_ef1c4ca915474d569c91296213346105 [id=9dd03042ff0948b99b39b899255848fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.435110 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19d0aa259ac94f36b7f6a24d11a3d420. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.441123 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet bc51bc0821a8484caee39c534eb4e6af (table default.Table_cbe68d84efef4745822f6aef5a723f11 [id=f9c1230a7d71435cac705d4675a9908d]) successfully deleted
I20251211 20:52:24.444620 894 tablet_service.cc:1505] Processing CreateTablet for tablet dc5de12701714fcd81b9dba6973a28b5 (DEFAULT_TABLE table=default.Table_f572100667d148e8a69e7930596f0f52 [id=cb97ccf262fd4c41a180267fd0678cec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.445165 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc5de12701714fcd81b9dba6973a28b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.452736 884 tablet_service.cc:1505] Processing CreateTablet for tablet 88066d94d5d34e0fab20da9fb6ef178e (DEFAULT_TABLE table=default.Table_b06f9587f78645bb9712c3b080f3c0d4 [id=ea12bdaa894f446da22635f95d97ac2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.453292 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88066d94d5d34e0fab20da9fb6ef178e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.460713 883 tablet_service.cc:1505] Processing CreateTablet for tablet 5fd2a6cf80a04f20aef8ff9c84bc88db (DEFAULT_TABLE table=default.Table_09578b67c6ea4eb180bc57baafec6d07 [id=10fd0368198c447c95b692b1f3e6a023]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.461297 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fd2a6cf80a04f20aef8ff9c84bc88db. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.468076 891 tablet_service.cc:1505] Processing CreateTablet for tablet ae1abef61473443b9922eb13f2ae92fa (DEFAULT_TABLE table=default.Table_40d581fe03ef48d1b5639a95ffc00b7b [id=66810eaf071c4613a4d30dfa016788a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.468806 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae1abef61473443b9922eb13f2ae92fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.475255 878 tablet_service.cc:1505] Processing CreateTablet for tablet 49a33084c0fc460eb9592224b6dfc279 (DEFAULT_TABLE table=default.Table_b06a3971c3be404e9787e05466931423 [id=9c24845e4efe475fa792a51ce8a66be6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.475899 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49a33084c0fc460eb9592224b6dfc279. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.484424 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c2300bb8bfe44c689969a2b233d69353" } modify_external_catalogs: true
I20251211 20:52:24.484696 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c2300bb8bfe44c689969a2b233d69353" } modify_external_catalogs: true
I20251211 20:52:24.485812 894 tablet_service.cc:1552] Processing DeleteTablet for tablet bc51bc0821a8484caee39c534eb4e6af with delete_type TABLET_DATA_DELETED (replaced by tablet 6a2442a4ebc940a281f9ea1a394f6621) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:24.486169 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_03fdaef4fcc144f485f238db0e76fab4" } modify_external_catalogs: true
I20251211 20:52:24.486378 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_03fdaef4fcc144f485f238db0e76fab4" } modify_external_catalogs: true
I20251211 20:52:24.486546 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet d703a5a42b2f47d0a7ffe1bb534a1b05 (table default.Table_5b8aa62e6fdf46089c4c8ae541856fbe [id=e5caa22bce84422290707816a93bd3f5]) successfully deleted
I20251211 20:52:24.489820 880 tablet_service.cc:1505] Processing CreateTablet for tablet 27a08a64d92342d08ab4406e8006a97b (DEFAULT_TABLE table=default.Table_19db6b94abda4124a0e45f11b8ca0990 [id=383392d835da4c50848068d83261cecc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.490351 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27a08a64d92342d08ab4406e8006a97b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.496320 1164 tablet_replica.cc:333] T bc51bc0821a8484caee39c534eb4e6af P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:24.496551 1164 raft_consensus.cc:2243] T bc51bc0821a8484caee39c534eb4e6af P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.496747 1166 tablet_bootstrap.cc:492] T 97a519a8ca7c4f5380a230a70ee84052 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:24.496765 1164 raft_consensus.cc:2272] T bc51bc0821a8484caee39c534eb4e6af P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.496939 1166 ts_tablet_manager.cc:1403] T 97a519a8ca7c4f5380a230a70ee84052 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.109s user 0.048s sys 0.042s
I20251211 20:52:24.497745 1166 raft_consensus.cc:359] T 97a519a8ca7c4f5380a230a70ee84052 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:24.498018 1166 raft_consensus.cc:385] T 97a519a8ca7c4f5380a230a70ee84052 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.498097 1166 raft_consensus.cc:740] T 97a519a8ca7c4f5380a230a70ee84052 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.498347 1166 consensus_queue.cc:260] T 97a519a8ca7c4f5380a230a70ee84052 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:24.499989 1166 ts_tablet_manager.cc:1434] T 97a519a8ca7c4f5380a230a70ee84052 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:24.501097 1166 tablet_bootstrap.cc:492] T 1951a67964834152afb31f95741d981a P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:24.502892 1166 tablet_bootstrap.cc:654] T 1951a67964834152afb31f95741d981a P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.505805 1164 ts_tablet_manager.cc:1916] T bc51bc0821a8484caee39c534eb4e6af P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:24.510818 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2630ad57845a4c31b20c8a104dd27335 consensus state has no leader
I20251211 20:52:24.510926 1166 tablet_bootstrap.cc:492] T 1951a67964834152afb31f95741d981a P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:24.511077 1166 ts_tablet_manager.cc:1403] T 1951a67964834152afb31f95741d981a P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20251211 20:52:24.511960 1166 raft_consensus.cc:359] T 1951a67964834152afb31f95741d981a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.512162 1166 raft_consensus.cc:385] T 1951a67964834152afb31f95741d981a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.512254 1166 raft_consensus.cc:740] T 1951a67964834152afb31f95741d981a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.512488 1166 consensus_queue.cc:260] T 1951a67964834152afb31f95741d981a P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.512732 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_ae7f102bdee3470990e863235f08d636"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:24.513082 1166 ts_tablet_manager.cc:1434] T 1951a67964834152afb31f95741d981a P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20251211 20:52:24.513357 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ae7f102bdee3470990e863235f08d636 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.514484 1166 tablet_bootstrap.cc:492] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:24.517077 1166 tablet_bootstrap.cc:654] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.519428 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 97a519a8ca7c4f5380a230a70ee84052 with delete_type TABLET_DATA_DELETED (replaced by tablet 27a08a64d92342d08ab4406e8006a97b) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.519753 1163 tablet_replica.cc:333] T 97a519a8ca7c4f5380a230a70ee84052 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:24.519968 1163 raft_consensus.cc:2243] T 97a519a8ca7c4f5380a230a70ee84052 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.520150 1163 raft_consensus.cc:2272] T 97a519a8ca7c4f5380a230a70ee84052 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.520750 1163 ts_tablet_manager.cc:1916] T 97a519a8ca7c4f5380a230a70ee84052 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.526701 1163 ts_tablet_manager.cc:1929] T 97a519a8ca7c4f5380a230a70ee84052 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.526871 1163 log.cc:1199] T 97a519a8ca7c4f5380a230a70ee84052 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/97a519a8ca7c4f5380a230a70ee84052
I20251211 20:52:24.527212 1163 ts_tablet_manager.cc:1950] T 97a519a8ca7c4f5380a230a70ee84052 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:24.529997 1168 tablet_bootstrap.cc:492] T 297dc26ade384213b364a876c54bdd14 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:24.530493 1168 ts_tablet_manager.cc:1403] T 297dc26ade384213b364a876c54bdd14 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.131s user 0.024s sys 0.027s
I20251211 20:52:24.532399 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet c910246b3d5d4e7e8a71d9f18e94d24c
W20251211 20:52:24.533200 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c58241f7e1404a8c8db92c4f9a47d143 consensus state has no leader
I20251211 20:52:24.533747 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_638eea7803484ee9965a1058a3526993"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:24.534474 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_638eea7803484ee9965a1058a3526993 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.535851 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 27466459d8e6411ab87ac474fb617735
I20251211 20:52:24.536341 592 catalog_manager.cc:6724] ReplaceTablet: tablet 27466459d8e6411ab87ac474fb617735 deleted and replaced by tablet 8feee01fc0aa4b32be926ff2b617b894
I20251211 20:52:24.536712 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_19db6b94abda4124a0e45f11b8ca0990" } new_table_name: "default.Table_c91a79fb73a549ea9656555dc2e1c780" modify_external_catalogs: true
I20251211 20:52:24.535651 1168 raft_consensus.cc:359] T 297dc26ade384213b364a876c54bdd14 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.537638 1168 raft_consensus.cc:385] T 297dc26ade384213b364a876c54bdd14 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.537771 1168 raft_consensus.cc:740] T 297dc26ade384213b364a876c54bdd14 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.538257 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 78922f26cedb4bb989aa1c1cb40bfa3a
I20251211 20:52:24.538383 1168 consensus_queue.cc:260] T 297dc26ade384213b364a876c54bdd14 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.539080 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_a6b9ed785d804b10b942d36ac7bfd209"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:24.539745 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a6b9ed785d804b10b942d36ac7bfd209 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.541141 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet c38c1a4c0e574eb0957de127d41c26ad
I20251211 20:52:24.541759 588 master_service.cc:954] ReplaceTablet: received request to replace tablet e69d53ac90a7463aa36ff15dadad7e0b from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:24.541921 1168 ts_tablet_manager.cc:1434] T 297dc26ade384213b364a876c54bdd14 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.011s user 0.000s sys 0.002s
I20251211 20:52:24.542295 1168 tablet_bootstrap.cc:492] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:24.542366 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_ef7b6f5df7224e2ebd3c6ec172e9aff9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:24.542907 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef7b6f5df7224e2ebd3c6ec172e9aff9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.543819 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 353da9106578450987a4f6544df8d7e6
I20251211 20:52:24.544190 595 catalog_manager.cc:6724] ReplaceTablet: tablet 353da9106578450987a4f6544df8d7e6 deleted and replaced by tablet d762b4afb5ff447dac3468393826205a
I20251211 20:52:24.544461 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_069bb219e15e43be995ff011e2289a84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:24.544934 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_069bb219e15e43be995ff011e2289a84 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.545094 1166 tablet_bootstrap.cc:492] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:24.545259 1166 ts_tablet_manager.cc:1403] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.031s user 0.006s sys 0.013s
I20251211 20:52:24.546056 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 5fc8d485cc234fd490c487a8bc881bf4
I20251211 20:52:24.546030 1166 raft_consensus.cc:359] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.546224 1166 raft_consensus.cc:385] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.546295 1166 raft_consensus.cc:740] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.546537 591 catalog_manager.cc:6724] ReplaceTablet: tablet 5fc8d485cc234fd490c487a8bc881bf4 deleted and replaced by tablet 28ef13be3b84410d9c786c43502a3478
I20251211 20:52:24.546563 1166 consensus_queue.cc:260] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.546859 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_03555bd3f502404a928ffdf4ef934e49"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:24.547420 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_03555bd3f502404a928ffdf4ef934e49 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.548122 1168 tablet_bootstrap.cc:654] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.548705 1215 raft_consensus.cc:493] T 694518e6c1ae4032b06b99e9bde2e5dd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:24.548867 1215 raft_consensus.cc:515] T 694518e6c1ae4032b06b99e9bde2e5dd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.549553 1215 leader_election.cc:290] T 694518e6c1ae4032b06b99e9bde2e5dd P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:24.551412 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 9adaee28edd74df68b44b25d505fdfcb
I20251211 20:52:24.552227 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b3e8496af4d340218ce858c81796a29c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:24.552863 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b3e8496af4d340218ce858c81796a29c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.554342 1166 ts_tablet_manager.cc:1434] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.009s user 0.004s sys 0.004s
I20251211 20:52:24.555519 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "694518e6c1ae4032b06b99e9bde2e5dd" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:24.555696 1166 tablet_bootstrap.cc:492] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:24.556177 1168 tablet_bootstrap.cc:492] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
W20251211 20:52:24.556174 982 leader_election.cc:343] T 694518e6c1ae4032b06b99e9bde2e5dd P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:24.556350 1168 ts_tablet_manager.cc:1403] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.003s
I20251211 20:52:24.557408 1166 tablet_bootstrap.cc:654] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.557293 1168 raft_consensus.cc:359] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.557543 1168 raft_consensus.cc:385] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.557618 1168 raft_consensus.cc:740] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.557884 1168 consensus_queue.cc:260] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.561332 1168 ts_tablet_manager.cc:1434] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251211 20:52:24.561761 1168 tablet_bootstrap.cc:492] T 73aa2a3fb2f44bbca870fb7f4cf6087c P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:24.565369 1166 tablet_bootstrap.cc:492] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:24.565544 1166 ts_tablet_manager.cc:1403] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.006s
I20251211 20:52:24.566337 1166 raft_consensus.cc:359] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.566525 1166 raft_consensus.cc:385] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.566610 1166 raft_consensus.cc:740] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.566848 1166 consensus_queue.cc:260] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.567548 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_d282d0b8f49e4f44998af55480ad3cb8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:24.568421 1166 ts_tablet_manager.cc:1434] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:24.569141 1168 tablet_bootstrap.cc:654] T 73aa2a3fb2f44bbca870fb7f4cf6087c P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.570245 1166 tablet_bootstrap.cc:492] T ec7df7b11ce34b8489048e6acdcb4e94 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:24.571971 1166 tablet_bootstrap.cc:654] T ec7df7b11ce34b8489048e6acdcb4e94 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.575466 1168 tablet_bootstrap.cc:492] T 73aa2a3fb2f44bbca870fb7f4cf6087c P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:24.575713 1168 ts_tablet_manager.cc:1403] T 73aa2a3fb2f44bbca870fb7f4cf6087c P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.002s
I20251211 20:52:24.576750 1168 raft_consensus.cc:359] T 73aa2a3fb2f44bbca870fb7f4cf6087c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.577020 1168 raft_consensus.cc:385] T 73aa2a3fb2f44bbca870fb7f4cf6087c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.577138 1168 raft_consensus.cc:740] T 73aa2a3fb2f44bbca870fb7f4cf6087c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
W20251211 20:52:24.579200 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d282d0b8f49e4f44998af55480ad3cb8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.580278 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet c910246b3d5d4e7e8a71d9f18e94d24c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.580304 1168 consensus_queue.cc:260] T 73aa2a3fb2f44bbca870fb7f4cf6087c P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.581101 1166 tablet_bootstrap.cc:492] T ec7df7b11ce34b8489048e6acdcb4e94 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:24.581285 1166 ts_tablet_manager.cc:1403] T ec7df7b11ce34b8489048e6acdcb4e94 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.001s
I20251211 20:52:24.581631 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 27466459d8e6411ab87ac474fb617735 with delete_type TABLET_DATA_DELETED (replaced by tablet 8feee01fc0aa4b32be926ff2b617b894) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.582193 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 27466459d8e6411ab87ac474fb617735 with delete_type TABLET_DATA_DELETED (replaced by tablet 8feee01fc0aa4b32be926ff2b617b894) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.582774 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 78922f26cedb4bb989aa1c1cb40bfa3a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.582823 1166 raft_consensus.cc:359] T ec7df7b11ce34b8489048e6acdcb4e94 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:24.583019 1166 raft_consensus.cc:385] T ec7df7b11ce34b8489048e6acdcb4e94 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.583093 1166 raft_consensus.cc:740] T ec7df7b11ce34b8489048e6acdcb4e94 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.583344 1166 consensus_queue.cc:260] T ec7df7b11ce34b8489048e6acdcb4e94 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:24.583752 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet c38c1a4c0e574eb0957de127d41c26ad with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.584877 1168 ts_tablet_manager.cc:1434] T 73aa2a3fb2f44bbca870fb7f4cf6087c P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20251211 20:52:24.584666 1166 ts_tablet_manager.cc:1434] T ec7df7b11ce34b8489048e6acdcb4e94 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:24.585198 1166 tablet_bootstrap.cc:492] T 732c76659c0540bc8b6909b430ffb146 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:24.585253 1168 tablet_bootstrap.cc:492] T 97a519a8ca7c4f5380a230a70ee84052 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:24.586308 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 78922f26cedb4bb989aa1c1cb40bfa3a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.587502 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 353da9106578450987a4f6544df8d7e6 with delete_type TABLET_DATA_DELETED (replaced by tablet d762b4afb5ff447dac3468393826205a) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.587755 1163 tablet_replica.cc:333] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
W20251211 20:52:24.587886 582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.61:40749, user_credentials={real_user=slave}} blocked reactor thread for 61018us
I20251211 20:52:24.587934 1163 raft_consensus.cc:2243] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.588101 1163 raft_consensus.cc:2272] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.588663 1163 ts_tablet_manager.cc:1916] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.589393 890 tablet_service.cc:1552] Processing DeleteTablet for tablet c910246b3d5d4e7e8a71d9f18e94d24c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:24.591848 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "694518e6c1ae4032b06b99e9bde2e5dd" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:24.592414 983 leader_election.cc:343] T 694518e6c1ae4032b06b99e9bde2e5dd P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:24.592559 983 leader_election.cc:304] T 694518e6c1ae4032b06b99e9bde2e5dd P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:24.592931 1340 raft_consensus.cc:2749] T 694518e6c1ae4032b06b99e9bde2e5dd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:24.593516 1168 tablet_bootstrap.cc:654] T 97a519a8ca7c4f5380a230a70ee84052 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.593870 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fc8d485cc234fd490c487a8bc881bf4 with delete_type TABLET_DATA_DELETED (replaced by tablet 28ef13be3b84410d9c786c43502a3478) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.594337 739 tablet_service.cc:1552] Processing DeleteTablet for tablet c38c1a4c0e574eb0957de127d41c26ad with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.595283 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 9adaee28edd74df68b44b25d505fdfcb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.596405 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fc8d485cc234fd490c487a8bc881bf4 with delete_type TABLET_DATA_DELETED (replaced by tablet 28ef13be3b84410d9c786c43502a3478) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.597096 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 9adaee28edd74df68b44b25d505fdfcb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.597535 739 tablet_service.cc:1552] Processing DeleteTablet for tablet 353da9106578450987a4f6544df8d7e6 with delete_type TABLET_DATA_DELETED (replaced by tablet d762b4afb5ff447dac3468393826205a) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.604367 1168 tablet_bootstrap.cc:492] T 97a519a8ca7c4f5380a230a70ee84052 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:24.604560 1168 ts_tablet_manager.cc:1403] T 97a519a8ca7c4f5380a230a70ee84052 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.005s
I20251211 20:52:24.604836 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 97a519a8ca7c4f5380a230a70ee84052 (table default.Table_19db6b94abda4124a0e45f11b8ca0990 [id=383392d835da4c50848068d83261cecc]) successfully deleted
W20251211 20:52:24.605135 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet c910246b3d5d4e7e8a71d9f18e94d24c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c910246b3d5d4e7e8a71d9f18e94d24c already in progress: creating tablet
I20251211 20:52:24.605275 1168 raft_consensus.cc:359] T 97a519a8ca7c4f5380a230a70ee84052 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:24.605365 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 27466459d8e6411ab87ac474fb617735 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27466459d8e6411ab87ac474fb617735 already in progress: creating tablet
I20251211 20:52:24.605484 1168 raft_consensus.cc:385] T 97a519a8ca7c4f5380a230a70ee84052 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.605559 1168 raft_consensus.cc:740] T 97a519a8ca7c4f5380a230a70ee84052 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
W20251211 20:52:24.605587 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 78922f26cedb4bb989aa1c1cb40bfa3a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78922f26cedb4bb989aa1c1cb40bfa3a already in progress: creating tablet
W20251211 20:52:24.605753 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet c38c1a4c0e574eb0957de127d41c26ad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c38c1a4c0e574eb0957de127d41c26ad already in progress: creating tablet
I20251211 20:52:24.605808 1168 consensus_queue.cc:260] T 97a519a8ca7c4f5380a230a70ee84052 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:24.606041 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 27466459d8e6411ab87ac474fb617735 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27466459d8e6411ab87ac474fb617735 already in progress: creating tablet
W20251211 20:52:24.606216 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 78922f26cedb4bb989aa1c1cb40bfa3a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78922f26cedb4bb989aa1c1cb40bfa3a already in progress: creating tablet
I20251211 20:52:24.609088 741 tablet_service.cc:1552] Processing DeleteTablet for tablet c910246b3d5d4e7e8a71d9f18e94d24c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.609153 1168 ts_tablet_manager.cc:1434] T 97a519a8ca7c4f5380a230a70ee84052 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251211 20:52:24.609766 1168 tablet_bootstrap.cc:492] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:24.610247 1164 ts_tablet_manager.cc:1929] T bc51bc0821a8484caee39c534eb4e6af P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.610386 1164 log.cc:1199] T bc51bc0821a8484caee39c534eb4e6af P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/bc51bc0821a8484caee39c534eb4e6af
I20251211 20:52:24.610769 1164 ts_tablet_manager.cc:1950] T bc51bc0821a8484caee39c534eb4e6af P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:24.612340 1168 tablet_bootstrap.cc:654] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.612567 1160 tablet_bootstrap.cc:492] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:24.612736 1160 ts_tablet_manager.cc:1403] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.214s user 0.071s sys 0.082s
W20251211 20:52:24.612613 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet c38c1a4c0e574eb0957de127d41c26ad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c38c1a4c0e574eb0957de127d41c26ad already in progress: creating tablet
I20251211 20:52:24.614528 1166 tablet_bootstrap.cc:654] T 732c76659c0540bc8b6909b430ffb146 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.615782 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 353da9106578450987a4f6544df8d7e6 with delete_type TABLET_DATA_DELETED (replaced by tablet d762b4afb5ff447dac3468393826205a) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:24.616004 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 9adaee28edd74df68b44b25d505fdfcb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:24.616122 890 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fc8d485cc234fd490c487a8bc881bf4 with delete_type TABLET_DATA_DELETED (replaced by tablet 28ef13be3b84410d9c786c43502a3478) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:24.616209 1160 raft_consensus.cc:359] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.616354 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 78922f26cedb4bb989aa1c1cb40bfa3a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:24.616426 1160 raft_consensus.cc:385] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.616523 1160 raft_consensus.cc:740] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.616617 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 27466459d8e6411ab87ac474fb617735 with delete_type TABLET_DATA_DELETED (replaced by tablet 8feee01fc0aa4b32be926ff2b617b894) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:24.616776 1160 consensus_queue.cc:260] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.616895 878 tablet_service.cc:1552] Processing DeleteTablet for tablet c38c1a4c0e574eb0957de127d41c26ad with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:24.617300 1160 ts_tablet_manager.cc:1434] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251211 20:52:24.617723 1160 tablet_bootstrap.cc:492] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:24.619396 1168 tablet_bootstrap.cc:492] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:24.619568 1168 ts_tablet_manager.cc:1403] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20251211 20:52:24.620059 1160 tablet_bootstrap.cc:654] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.620498 1168 raft_consensus.cc:359] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.620712 1168 raft_consensus.cc:385] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.620811 1168 raft_consensus.cc:740] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.621045 1168 consensus_queue.cc:260] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.621659 1168 ts_tablet_manager.cc:1434] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:24.625577 1182 raft_consensus.cc:493] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:24.625768 1182 raft_consensus.cc:515] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.625984 1168 tablet_bootstrap.cc:492] T 353da9106578450987a4f6544df8d7e6 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:24.626314 1182 leader_election.cc:290] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:24.627008 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "acdb7dddf3e74ce1bc92f232b6180cca" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:24.627208 1049 raft_consensus.cc:2468] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:24.627874 710 leader_election.cc:304] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:24.628166 1182 raft_consensus.cc:2804] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:24.628252 1182 raft_consensus.cc:493] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:24.628357 1182 raft_consensus.cc:3060] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:24.629096 1166 tablet_bootstrap.cc:492] T 732c76659c0540bc8b6909b430ffb146 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:24.629256 1166 ts_tablet_manager.cc:1403] T 732c76659c0540bc8b6909b430ffb146 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.044s user 0.001s sys 0.002s
I20251211 20:52:24.630247 1166 raft_consensus.cc:359] T 732c76659c0540bc8b6909b430ffb146 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.630465 1166 raft_consensus.cc:385] T 732c76659c0540bc8b6909b430ffb146 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.630560 1166 raft_consensus.cc:740] T 732c76659c0540bc8b6909b430ffb146 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.630832 1166 consensus_queue.cc:260] T 732c76659c0540bc8b6909b430ffb146 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.631036 1182 raft_consensus.cc:515] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.631659 1166 ts_tablet_manager.cc:1434] T 732c76659c0540bc8b6909b430ffb146 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20251211 20:52:24.632647 1166 tablet_bootstrap.cc:492] T 4870da0224e048ed82f227d7ba676a94 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:24.632637 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "acdb7dddf3e74ce1bc92f232b6180cca" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:24.633044 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "acdb7dddf3e74ce1bc92f232b6180cca" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:24.633503 711 leader_election.cc:343] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:24.633718 711 leader_election.cc:343] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:24.634547 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "acdb7dddf3e74ce1bc92f232b6180cca" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:24.634717 1049 raft_consensus.cc:3060] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:24.634963 1166 tablet_bootstrap.cc:654] T 4870da0224e048ed82f227d7ba676a94 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.635742 1182 leader_election.cc:290] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:24.637887 1049 raft_consensus.cc:2468] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:24.638345 710 leader_election.cc:304] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:24.639890 1182 raft_consensus.cc:2804] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:24.639979 1168 tablet_bootstrap.cc:654] T 353da9106578450987a4f6544df8d7e6 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.647930 1236 raft_consensus.cc:493] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:24.648128 1236 raft_consensus.cc:515] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.648759 1236 leader_election.cc:290] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
W20251211 20:52:24.649745 847 leader_election.cc:343] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: edc4300dc5a84173aa95b587d77fc95c
W20251211 20:52:24.650151 847 leader_election.cc:343] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: edc4300dc5a84173aa95b587d77fc95c
I20251211 20:52:24.650265 847 leader_election.cc:304] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:24.618849 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet bc51bc0821a8484caee39c534eb4e6af (table default.Table_cbe68d84efef4745822f6aef5a723f11 [id=f9c1230a7d71435cac705d4675a9908d]) successfully deleted
I20251211 20:52:24.651021 1236 raft_consensus.cc:2749] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:24.651165 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet c910246b3d5d4e7e8a71d9f18e94d24c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c910246b3d5d4e7e8a71d9f18e94d24c already in progress: creating tablet
W20251211 20:52:24.651404 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 353da9106578450987a4f6544df8d7e6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 353da9106578450987a4f6544df8d7e6 already in progress: creating tablet
W20251211 20:52:24.651661 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 9adaee28edd74df68b44b25d505fdfcb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9adaee28edd74df68b44b25d505fdfcb already in progress: creating tablet
W20251211 20:52:24.651865 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 5fc8d485cc234fd490c487a8bc881bf4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5fc8d485cc234fd490c487a8bc881bf4 already in progress: creating tablet
W20251211 20:52:24.652057 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 78922f26cedb4bb989aa1c1cb40bfa3a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78922f26cedb4bb989aa1c1cb40bfa3a already in progress: creating tablet
W20251211 20:52:24.652245 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 27466459d8e6411ab87ac474fb617735 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27466459d8e6411ab87ac474fb617735 already in progress: creating tablet
W20251211 20:52:24.652494 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet c38c1a4c0e574eb0957de127d41c26ad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c38c1a4c0e574eb0957de127d41c26ad already in progress: creating tablet
I20251211 20:52:24.659572 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b7c88854437f4dfd9998ccb8246e6d0d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:24.663442 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b7c88854437f4dfd9998ccb8246e6d0d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.660533 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_ddb3c1574c924ada922b623f332a3613"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:24.665007 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ddb3c1574c924ada922b623f332a3613 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.661283 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_897d3e48834a42f59e057317699b566d" } new_table_name: "default.Table_da9a07e5af4a433c8e02a128e8843084" modify_external_catalogs: true
I20251211 20:52:24.661724 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_09578b67c6ea4eb180bc57baafec6d07" } new_table_name: "default.Table_15a4631cb9ae4a8d892505a404f973f1" modify_external_catalogs: true
I20251211 20:52:24.662472 588 master_service.cc:954] ReplaceTablet: received request to replace tablet 2433068ec179490dac3e91d046a42b70 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:24.674152 1163 ts_tablet_manager.cc:1929] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.674295 1163 log.cc:1199] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/353da9106578450987a4f6544df8d7e6
I20251211 20:52:24.674625 1163 ts_tablet_manager.cc:1950] T 353da9106578450987a4f6544df8d7e6 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
W20251211 20:52:24.676687 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 5fc8d485cc234fd490c487a8bc881bf4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5fc8d485cc234fd490c487a8bc881bf4 already in progress: creating tablet
W20251211 20:52:24.677032 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 9adaee28edd74df68b44b25d505fdfcb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9adaee28edd74df68b44b25d505fdfcb already in progress: creating tablet
W20251211 20:52:24.677253 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 353da9106578450987a4f6544df8d7e6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 353da9106578450987a4f6544df8d7e6 already in progress: creating tablet
W20251211 20:52:24.677477 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet c910246b3d5d4e7e8a71d9f18e94d24c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c910246b3d5d4e7e8a71d9f18e94d24c already in progress: creating tablet
I20251211 20:52:24.678088 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 297dc26ade384213b364a876c54bdd14 with delete_type TABLET_DATA_DELETED (replaced by tablet 8dff13e22899452c8d4f2731b8dac7d5) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.678370 1165 tablet_replica.cc:333] T 297dc26ade384213b364a876c54bdd14 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:24.678575 1165 raft_consensus.cc:2243] T 297dc26ade384213b364a876c54bdd14 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.678751 1165 raft_consensus.cc:2272] T 297dc26ade384213b364a876c54bdd14 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.679710 1165 ts_tablet_manager.cc:1916] T 297dc26ade384213b364a876c54bdd14 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.684113 590 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 1951a67964834152afb31f95741d981a (table default.Table_39124c0c70034d00818c18db160416cf [id=016a0bf9d2704b74af4e64d2f9a23878]). Expected version 2 got 0
W20251211 20:52:24.685829 592 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 27a08a64d92342d08ab4406e8006a97b consensus state has no leader
I20251211 20:52:24.686451 878 tablet_service.cc:1505] Processing CreateTablet for tablet cf1201bba4b6421f8d7ce593f4bd892b (DEFAULT_TABLE table=default.Table_ca483456010c49c6a291fbaeb6f864ef [id=ef458da415914c138373fbd5e0b8dec4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.686918 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf1201bba4b6421f8d7ce593f4bd892b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.690189 590 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 1951a67964834152afb31f95741d981a (table default.Table_39124c0c70034d00818c18db160416cf [id=016a0bf9d2704b74af4e64d2f9a23878]). Expected version 2 got 0
I20251211 20:52:24.690380 590 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet a5eeb56709f046d9be92ac600ebc330e (table default.Table_f2ce490e0c5d445ab84989aae77c2f19 [id=f78e1da479af4d23a552af1c4fc55f81]). Expected version 1 got 0
I20251211 20:52:24.690580 590 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 33e1d8a576cd41b1a34778b1458df396 (table default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]). Expected version 2 got 0
I20251211 20:52:24.695685 1165 ts_tablet_manager.cc:1929] T 297dc26ade384213b364a876c54bdd14 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.696162 1165 log.cc:1199] T 297dc26ade384213b364a876c54bdd14 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/297dc26ade384213b364a876c54bdd14
I20251211 20:52:24.696856 1165 ts_tablet_manager.cc:1950] T 297dc26ade384213b364a876c54bdd14 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:24.698836 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 353da9106578450987a4f6544df8d7e6 (table default.Table_a0c412985adf4382a7ea28c77654fd6c [id=4a74b9a60cbb4198b55cee630b2f4bef]) successfully deleted
W20251211 20:52:24.699683 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 5fc8d485cc234fd490c487a8bc881bf4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5fc8d485cc234fd490c487a8bc881bf4 already in progress: creating tablet
I20251211 20:52:24.700071 878 tablet_service.cc:1505] Processing CreateTablet for tablet bb5c5523e175406d9f6f9cd2736a2045 (DEFAULT_TABLE table=default.Table_349fa502a7c44fe79baed28f7da89ff9 [id=1bd6b7930f2447659da365b7c06b1640]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.700606 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb5c5523e175406d9f6f9cd2736a2045. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:24.700747 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 9adaee28edd74df68b44b25d505fdfcb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9adaee28edd74df68b44b25d505fdfcb already in progress: creating tablet
I20251211 20:52:24.703088 891 tablet_service.cc:1552] Processing DeleteTablet for tablet edc4300dc5a84173aa95b587d77fc95c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:24.703374 1164 tablet_replica.cc:333] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:24.703333 741 tablet_service.cc:1505] Processing CreateTablet for tablet cf1201bba4b6421f8d7ce593f4bd892b (DEFAULT_TABLE table=default.Table_ca483456010c49c6a291fbaeb6f864ef [id=ef458da415914c138373fbd5e0b8dec4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.703603 1164 raft_consensus.cc:2243] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.703759 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf1201bba4b6421f8d7ce593f4bd892b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.703804 1164 raft_consensus.cc:2272] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.704596 1164 ts_tablet_manager.cc:1916] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.708029 1182 raft_consensus.cc:697] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:24.708407 1182 consensus_queue.cc:237] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.710529 739 tablet_service.cc:1505] Processing CreateTablet for tablet bb5c5523e175406d9f6f9cd2736a2045 (DEFAULT_TABLE table=default.Table_349fa502a7c44fe79baed28f7da89ff9 [id=1bd6b7930f2447659da365b7c06b1640]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.710947 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb5c5523e175406d9f6f9cd2736a2045. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.712100 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet ec7df7b11ce34b8489048e6acdcb4e94 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.712512 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 732c76659c0540bc8b6909b430ffb146 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.713084 1020 tablet_service.cc:1505] Processing CreateTablet for tablet cf1201bba4b6421f8d7ce593f4bd892b (DEFAULT_TABLE table=default.Table_ca483456010c49c6a291fbaeb6f864ef [id=ef458da415914c138373fbd5e0b8dec4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.713503 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf1201bba4b6421f8d7ce593f4bd892b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.715798 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 97a519a8ca7c4f5380a230a70ee84052 with delete_type TABLET_DATA_DELETED (replaced by tablet 27a08a64d92342d08ab4406e8006a97b) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.717116 1163 tablet_replica.cc:333] T ec7df7b11ce34b8489048e6acdcb4e94 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:24.717392 1163 raft_consensus.cc:2243] T ec7df7b11ce34b8489048e6acdcb4e94 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.717628 1163 raft_consensus.cc:2272] T ec7df7b11ce34b8489048e6acdcb4e94 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.718454 1163 ts_tablet_manager.cc:1916] T ec7df7b11ce34b8489048e6acdcb4e94 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.719990 1164 ts_tablet_manager.cc:1929] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.720181 1164 log.cc:1199] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/edc4300dc5a84173aa95b587d77fc95c
I20251211 20:52:24.720520 1164 ts_tablet_manager.cc:1950] T edc4300dc5a84173aa95b587d77fc95c P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:24.721617 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9540f76e3f984e92b632bc45a25f4b8c" } new_table_name: "default.Table_cec7f350a9d04bd590db8a29fb8ceb6b" modify_external_catalogs: true
I20251211 20:52:24.723331 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 297dc26ade384213b364a876c54bdd14 (table default.Table_f572100667d148e8a69e7930596f0f52 [id=cb97ccf262fd4c41a180267fd0678cec]) successfully deleted
I20251211 20:52:24.724906 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet edc4300dc5a84173aa95b587d77fc95c (table default.Table_4f9b0d5d6ec647df87f89a25344ca9f8 [id=32fff752ffaa46e6af5221a3ecf2adc9]) successfully deleted
W20251211 20:52:24.727605 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ee893e33255745debf73c486d8c70267 consensus state has no leader
W20251211 20:52:24.728120 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5fd2a6cf80a04f20aef8ff9c84bc88db consensus state has no leader
I20251211 20:52:24.731331 1166 tablet_bootstrap.cc:492] T 4870da0224e048ed82f227d7ba676a94 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:24.731572 1166 ts_tablet_manager.cc:1403] T 4870da0224e048ed82f227d7ba676a94 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.100s user 0.006s sys 0.020s
I20251211 20:52:24.732242 891 tablet_service.cc:1505] Processing CreateTablet for tablet f93006227cc6489e91f3593e11789b54 (DEFAULT_TABLE table=default.Table_4cd9b395cfa94302b298a3df4971d631 [id=42b8ff20f93f4d3fa10fe7ff92dc598d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.732705 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f93006227cc6489e91f3593e11789b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.732672 739 tablet_service.cc:1505] Processing CreateTablet for tablet f93006227cc6489e91f3593e11789b54 (DEFAULT_TABLE table=default.Table_4cd9b395cfa94302b298a3df4971d631 [id=42b8ff20f93f4d3fa10fe7ff92dc598d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.732682 1166 raft_consensus.cc:359] T 4870da0224e048ed82f227d7ba676a94 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.732942 1166 raft_consensus.cc:385] T 4870da0224e048ed82f227d7ba676a94 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.733021 1166 raft_consensus.cc:740] T 4870da0224e048ed82f227d7ba676a94 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.733083 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f93006227cc6489e91f3593e11789b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.733304 1166 consensus_queue.cc:260] T 4870da0224e048ed82f227d7ba676a94 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.740024 1020 tablet_service.cc:1505] Processing CreateTablet for tablet bb5c5523e175406d9f6f9cd2736a2045 (DEFAULT_TABLE table=default.Table_349fa502a7c44fe79baed28f7da89ff9 [id=1bd6b7930f2447659da365b7c06b1640]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.740600 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb5c5523e175406d9f6f9cd2736a2045. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.745302 1163 ts_tablet_manager.cc:1929] T ec7df7b11ce34b8489048e6acdcb4e94 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.745458 1163 log.cc:1199] T ec7df7b11ce34b8489048e6acdcb4e94 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/ec7df7b11ce34b8489048e6acdcb4e94
I20251211 20:52:24.745783 1163 ts_tablet_manager.cc:1950] T ec7df7b11ce34b8489048e6acdcb4e94 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:24.747419 1168 tablet_bootstrap.cc:492] T 353da9106578450987a4f6544df8d7e6 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:24.747709 1168 ts_tablet_manager.cc:1403] T 353da9106578450987a4f6544df8d7e6 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.122s user 0.020s sys 0.004s
I20251211 20:52:24.749122 1168 raft_consensus.cc:359] T 353da9106578450987a4f6544df8d7e6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.749329 1168 raft_consensus.cc:385] T 353da9106578450987a4f6544df8d7e6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.749405 1168 raft_consensus.cc:740] T 353da9106578450987a4f6544df8d7e6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.749606 891 tablet_service.cc:1505] Processing CreateTablet for tablet 605172f066d242f2abea133c08a145ca (DEFAULT_TABLE table=default.Table_40b5190e58a440e9a13d6c9221572914 [id=e2b03132f53845138e4fd9731c2d79bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.749678 1168 consensus_queue.cc:260] T 353da9106578450987a4f6544df8d7e6 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.750173 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 605172f066d242f2abea133c08a145ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.750170 1168 ts_tablet_manager.cc:1434] T 353da9106578450987a4f6544df8d7e6 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:24.752169 1166 ts_tablet_manager.cc:1434] T 4870da0224e048ed82f227d7ba676a94 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.020s user 0.002s sys 0.000s
I20251211 20:52:24.754401 1166 tablet_bootstrap.cc:492] T 9662efc97a53461d888a062463e80ee6 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:24.755182 595 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 4870da0224e048ed82f227d7ba676a94 (table default.Table_5cb009a3a8b24126af4e28df1795d049 [id=28f3ebe9b7a64b5e89404b0407b0e5ae]). Expected version 1 got 0
I20251211 20:52:24.756464 1166 tablet_bootstrap.cc:654] T 9662efc97a53461d888a062463e80ee6 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.758160 593 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 4870da0224e048ed82f227d7ba676a94 (table default.Table_5cb009a3a8b24126af4e28df1795d049 [id=28f3ebe9b7a64b5e89404b0407b0e5ae]). Expected version 1 got 0
I20251211 20:52:24.764120 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 0246f1df99684d0da43d97acbf1182a0
I20251211 20:52:24.767310 1163 tablet_replica.cc:333] T 732c76659c0540bc8b6909b430ffb146 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:24.767380 1340 raft_consensus.cc:493] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:24.767640 1163 raft_consensus.cc:2243] T 732c76659c0540bc8b6909b430ffb146 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.767560 1340 raft_consensus.cc:515] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.767896 1163 raft_consensus.cc:2272] T 732c76659c0540bc8b6909b430ffb146 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.768440 1340 leader_election.cc:290] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:24.769002 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e41f052151624df39a3c83744870c94e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:24.769146 1166 tablet_bootstrap.cc:492] T 9662efc97a53461d888a062463e80ee6 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:24.769316 1166 ts_tablet_manager.cc:1403] T 9662efc97a53461d888a062463e80ee6 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.000s
I20251211 20:52:24.769789 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet ec7df7b11ce34b8489048e6acdcb4e94 (table default.Table_02014bfadb7b434f960168f2db70b838 [id=14a0dbb8486246b09f3e06fc2120cd28]) successfully deleted
I20251211 20:52:24.769867 889 tablet_service.cc:1505] Processing CreateTablet for tablet d30376baf75b4761971f91da3493a142 (DEFAULT_TABLE table=default.Table_9540f76e3f984e92b632bc45a25f4b8c [id=62d2078174bb4b5f952896c6ada6a414]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.769994 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e41f052151624df39a3c83744870c94e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:24.770340 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d30376baf75b4761971f91da3493a142. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.770255 1166 raft_consensus.cc:359] T 9662efc97a53461d888a062463e80ee6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.770457 1166 raft_consensus.cc:385] T 9662efc97a53461d888a062463e80ee6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.770576 1166 raft_consensus.cc:740] T 9662efc97a53461d888a062463e80ee6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
W20251211 20:52:24.770854 982 leader_election.cc:343] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:24.771075 983 leader_election.cc:343] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:24.771183 983 leader_election.cc:304] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:24.771617 1340 raft_consensus.cc:2749] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:24.772380 1163 ts_tablet_manager.cc:1916] T 732c76659c0540bc8b6909b430ffb146 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.772439 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 605172f066d242f2abea133c08a145ca (DEFAULT_TABLE table=default.Table_40b5190e58a440e9a13d6c9221572914 [id=e2b03132f53845138e4fd9731c2d79bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.772866 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 605172f066d242f2abea133c08a145ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.773635 890 tablet_service.cc:1505] Processing CreateTablet for tablet dcdcff8235814b58b3683104a2ff0cbf (DEFAULT_TABLE table=default.Table_0209a7c0449b42e9ac582543b779f5c7 [id=3aa2f2349ee9431fb9bfb0634e5768b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.774083 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcdcff8235814b58b3683104a2ff0cbf. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:24.777712 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d30376baf75b4761971f91da3493a142 consensus state has no leader
I20251211 20:52:24.778249 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet 0246f1df99684d0da43d97acbf1182a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.780185 1166 consensus_queue.cc:260] T 9662efc97a53461d888a062463e80ee6 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.780913 1166 ts_tablet_manager.cc:1434] T 9662efc97a53461d888a062463e80ee6 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.011s user 0.002s sys 0.001s
I20251211 20:52:24.782394 1166 tablet_bootstrap.cc:492] T f92d26bcde3b410aa56bc23a61b1ac35 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:24.784560 1166 tablet_bootstrap.cc:654] T f92d26bcde3b410aa56bc23a61b1ac35 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.784673 589 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 694518e6c1ae4032b06b99e9bde2e5dd (table default.Table_86450313ee0040bb87db3b340566266b [id=d49d5216afba41d68497f42fc72b1bf0]). Expected version 2 got 0
I20251211 20:52:24.784942 589 catalog_manager.cc:5654] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } }
I20251211 20:52:24.785778 589 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet acdb7dddf3e74ce1bc92f232b6180cca (table default.Table_28b427311c9b4699a47c38ebb4d93e01 [id=b68aac5147654707b677d020a2cc37cd]). Expected version 1 got 0
I20251211 20:52:24.783100 878 tablet_service.cc:1552] Processing DeleteTablet for tablet 0246f1df99684d0da43d97acbf1182a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:24.790503 750 tablet_service.cc:1505] Processing CreateTablet for tablet d30376baf75b4761971f91da3493a142 (DEFAULT_TABLE table=default.Table_9540f76e3f984e92b632bc45a25f4b8c [id=62d2078174bb4b5f952896c6ada6a414]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.790953 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d30376baf75b4761971f91da3493a142. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:24.791363 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 0246f1df99684d0da43d97acbf1182a0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0246f1df99684d0da43d97acbf1182a0 already in progress: creating tablet
I20251211 20:52:24.792656 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet 9662efc97a53461d888a062463e80ee6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.797441 1023 tablet_service.cc:1505] Processing CreateTablet for tablet dcdcff8235814b58b3683104a2ff0cbf (DEFAULT_TABLE table=default.Table_0209a7c0449b42e9ac582543b779f5c7 [id=3aa2f2349ee9431fb9bfb0634e5768b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.798084 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcdcff8235814b58b3683104a2ff0cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.798677 1020 tablet_service.cc:1505] Processing CreateTablet for tablet f93006227cc6489e91f3593e11789b54 (DEFAULT_TABLE table=default.Table_4cd9b395cfa94302b298a3df4971d631 [id=42b8ff20f93f4d3fa10fe7ff92dc598d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.799055 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f93006227cc6489e91f3593e11789b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.800473 1166 tablet_bootstrap.cc:492] T f92d26bcde3b410aa56bc23a61b1ac35 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:24.800712 1166 ts_tablet_manager.cc:1403] T f92d26bcde3b410aa56bc23a61b1ac35 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.000s
I20251211 20:52:24.801035 589 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet acdb7dddf3e74ce1bc92f232b6180cca (table default.Table_28b427311c9b4699a47c38ebb4d93e01 [id=b68aac5147654707b677d020a2cc37cd]). Expected version 1 got 0
I20251211 20:52:24.801470 1029 tablet_service.cc:1505] Processing CreateTablet for tablet d30376baf75b4761971f91da3493a142 (DEFAULT_TABLE table=default.Table_9540f76e3f984e92b632bc45a25f4b8c [id=62d2078174bb4b5f952896c6ada6a414]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.801627 1166 raft_consensus.cc:359] T f92d26bcde3b410aa56bc23a61b1ac35 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.801927 1166 raft_consensus.cc:385] T f92d26bcde3b410aa56bc23a61b1ac35 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.802021 1166 raft_consensus.cc:740] T f92d26bcde3b410aa56bc23a61b1ac35 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.802309 1166 consensus_queue.cc:260] T f92d26bcde3b410aa56bc23a61b1ac35 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.802655 738 tablet_service.cc:1505] Processing CreateTablet for tablet dcdcff8235814b58b3683104a2ff0cbf (DEFAULT_TABLE table=default.Table_0209a7c0449b42e9ac582543b779f5c7 [id=3aa2f2349ee9431fb9bfb0634e5768b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.802881 1166 ts_tablet_manager.cc:1434] T f92d26bcde3b410aa56bc23a61b1ac35 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:24.803071 738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcdcff8235814b58b3683104a2ff0cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.803294 1166 tablet_bootstrap.cc:492] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:24.804991 1163 ts_tablet_manager.cc:1929] T 732c76659c0540bc8b6909b430ffb146 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.805158 1163 log.cc:1199] T 732c76659c0540bc8b6909b430ffb146 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/732c76659c0540bc8b6909b430ffb146
I20251211 20:52:24.805603 1163 ts_tablet_manager.cc:1950] T 732c76659c0540bc8b6909b430ffb146 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:24.811722 589 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet f92d26bcde3b410aa56bc23a61b1ac35 (table default.Table_7c42321f3e494dc8835cc2db68e653e1 [id=bee5829ed7d44761a9ae57d3134d38dc]). Expected version 1 got 0
I20251211 20:52:24.811830 1166 tablet_bootstrap.cc:654] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.812826 878 tablet_service.cc:1505] Processing CreateTablet for tablet c85624ad640041c29bb87416e1992a9e (DEFAULT_TABLE table=default.Table_01bf09a9b23a450493cea3c5bf9bdfe2 [id=a6ab64c3e0d94462b716f2d153bc3314]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.813738 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c85624ad640041c29bb87416e1992a9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.814783 1163 tablet_replica.cc:333] T 9662efc97a53461d888a062463e80ee6 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:24.815028 1163 raft_consensus.cc:2243] T 9662efc97a53461d888a062463e80ee6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.815209 1163 raft_consensus.cc:2272] T 9662efc97a53461d888a062463e80ee6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.816707 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_28b427311c9b4699a47c38ebb4d93e01" } modify_external_catalogs: true
I20251211 20:52:24.816771 1163 ts_tablet_manager.cc:1916] T 9662efc97a53461d888a062463e80ee6 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.816895 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_28b427311c9b4699a47c38ebb4d93e01" } modify_external_catalogs: true
I20251211 20:52:24.816314 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_bb77b0fc048e4501a80c73e72e428a52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:24.818655 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bb77b0fc048e4501a80c73e72e428a52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.819900 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4a3a3fe6b0294c939ebc90871cbcb4de" } modify_external_catalogs: true
I20251211 20:52:24.820114 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4a3a3fe6b0294c939ebc90871cbcb4de" } modify_external_catalogs: true
I20251211 20:52:24.821354 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_cec7f350a9d04bd590db8a29fb8ceb6b" } modify_external_catalogs: true
I20251211 20:52:24.821563 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_cec7f350a9d04bd590db8a29fb8ceb6b" } modify_external_catalogs: true
I20251211 20:52:24.822873 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_10dc3bcef378487d8d4ade7100dd7913" } new_table_name: "default.Table_d4afd6ed615a41ac8f6714c8aac27857" modify_external_catalogs: true
I20251211 20:52:24.823654 1166 tablet_bootstrap.cc:492] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:24.823819 1166 ts_tablet_manager.cc:1403] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.003s
I20251211 20:52:24.824182 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 0246f1df99684d0da43d97acbf1182a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.824687 1166 raft_consensus.cc:359] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.825084 744 tablet_service.cc:1505] Processing CreateTablet for tablet c85624ad640041c29bb87416e1992a9e (DEFAULT_TABLE table=default.Table_01bf09a9b23a450493cea3c5bf9bdfe2 [id=a6ab64c3e0d94462b716f2d153bc3314]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.825393 1166 raft_consensus.cc:385] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.825444 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c85624ad640041c29bb87416e1992a9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.825495 1166 raft_consensus.cc:740] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.825716 1166 consensus_queue.cc:260] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.831481 1163 ts_tablet_manager.cc:1929] T 9662efc97a53461d888a062463e80ee6 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.831663 1163 log.cc:1199] T 9662efc97a53461d888a062463e80ee6 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/9662efc97a53461d888a062463e80ee6
I20251211 20:52:24.832011 1163 ts_tablet_manager.cc:1950] T 9662efc97a53461d888a062463e80ee6 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:24.832033 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 353da9106578450987a4f6544df8d7e6 with delete_type TABLET_DATA_DELETED (replaced by tablet d762b4afb5ff447dac3468393826205a) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.832370 738 tablet_service.cc:1552] Processing DeleteTablet for tablet 97a519a8ca7c4f5380a230a70ee84052 with delete_type TABLET_DATA_DELETED (replaced by tablet 27a08a64d92342d08ab4406e8006a97b) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.832696 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d30376baf75b4761971f91da3493a142. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.834553 1166 ts_tablet_manager.cc:1434] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20251211 20:52:24.834880 1166 tablet_bootstrap.cc:492] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:24.835709 591 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet c58241f7e1404a8c8db92c4f9a47d143 (table default.Table_3570af5d8bff402ab3140ae51bae174c [id=90602e87d8b6425295bb04a5fca84ba7]). Expected version 3 got 0
I20251211 20:52:24.838814 591 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet c58241f7e1404a8c8db92c4f9a47d143 (table default.Table_3570af5d8bff402ab3140ae51bae174c [id=90602e87d8b6425295bb04a5fca84ba7]). Expected version 3 got 0
I20251211 20:52:24.841534 1166 tablet_bootstrap.cc:654] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.842528 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 73aa2a3fb2f44bbca870fb7f4cf6087c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.843164 1165 tablet_replica.cc:333] T 353da9106578450987a4f6544df8d7e6 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:24.843447 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_39124c0c70034d00818c18db160416cf" } modify_external_catalogs: true
I20251211 20:52:24.843596 596 master_service.cc:954] ReplaceTablet: received request to replace tablet 19d0aa259ac94f36b7f6a24d11a3d420 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:24.843675 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_39124c0c70034d00818c18db160416cf" } modify_external_catalogs: true
I20251211 20:52:24.844799 592 master_service.cc:954] ReplaceTablet: received request to replace tablet dc40b9826ea84728910b1fd61ce62a25 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:24.846094 1165 raft_consensus.cc:2243] T 353da9106578450987a4f6544df8d7e6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.846364 1165 raft_consensus.cc:2272] T 353da9106578450987a4f6544df8d7e6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.847316 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet acdb7dddf3e74ce1bc92f232b6180cca
I20251211 20:52:24.851451 1166 tablet_bootstrap.cc:492] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:24.851707 1166 ts_tablet_manager.cc:1403] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.002s
I20251211 20:52:24.853045 1166 raft_consensus.cc:359] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.853284 1166 raft_consensus.cc:385] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.853354 1166 raft_consensus.cc:740] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.853577 1166 consensus_queue.cc:260] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.854195 1165 ts_tablet_manager.cc:1916] T 353da9106578450987a4f6544df8d7e6 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.856909 1166 ts_tablet_manager.cc:1434] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251211 20:52:24.857281 1166 tablet_bootstrap.cc:492] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:24.859289 594 master_service.cc:954] ReplaceTablet: received request to replace tablet 034f47d8564f4094a9d76aee2fea3119 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:24.860387 588 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 33cce042f33b4be198d11fd39f5f8838 (table default.Table_fdd4d94139714e3c954d6daf3c82a649 [id=5a7b1e562b084b109569a8733fd470a3]). Expected version 1 got 0
I20251211 20:52:24.863426 588 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 33cce042f33b4be198d11fd39f5f8838 (table default.Table_fdd4d94139714e3c954d6daf3c82a649 [id=5a7b1e562b084b109569a8733fd470a3]). Expected version 1 got 0
I20251211 20:52:24.863900 890 tablet_service.cc:1552] Processing DeleteTablet for tablet acdb7dddf3e74ce1bc92f232b6180cca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:24.864740 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet f19a83b557e24f929b172d926996ae64
I20251211 20:52:24.866243 1166 tablet_bootstrap.cc:654] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.867244 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet dc40b9826ea84728910b1fd61ce62a25
I20251211 20:52:24.867743 592 catalog_manager.cc:6724] ReplaceTablet: tablet dc40b9826ea84728910b1fd61ce62a25 deleted and replaced by tablet bc32a778e5424850b23ccaf74bb9df07
I20251211 20:52:24.868065 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_4358d588f1664e98accbfc761378c61f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:24.868628 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4358d588f1664e98accbfc761378c61f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.869125 889 tablet_service.cc:1552] Processing DeleteTablet for tablet dc40b9826ea84728910b1fd61ce62a25 with delete_type TABLET_DATA_DELETED (replaced by tablet bc32a778e5424850b23ccaf74bb9df07) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:24.869145 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet acdb7dddf3e74ce1bc92f232b6180cca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet acdb7dddf3e74ce1bc92f232b6180cca already in progress: creating tablet
I20251211 20:52:24.869786 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_75c7933bd82449a1a24a7481815ee23b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:24.870299 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_75c7933bd82449a1a24a7481815ee23b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.869760 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_45aae1bb922143a2b0fb90dd4b385f0c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:24.870957 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_45aae1bb922143a2b0fb90dd4b385f0c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.871773 890 tablet_service.cc:1552] Processing DeleteTablet for tablet f19a83b557e24f929b172d926996ae64 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:24.871816 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 19d0aa259ac94f36b7f6a24d11a3d420
I20251211 20:52:24.872242 596 catalog_manager.cc:6724] ReplaceTablet: tablet 19d0aa259ac94f36b7f6a24d11a3d420 deleted and replaced by tablet b9e5ad8030a44e03acfe73124b156ac8
I20251211 20:52:24.873414 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_6f75dc9bf2844fc7afd79d4a5b893811"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:24.874001 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f75dc9bf2844fc7afd79d4a5b893811 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.875367 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_233d5484bd4e4456aed925bc271edfa5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:24.875965 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_233d5484bd4e4456aed925bc271edfa5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.876814 1166 tablet_bootstrap.cc:492] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:24.876971 1166 ts_tablet_manager.cc:1403] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.003s
I20251211 20:52:24.877305 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_227fbf051d5643528315ca00194583bc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:24.877789 1166 raft_consensus.cc:359] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.878003 1166 raft_consensus.cc:385] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20251211 20:52:24.877909 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_227fbf051d5643528315ca00194583bc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.878094 1166 raft_consensus.cc:740] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.879212 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet d30376baf75b4761971f91da3493a142
I20251211 20:52:24.880196 1166 consensus_queue.cc:260] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.881309 1166 ts_tablet_manager.cc:1434] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251211 20:52:24.881655 1166 tablet_bootstrap.cc:492] T ef87e052644c41bbbc54b8dad68ce152 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:24.883608 1166 tablet_bootstrap.cc:654] T ef87e052644c41bbbc54b8dad68ce152 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:24.884948 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dd425601d89148eb95525f796a3abf1e consensus state has no leader
I20251211 20:52:24.885809 889 tablet_service.cc:1552] Processing DeleteTablet for tablet d30376baf75b4761971f91da3493a142 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:24.887864 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet dc40b9826ea84728910b1fd61ce62a25 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dc40b9826ea84728910b1fd61ce62a25 already in progress: creating tablet
I20251211 20:52:24.888186 589 master_service.cc:954] ReplaceTablet: received request to replace tablet 0246f1df99684d0da43d97acbf1182a0 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:24.888413 890 tablet_service.cc:1552] Processing DeleteTablet for tablet 19d0aa259ac94f36b7f6a24d11a3d420 with delete_type TABLET_DATA_DELETED (replaced by tablet b9e5ad8030a44e03acfe73124b156ac8) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:24.888226 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet f19a83b557e24f929b172d926996ae64 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f19a83b557e24f929b172d926996ae64 already in progress: creating tablet
W20251211 20:52:24.889178 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet d30376baf75b4761971f91da3493a142 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d30376baf75b4761971f91da3493a142 already in progress: creating tablet
W20251211 20:52:24.889468 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 19d0aa259ac94f36b7f6a24d11a3d420 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 19d0aa259ac94f36b7f6a24d11a3d420 already in progress: creating tablet
I20251211 20:52:24.890821 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_aa53337337ff420e8030af1bb87dd3f7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:24.891376 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aa53337337ff420e8030af1bb87dd3f7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.898114 741 tablet_service.cc:1505] Processing CreateTablet for tablet 605172f066d242f2abea133c08a145ca (DEFAULT_TABLE table=default.Table_40b5190e58a440e9a13d6c9221572914 [id=e2b03132f53845138e4fd9731c2d79bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.898705 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 605172f066d242f2abea133c08a145ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.903924 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 1951a67964834152afb31f95741d981a
I20251211 20:52:24.816717 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 732c76659c0540bc8b6909b430ffb146 (table default.Table_a2a9717ed78046cab7c00077ac646665 [id=65dab4128d204227947dd4651a5aeb6c]) successfully deleted
W20251211 20:52:24.906713 581 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 90011.3us
W20251211 20:52:24.906991 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 97a519a8ca7c4f5380a230a70ee84052 because the tablet was not found. No further retry: Not found: Tablet not found: 97a519a8ca7c4f5380a230a70ee84052
W20251211 20:52:24.907223 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 0246f1df99684d0da43d97acbf1182a0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0246f1df99684d0da43d97acbf1182a0 already in progress: creating tablet
I20251211 20:52:24.907425 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 9662efc97a53461d888a062463e80ee6 (table default.Table_9d6a8e006ed241819eaecb22446277f5 [id=655f70b9f7e0400aa42179b922af2fa2]) successfully deleted
I20251211 20:52:24.908382 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet acdb7dddf3e74ce1bc92f232b6180cca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.908737 1163 tablet_replica.cc:333] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:24.908947 1163 raft_consensus.cc:2243] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.909166 1163 raft_consensus.cc:2272] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.909333 750 tablet_service.cc:1552] Processing DeleteTablet for tablet f19a83b557e24f929b172d926996ae64 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.909755 1163 ts_tablet_manager.cc:1916] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.909888 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet f19a83b557e24f929b172d926996ae64 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.910390 750 tablet_service.cc:1552] Processing DeleteTablet for tablet dc40b9826ea84728910b1fd61ce62a25 with delete_type TABLET_DATA_DELETED (replaced by tablet bc32a778e5424850b23ccaf74bb9df07) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.910977 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet dc40b9826ea84728910b1fd61ce62a25 with delete_type TABLET_DATA_DELETED (replaced by tablet bc32a778e5424850b23ccaf74bb9df07) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.911474 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 19d0aa259ac94f36b7f6a24d11a3d420 with delete_type TABLET_DATA_DELETED (replaced by tablet b9e5ad8030a44e03acfe73124b156ac8) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.912634 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 19d0aa259ac94f36b7f6a24d11a3d420 with delete_type TABLET_DATA_DELETED (replaced by tablet b9e5ad8030a44e03acfe73124b156ac8) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.913127 750 tablet_service.cc:1552] Processing DeleteTablet for tablet d30376baf75b4761971f91da3493a142 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.913650 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet d30376baf75b4761971f91da3493a142 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.914170 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 1951a67964834152afb31f95741d981a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:24.914693 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 1951a67964834152afb31f95741d981a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:24.914989 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 0246f1df99684d0da43d97acbf1182a0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0246f1df99684d0da43d97acbf1182a0 already in progress: creating tablet
I20251211 20:52:24.916733 1163 ts_tablet_manager.cc:1929] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.916867 1163 log.cc:1199] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/acdb7dddf3e74ce1bc92f232b6180cca
I20251211 20:52:24.917274 1163 ts_tablet_manager.cc:1950] T acdb7dddf3e74ce1bc92f232b6180cca P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:24.918857 1163 tablet_replica.cc:333] T 1951a67964834152afb31f95741d981a P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:24.919083 1163 raft_consensus.cc:2243] T 1951a67964834152afb31f95741d981a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.919277 1163 raft_consensus.cc:2272] T 1951a67964834152afb31f95741d981a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.919622 1168 tablet_bootstrap.cc:492] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:24.920148 1163 ts_tablet_manager.cc:1916] T 1951a67964834152afb31f95741d981a P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.920992 1168 tablet_bootstrap.cc:654] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.921844 741 tablet_service.cc:1552] Processing DeleteTablet for tablet acdb7dddf3e74ce1bc92f232b6180cca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:24.923214 590 master_service.cc:954] ReplaceTablet: received request to replace tablet ec7df7b11ce34b8489048e6acdcb4e94 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:24.923492 890 tablet_service.cc:1505] Processing CreateTablet for tablet 6a2442a4ebc940a281f9ea1a394f6621 (DEFAULT_TABLE table=default.Table_cbe68d84efef4745822f6aef5a723f11 [id=f9c1230a7d71435cac705d4675a9908d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.923942 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a2442a4ebc940a281f9ea1a394f6621. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.927960 889 tablet_service.cc:1505] Processing CreateTablet for tablet 1843dc4548134f29969d46fbc760f5d7 (DEFAULT_TABLE table=default.Table_36ad93a3bb49488b87ef36eb462443b4 [id=6abb187bfd214e91b2e11267065a2841]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.928359 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1843dc4548134f29969d46fbc760f5d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.932168 894 tablet_service.cc:1505] Processing CreateTablet for tablet 804a11c8f7414eb1a6bcff2952be9ae2 (DEFAULT_TABLE table=default.Table_1e40c473960c4fbebfc86c9680481ec1 [id=7f9092a9ef3945a7851d0d9f762e161a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.932636 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 804a11c8f7414eb1a6bcff2952be9ae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.932868 1163 ts_tablet_manager.cc:1929] T 1951a67964834152afb31f95741d981a P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.933025 1163 log.cc:1199] T 1951a67964834152afb31f95741d981a P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/1951a67964834152afb31f95741d981a
I20251211 20:52:24.933365 1163 ts_tablet_manager.cc:1950] T 1951a67964834152afb31f95741d981a P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:24.935964 880 tablet_service.cc:1505] Processing CreateTablet for tablet d5f0ba3ab16240f9822be69f941f5da2 (DEFAULT_TABLE table=default.Table_d73d156809724a399ad27846bfcc460b [id=baf6ca043ec048b894cf9716fad105b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.936386 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5f0ba3ab16240f9822be69f941f5da2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.937264 1165 ts_tablet_manager.cc:1929] T 353da9106578450987a4f6544df8d7e6 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.937379 1165 log.cc:1199] T 353da9106578450987a4f6544df8d7e6 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/353da9106578450987a4f6544df8d7e6
I20251211 20:52:24.937736 1165 ts_tablet_manager.cc:1950] T 353da9106578450987a4f6544df8d7e6 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:24.938714 1165 tablet_replica.cc:333] T 97a519a8ca7c4f5380a230a70ee84052 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:24.938905 1165 raft_consensus.cc:2243] T 97a519a8ca7c4f5380a230a70ee84052 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.939059 1165 raft_consensus.cc:2272] T 97a519a8ca7c4f5380a230a70ee84052 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.941622 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a6b9ed785d804b10b942d36ac7bfd209" } new_table_name: "default.Table_4aad4cf01eff415f86b8446c29977007" modify_external_catalogs: true
I20251211 20:52:24.941756 1165 ts_tablet_manager.cc:1916] T 97a519a8ca7c4f5380a230a70ee84052 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.944550 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ef7b6f5df7224e2ebd3c6ec172e9aff9" } new_table_name: "default.Table_cd52450ec0cc4892a722eef698ddca35" modify_external_catalogs: true
I20251211 20:52:24.946198 588 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8ab2da28cfc84df5b1f6efe450cde2a3" } new_table_name: "default.Table_5d36b67ed10545eb97ac64602292ca7b" modify_external_catalogs: true
I20251211 20:52:24.947515 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_045fee0e26fa49119877f83dbd4b9159"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:24.948104 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_045fee0e26fa49119877f83dbd4b9159 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.949474 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_28b13655ab2b47e68f2bd01d618a2b58"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:24.949958 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_28b13655ab2b47e68f2bd01d618a2b58 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:24.954855 1165 ts_tablet_manager.cc:1929] T 97a519a8ca7c4f5380a230a70ee84052 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.955015 1165 log.cc:1199] T 97a519a8ca7c4f5380a230a70ee84052 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/97a519a8ca7c4f5380a230a70ee84052
I20251211 20:52:24.955366 1165 ts_tablet_manager.cc:1950] T 97a519a8ca7c4f5380a230a70ee84052 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:24.955562 1029 tablet_service.cc:1505] Processing CreateTablet for tablet c85624ad640041c29bb87416e1992a9e (DEFAULT_TABLE table=default.Table_01bf09a9b23a450493cea3c5bf9bdfe2 [id=a6ab64c3e0d94462b716f2d153bc3314]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.956000 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c85624ad640041c29bb87416e1992a9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.959924 1165 tablet_replica.cc:333] T 73aa2a3fb2f44bbca870fb7f4cf6087c P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:24.960139 1165 raft_consensus.cc:2243] T 73aa2a3fb2f44bbca870fb7f4cf6087c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:24.960333 1165 raft_consensus.cc:2272] T 73aa2a3fb2f44bbca870fb7f4cf6087c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.960713 1168 tablet_bootstrap.cc:492] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:24.960930 1168 ts_tablet_manager.cc:1403] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.211s user 0.002s sys 0.001s
I20251211 20:52:24.961838 739 tablet_service.cc:1505] Processing CreateTablet for tablet 6a2442a4ebc940a281f9ea1a394f6621 (DEFAULT_TABLE table=default.Table_cbe68d84efef4745822f6aef5a723f11 [id=f9c1230a7d71435cac705d4675a9908d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.961920 1165 ts_tablet_manager.cc:1916] T 73aa2a3fb2f44bbca870fb7f4cf6087c P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.962800 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a2442a4ebc940a281f9ea1a394f6621. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.963222 1168 raft_consensus.cc:359] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.963451 1168 raft_consensus.cc:385] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.963558 1168 raft_consensus.cc:740] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.963827 1168 consensus_queue.cc:260] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.964861 1168 ts_tablet_manager.cc:1434] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251211 20:52:24.965332 1168 tablet_bootstrap.cc:492] T e69d53ac90a7463aa36ff15dadad7e0b P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:24.967284 1168 tablet_bootstrap.cc:654] T e69d53ac90a7463aa36ff15dadad7e0b P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.972659 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ef1c4ca915474d569c91296213346105" } modify_external_catalogs: true
I20251211 20:52:24.972898 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ef1c4ca915474d569c91296213346105" } modify_external_catalogs: true
I20251211 20:52:24.975950 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 1843dc4548134f29969d46fbc760f5d7 (DEFAULT_TABLE table=default.Table_36ad93a3bb49488b87ef36eb462443b4 [id=6abb187bfd214e91b2e11267065a2841]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.975988 750 tablet_service.cc:1505] Processing CreateTablet for tablet 804a11c8f7414eb1a6bcff2952be9ae2 (DEFAULT_TABLE table=default.Table_1e40c473960c4fbebfc86c9680481ec1 [id=7f9092a9ef3945a7851d0d9f762e161a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.976056 1165 ts_tablet_manager.cc:1929] T 73aa2a3fb2f44bbca870fb7f4cf6087c P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:24.976189 1165 log.cc:1199] T 73aa2a3fb2f44bbca870fb7f4cf6087c P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/73aa2a3fb2f44bbca870fb7f4cf6087c
I20251211 20:52:24.976402 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1843dc4548134f29969d46fbc760f5d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.976434 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 804a11c8f7414eb1a6bcff2952be9ae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.976620 1165 ts_tablet_manager.cc:1950] T 73aa2a3fb2f44bbca870fb7f4cf6087c P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:24.983482 741 tablet_service.cc:1505] Processing CreateTablet for tablet 1843dc4548134f29969d46fbc760f5d7 (DEFAULT_TABLE table=default.Table_36ad93a3bb49488b87ef36eb462443b4 [id=6abb187bfd214e91b2e11267065a2841]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.984302 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 6a2442a4ebc940a281f9ea1a394f6621 (DEFAULT_TABLE table=default.Table_cbe68d84efef4745822f6aef5a723f11 [id=f9c1230a7d71435cac705d4675a9908d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:24.984835 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a2442a4ebc940a281f9ea1a394f6621. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.988255 1165 tablet_replica.cc:333] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:24.988561 1165 raft_consensus.cc:2243] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:24.988900 1168 tablet_bootstrap.cc:492] T e69d53ac90a7463aa36ff15dadad7e0b P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:24.989073 1168 ts_tablet_manager.cc:1403] T e69d53ac90a7463aa36ff15dadad7e0b P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.024s user 0.009s sys 0.002s
I20251211 20:52:24.989256 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1843dc4548134f29969d46fbc760f5d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:24.989916 1168 raft_consensus.cc:359] T e69d53ac90a7463aa36ff15dadad7e0b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.990124 1168 raft_consensus.cc:385] T e69d53ac90a7463aa36ff15dadad7e0b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:24.990195 1168 raft_consensus.cc:740] T e69d53ac90a7463aa36ff15dadad7e0b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:24.988898 1165 pending_rounds.cc:70] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919: Trying to abort 1 pending ops.
I20251211 20:52:24.990413 1165 pending_rounds.cc:77] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432067929022464 op_type: NO_OP noop_request { }
I20251211 20:52:24.990530 1165 raft_consensus.cc:2889] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:24.990648 1165 raft_consensus.cc:2272] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:24.990952 1168 consensus_queue.cc:260] T e69d53ac90a7463aa36ff15dadad7e0b P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:24.991459 1168 ts_tablet_manager.cc:1434] T e69d53ac90a7463aa36ff15dadad7e0b P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:24.992148 1165 ts_tablet_manager.cc:1916] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:24.992398 1168 tablet_bootstrap.cc:492] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:24.995152 1168 tablet_bootstrap.cc:654] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:24.999989 1165 ts_tablet_manager.cc:1929] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:25.000121 1165 log.cc:1199] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/acdb7dddf3e74ce1bc92f232b6180cca
I20251211 20:52:25.000267 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c91a79fb73a549ea9656555dc2e1c780" } new_table_name: "default.Table_56733fc3806a4af7bf52401c572bb186" modify_external_catalogs: true
I20251211 20:52:25.000535 1165 ts_tablet_manager.cc:1950] T acdb7dddf3e74ce1bc92f232b6180cca P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
W20251211 20:52:25.001737 591 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 822abda3c85848728205b71847ef7029 has no consensus state
W20251211 20:52:25.003867 588 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8308ad46551447439a248d20ed1e5922 consensus state has no leader
W20251211 20:52:25.004206 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 340730a17a834a77bbc20b95aba98e2c has no consensus state
I20251211 20:52:25.006178 1340 raft_consensus.cc:493] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:25.006405 1340 raft_consensus.cc:515] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.007097 1340 leader_election.cc:290] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:25.008082 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a5eeb56709f046d9be92ac600ebc330e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:25.008704 982 leader_election.cc:343] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.010351 750 tablet_service.cc:1505] Processing CreateTablet for tablet d5f0ba3ab16240f9822be69f941f5da2 (DEFAULT_TABLE table=default.Table_d73d156809724a399ad27846bfcc460b [id=baf6ca043ec048b894cf9716fad105b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.010712 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 804a11c8f7414eb1a6bcff2952be9ae2 (DEFAULT_TABLE table=default.Table_1e40c473960c4fbebfc86c9680481ec1 [id=7f9092a9ef3945a7851d0d9f762e161a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.010847 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5f0ba3ab16240f9822be69f941f5da2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.011109 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 804a11c8f7414eb1a6bcff2952be9ae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.020422 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 353da9106578450987a4f6544df8d7e6 (table default.Table_a0c412985adf4382a7ea28c77654fd6c [id=4a74b9a60cbb4198b55cee630b2f4bef]) successfully deleted
I20251211 20:52:25.020709 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 97a519a8ca7c4f5380a230a70ee84052 (table default.Table_c91a79fb73a549ea9656555dc2e1c780 [id=383392d835da4c50848068d83261cecc]) successfully deleted
I20251211 20:52:25.020889 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 73aa2a3fb2f44bbca870fb7f4cf6087c (table default.Table_3fca4e8f1b6a427d8d016e27f88ea1ce [id=4f93b23edd4548aeb6bcb8f92878907d]) successfully deleted
W20251211 20:52:25.021088 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet f19a83b557e24f929b172d926996ae64 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f19a83b557e24f929b172d926996ae64 already in progress: creating tablet
W20251211 20:52:25.021411 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet dc40b9826ea84728910b1fd61ce62a25 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dc40b9826ea84728910b1fd61ce62a25 already in progress: creating tablet
W20251211 20:52:25.021615 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 19d0aa259ac94f36b7f6a24d11a3d420 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 19d0aa259ac94f36b7f6a24d11a3d420 already in progress: creating tablet
W20251211 20:52:25.021853 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet d30376baf75b4761971f91da3493a142 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d30376baf75b4761971f91da3493a142 already in progress: creating tablet
W20251211 20:52:25.022050 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 1951a67964834152afb31f95741d981a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1951a67964834152afb31f95741d981a already in progress: creating tablet
I20251211 20:52:25.022236 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet acdb7dddf3e74ce1bc92f232b6180cca (table default.Table_28b427311c9b4699a47c38ebb4d93e01 [id=b68aac5147654707b677d020a2cc37cd]) successfully deleted
I20251211 20:52:25.022522 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet acdb7dddf3e74ce1bc92f232b6180cca (table default.Table_28b427311c9b4699a47c38ebb4d93e01 [id=b68aac5147654707b677d020a2cc37cd]) successfully deleted
W20251211 20:52:25.022720 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet f19a83b557e24f929b172d926996ae64 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f19a83b557e24f929b172d926996ae64 already in progress: creating tablet
W20251211 20:52:25.022879 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet dc40b9826ea84728910b1fd61ce62a25 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dc40b9826ea84728910b1fd61ce62a25 already in progress: creating tablet
W20251211 20:52:25.023103 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 19d0aa259ac94f36b7f6a24d11a3d420 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 19d0aa259ac94f36b7f6a24d11a3d420 already in progress: creating tablet
W20251211 20:52:25.023274 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet d30376baf75b4761971f91da3493a142 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d30376baf75b4761971f91da3493a142 already in progress: creating tablet
I20251211 20:52:25.023445 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 1951a67964834152afb31f95741d981a (table default.Table_39124c0c70034d00818c18db160416cf [id=016a0bf9d2704b74af4e64d2f9a23878]) successfully deleted
I20251211 20:52:25.028344 1020 tablet_service.cc:1505] Processing CreateTablet for tablet d5f0ba3ab16240f9822be69f941f5da2 (DEFAULT_TABLE table=default.Table_d73d156809724a399ad27846bfcc460b [id=baf6ca043ec048b894cf9716fad105b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.028834 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5f0ba3ab16240f9822be69f941f5da2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.036630 596 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet b9e5ad8030a44e03acfe73124b156ac8
I20251211 20:52:25.042663 878 tablet_service.cc:1552] Processing DeleteTablet for tablet 1951a67964834152afb31f95741d981a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:25.042994 1182 raft_consensus.cc:493] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:25.043150 1182 raft_consensus.cc:515] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.043872 1182 leader_election.cc:290] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:25.046180 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 27a08a64d92342d08ab4406e8006a97b consensus state has no leader
I20251211 20:52:25.047538 1166 tablet_bootstrap.cc:492] T ef87e052644c41bbbc54b8dad68ce152 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:25.047762 1166 ts_tablet_manager.cc:1403] T ef87e052644c41bbbc54b8dad68ce152 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.166s user 0.018s sys 0.042s
I20251211 20:52:25.048658 1166 raft_consensus.cc:359] T ef87e052644c41bbbc54b8dad68ce152 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.048897 1166 raft_consensus.cc:385] T ef87e052644c41bbbc54b8dad68ce152 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20251211 20:52:25.048889 710 leader_election.cc:343] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 7f086c7c56fe4abb8df7d3c0978cce6e
I20251211 20:52:25.048975 1166 raft_consensus.cc:740] T ef87e052644c41bbbc54b8dad68ce152 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.049249 1166 consensus_queue.cc:260] T ef87e052644c41bbbc54b8dad68ce152 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.049986 1166 ts_tablet_manager.cc:1434] T ef87e052644c41bbbc54b8dad68ce152 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:25.050354 1166 tablet_bootstrap.cc:492] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:25.052327 1166 tablet_bootstrap.cc:654] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.056438 1166 tablet_bootstrap.cc:492] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:25.056608 1166 ts_tablet_manager.cc:1403] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20251211 20:52:25.057475 1166 raft_consensus.cc:359] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.057703 1166 raft_consensus.cc:385] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.057816 1166 raft_consensus.cc:740] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.060405 1166 consensus_queue.cc:260] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.061482 1166 ts_tablet_manager.cc:1434] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251211 20:52:25.061825 1166 tablet_bootstrap.cc:492] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:25.064356 1166 tablet_bootstrap.cc:654] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.067492 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a5eeb56709f046d9be92ac600ebc330e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:25.067839 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_5b8aa62e6fdf46089c4c8ae541856fbe" } modify_external_catalogs: true
I20251211 20:52:25.067914 1236 raft_consensus.cc:493] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:25.068039 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_5b8aa62e6fdf46089c4c8ae541856fbe" } modify_external_catalogs: true
W20251211 20:52:25.068673 983 leader_election.cc:343] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.068820 983 leader_election.cc:304] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:25.069027 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 1951a67964834152afb31f95741d981a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1951a67964834152afb31f95741d981a already in progress: creating tablet
I20251211 20:52:25.069592 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0209a7c0449b42e9ac582543b779f5c7" } modify_external_catalogs: true
I20251211 20:52:25.069768 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0209a7c0449b42e9ac582543b779f5c7" } modify_external_catalogs: true
I20251211 20:52:25.069911 1340 raft_consensus.cc:2749] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:25.070334 1236 raft_consensus.cc:515] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.070628 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_cc48404bbbf24a7caae7641a8d42c247"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:25.071194 1236 leader_election.cc:290] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
W20251211 20:52:25.071290 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cc48404bbbf24a7caae7641a8d42c247 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:25.071617 1166 tablet_bootstrap.cc:492] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:25.071796 1166 ts_tablet_manager.cc:1403] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.004s
I20251211 20:52:25.072602 1166 raft_consensus.cc:359] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.072800 1166 raft_consensus.cc:385] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.072862 1166 raft_consensus.cc:740] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.074343 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01ada50100bc48e39d1bc67082da750e" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:25.074548 777 raft_consensus.cc:2468] T 01ada50100bc48e39d1bc67082da750e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 0.
I20251211 20:52:25.075068 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01ada50100bc48e39d1bc67082da750e" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:25.075294 1049 raft_consensus.cc:2468] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 0.
I20251211 20:52:25.075266 1166 consensus_queue.cc:260] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.076399 1166 ts_tablet_manager.cc:1434] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20251211 20:52:25.077188 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7f086c7c56fe4abb8df7d3c0978cce6e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:25.077596 1166 tablet_bootstrap.cc:492] T 4ddf1bd1e18f4187bf050d6f7767b722 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:25.078104 711 leader_election.cc:343] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.078261 711 leader_election.cc:304] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:25.078687 1182 raft_consensus.cc:2749] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:25.081231 1166 tablet_bootstrap.cc:654] T 4ddf1bd1e18f4187bf050d6f7767b722 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.084757 1397 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.62:41123 every 8 connection(s)
I20251211 20:52:25.085064 1204 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.62:41123
I20251211 20:52:25.087106 1204 server_base.cc:1179] Dumped server information to /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-0/data/info.pb
I20251211 20:52:25.087345 847 leader_election.cc:304] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41; no voters:
I20251211 20:52:25.088587 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_028df988934346debc0f8c62b4740309" } modify_external_catalogs: true
I20251211 20:52:25.088830 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_028df988934346debc0f8c62b4740309" } modify_external_catalogs: true
I20251211 20:52:25.089435 1166 tablet_bootstrap.cc:492] T 4ddf1bd1e18f4187bf050d6f7767b722 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:25.089607 1166 ts_tablet_manager.cc:1403] T 4ddf1bd1e18f4187bf050d6f7767b722 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.002s
I20251211 20:52:25.090462 1166 raft_consensus.cc:359] T 4ddf1bd1e18f4187bf050d6f7767b722 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.090653 1166 raft_consensus.cc:385] T 4ddf1bd1e18f4187bf050d6f7767b722 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.090747 1166 raft_consensus.cc:740] T 4ddf1bd1e18f4187bf050d6f7767b722 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.090984 1166 consensus_queue.cc:260] T 4ddf1bd1e18f4187bf050d6f7767b722 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.095039 416 external_mini_cluster.cc:1428] Started /tmp/dist-test-task5TWsQc/build/asan/bin/kudu as pid 1204
I20251211 20:52:25.096135 588 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_6f75dc9bf2844fc7afd79d4a5b893811" } new_table_name: "default.Table_1bf756c0a806499fbfd10745831a653f" modify_external_catalogs: true
I20251211 20:52:25.097115 1340 raft_consensus.cc:493] T 4870da0224e048ed82f227d7ba676a94 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:25.097294 1340 raft_consensus.cc:515] T 4870da0224e048ed82f227d7ba676a94 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.097908 1340 leader_election.cc:290] T 4870da0224e048ed82f227d7ba676a94 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:25.098135 591 master_service.cc:954] ReplaceTablet: received request to replace tablet dcdcff8235814b58b3683104a2ff0cbf from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:25.098771 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4870da0224e048ed82f227d7ba676a94" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:25.099366 982 leader_election.cc:343] T 4870da0224e048ed82f227d7ba676a94 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.102226 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet ef87e052644c41bbbc54b8dad68ce152 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:25.102545 1163 tablet_replica.cc:333] T ef87e052644c41bbbc54b8dad68ce152 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:25.102751 1163 raft_consensus.cc:2243] T ef87e052644c41bbbc54b8dad68ce152 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.102928 1163 raft_consensus.cc:2272] T ef87e052644c41bbbc54b8dad68ce152 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.103605 1160 tablet_bootstrap.cc:492] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:25.103632 1163 ts_tablet_manager.cc:1916] T ef87e052644c41bbbc54b8dad68ce152 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.103533 1236 raft_consensus.cc:2804] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:25.105008 1160 ts_tablet_manager.cc:1403] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.486s user 0.156s sys 0.216s
I20251211 20:52:25.106309 1160 raft_consensus.cc:359] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.106505 1236 raft_consensus.cc:493] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:25.106527 1166 ts_tablet_manager.cc:1434] T 4ddf1bd1e18f4187bf050d6f7767b722 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.017s user 0.001s sys 0.001s
I20251211 20:52:25.106532 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4870da0224e048ed82f227d7ba676a94" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:25.106845 1166 tablet_bootstrap.cc:492] T 4ef506349e5b434db7a78809dc44014e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:25.107081 1160 raft_consensus.cc:385] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.107185 1160 raft_consensus.cc:740] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
W20251211 20:52:25.107183 983 leader_election.cc:343] T 4870da0224e048ed82f227d7ba676a94 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.107323 983 leader_election.cc:304] T 4870da0224e048ed82f227d7ba676a94 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:25.108148 1340 raft_consensus.cc:2749] T 4870da0224e048ed82f227d7ba676a94 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:25.108407 1160 consensus_queue.cc:260] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.108975 1236 raft_consensus.cc:3060] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:25.111032 1163 ts_tablet_manager.cc:1929] T ef87e052644c41bbbc54b8dad68ce152 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:25.111166 1163 log.cc:1199] T ef87e052644c41bbbc54b8dad68ce152 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/ef87e052644c41bbbc54b8dad68ce152
I20251211 20:52:25.112933 1236 raft_consensus.cc:515] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.111565 1163 ts_tablet_manager.cc:1950] T ef87e052644c41bbbc54b8dad68ce152 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:25.116665 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet ef87e052644c41bbbc54b8dad68ce152 (table default.Table_9b96425fc6f34e15ab398770cb57bc98 [id=f8384ab62c6749b7b9c458e8e90d3c02]) successfully deleted
W20251211 20:52:25.121277 588 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fb2d30147cd74d1893783be1e1694d1f has no consensus state
I20251211 20:52:25.121464 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01ada50100bc48e39d1bc67082da750e" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:25.121717 777 raft_consensus.cc:3060] T 01ada50100bc48e39d1bc67082da750e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:25.123101 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 7f086c7c56fe4abb8df7d3c0978cce6e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:25.124876 1165 tablet_replica.cc:333] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:25.125093 1165 raft_consensus.cc:2243] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.125257 1165 raft_consensus.cc:2272] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.126303 1165 ts_tablet_manager.cc:1916] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.127641 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4cd9b395cfa94302b298a3df4971d631" } modify_external_catalogs: true
I20251211 20:52:25.127866 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4cd9b395cfa94302b298a3df4971d631" } modify_external_catalogs: true
I20251211 20:52:25.130018 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01ada50100bc48e39d1bc67082da750e" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:25.130277 1049 raft_consensus.cc:3060] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:25.131070 1166 tablet_bootstrap.cc:654] T 4ef506349e5b434db7a78809dc44014e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.133104 1182 raft_consensus.cc:493] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:25.133332 777 raft_consensus.cc:2468] T 01ada50100bc48e39d1bc67082da750e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 1.
I20251211 20:52:25.133550 1049 raft_consensus.cc:2468] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 1.
I20251211 20:52:25.133275 1182 raft_consensus.cc:515] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.133999 847 leader_election.cc:304] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41; no voters:
I20251211 20:52:25.134441 1182 leader_election.cc:290] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:25.137568 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "694518e6c1ae4032b06b99e9bde2e5dd" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:25.137884 1049 raft_consensus.cc:2468] T 694518e6c1ae4032b06b99e9bde2e5dd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:25.138329 710 leader_election.cc:304] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:25.138595 1182 raft_consensus.cc:2804] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:25.138710 1182 raft_consensus.cc:493] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:25.138803 1182 raft_consensus.cc:3060] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:25.142009 1236 leader_election.cc:290] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Requested vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:25.142484 1182 raft_consensus.cc:515] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.143491 1182 leader_election.cc:290] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:25.146449 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_33d44c934d124375827a2987a3ec3d3d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:25.147459 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_33d44c934d124375827a2987a3ec3d3d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:25.149180 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_45e60d0af4e64e4c9913708346ea407b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:25.149796 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_45e60d0af4e64e4c9913708346ea407b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:25.150667 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "694518e6c1ae4032b06b99e9bde2e5dd" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:25.150880 1049 raft_consensus.cc:3060] T 694518e6c1ae4032b06b99e9bde2e5dd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:25.152593 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_9eae16b0b8f141589dc1420df5181cb2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:25.153223 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9eae16b0b8f141589dc1420df5181cb2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:25.149796 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_87e2d779c1a74a57a8eb93d7163c1660"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:25.155059 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_87e2d779c1a74a57a8eb93d7163c1660 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:25.157934 741 tablet_service.cc:1552] Processing DeleteTablet for tablet e69d53ac90a7463aa36ff15dadad7e0b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:25.159046 1049 raft_consensus.cc:2468] T 694518e6c1ae4032b06b99e9bde2e5dd P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:25.159658 710 leader_election.cc:304] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:25.160422 1236 raft_consensus.cc:2804] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:25.160656 1236 raft_consensus.cc:697] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41 [term 1 LEADER]: Becoming Leader. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Running, Role: LEADER
I20251211 20:52:25.160990 1236 consensus_queue.cc:237] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.163480 1182 raft_consensus.cc:2804] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:25.165123 1182 raft_consensus.cc:697] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:25.165515 1182 consensus_queue.cc:237] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.168774 1165 ts_tablet_manager.cc:1929] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:25.168924 1165 log.cc:1199] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/7f086c7c56fe4abb8df7d3c0978cce6e
I20251211 20:52:25.169317 1165 ts_tablet_manager.cc:1950] T 7f086c7c56fe4abb8df7d3c0978cce6e P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:25.170204 1165 tablet_replica.cc:333] T e69d53ac90a7463aa36ff15dadad7e0b P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:25.170367 1165 raft_consensus.cc:2243] T e69d53ac90a7463aa36ff15dadad7e0b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.170524 1165 raft_consensus.cc:2272] T e69d53ac90a7463aa36ff15dadad7e0b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.171969 1160 ts_tablet_manager.cc:1434] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.067s user 0.001s sys 0.004s
I20251211 20:52:25.172183 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 7f086c7c56fe4abb8df7d3c0978cce6e (table default.Table_e5a162d6463e4ff5a4edaaae5a4d03cb [id=872ff521abd740ee924f22e9c0ff8585]) successfully deleted
I20251211 20:52:25.172335 1160 tablet_bootstrap.cc:492] T 5ed0a0141051462eb45ca3a66d41d009 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:25.174381 1160 tablet_bootstrap.cc:654] T 5ed0a0141051462eb45ca3a66d41d009 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.177850 1168 tablet_bootstrap.cc:492] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:25.178012 1168 ts_tablet_manager.cc:1403] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.186s user 0.013s sys 0.047s
I20251211 20:52:25.178833 1168 raft_consensus.cc:359] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.179020 1168 raft_consensus.cc:385] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.179107 1168 raft_consensus.cc:740] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.181288 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_6669f84cc5f04b8fb74c1d226054f2b8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:25.181908 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6669f84cc5f04b8fb74c1d226054f2b8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:25.182291 1165 ts_tablet_manager.cc:1916] T e69d53ac90a7463aa36ff15dadad7e0b P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.182629 1168 consensus_queue.cc:260] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.183179 1168 ts_tablet_manager.cc:1434] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251211 20:52:25.183966 589 master_service.cc:954] ReplaceTablet: received request to replace tablet e41f052151624df39a3c83744870c94e from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:25.184178 822 heartbeater.cc:344] Connected to a master server at 127.0.104.62:41123
I20251211 20:52:25.184684 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_51f7abf29f7247eb8a83e8b9ef1d26c5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:25.185222 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_51f7abf29f7247eb8a83e8b9ef1d26c5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:25.185458 1168 tablet_bootstrap.cc:492] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:25.187180 1168 tablet_bootstrap.cc:654] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.187203 595 master_service.cc:954] ReplaceTablet: received request to replace tablet c4b76a7e84d344268567c551236ce772 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:25.194867 1398 sys_catalog.cc:263] Verifying existing consensus state
I20251211 20:52:25.198074 1362 raft_consensus.cc:493] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:25.198239 1362 raft_consensus.cc:515] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.198690 594 master_service.cc:954] ReplaceTablet: received request to replace tablet 2433068ec179490dac3e91d046a42b70 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:25.199034 1362 leader_election.cc:290] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:25.200630 710 leader_election.cc:343] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 1951a67964834152afb31f95741d981a
I20251211 20:52:25.201503 1160 tablet_bootstrap.cc:492] T 5ed0a0141051462eb45ca3a66d41d009 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:25.201694 1160 ts_tablet_manager.cc:1403] T 5ed0a0141051462eb45ca3a66d41d009 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.029s user 0.005s sys 0.001s
I20251211 20:52:25.202522 1160 raft_consensus.cc:359] T 5ed0a0141051462eb45ca3a66d41d009 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:25.202713 1160 raft_consensus.cc:385] T 5ed0a0141051462eb45ca3a66d41d009 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.202814 1160 raft_consensus.cc:740] T 5ed0a0141051462eb45ca3a66d41d009 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.203068 1160 consensus_queue.cc:260] T 5ed0a0141051462eb45ca3a66d41d009 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:25.203219 741 tablet_service.cc:1505] Processing CreateTablet for tablet ef4fada332cf4b5c9da26ba5f908b4c6 (DEFAULT_TABLE table=default.Table_ddb3c1574c924ada922b623f332a3613 [id=ef058448a4ff49f59d0e943e11ed64b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.203552 1160 ts_tablet_manager.cc:1434] T 5ed0a0141051462eb45ca3a66d41d009 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:25.203825 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef4fada332cf4b5c9da26ba5f908b4c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.203935 1160 tablet_bootstrap.cc:492] T 353da9106578450987a4f6544df8d7e6 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:25.205418 1398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a: Bootstrap starting.
I20251211 20:52:25.205670 1160 tablet_bootstrap.cc:654] T 353da9106578450987a4f6544df8d7e6 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.208118 1168 tablet_bootstrap.cc:492] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:25.208391 889 tablet_service.cc:1505] Processing CreateTablet for tablet ef4fada332cf4b5c9da26ba5f908b4c6 (DEFAULT_TABLE table=default.Table_ddb3c1574c924ada922b623f332a3613 [id=ef058448a4ff49f59d0e943e11ed64b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.208858 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef4fada332cf4b5c9da26ba5f908b4c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.208298 1168 ts_tablet_manager.cc:1403] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.000s
I20251211 20:52:25.211026 1168 raft_consensus.cc:359] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.212117 1094 heartbeater.cc:344] Connected to a master server at 127.0.104.62:41123
I20251211 20:52:25.211233 1168 raft_consensus.cc:385] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.215302 1168 raft_consensus.cc:740] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.218488 1168 consensus_queue.cc:260] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.211887 1165 ts_tablet_manager.cc:1929] T e69d53ac90a7463aa36ff15dadad7e0b P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:25.219871 1165 log.cc:1199] T e69d53ac90a7463aa36ff15dadad7e0b P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/e69d53ac90a7463aa36ff15dadad7e0b
I20251211 20:52:25.220243 1165 ts_tablet_manager.cc:1950] T e69d53ac90a7463aa36ff15dadad7e0b P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:25.221016 1168 ts_tablet_manager.cc:1434] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20251211 20:52:25.221335 1168 tablet_bootstrap.cc:492] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:25.221990 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 6a206b37e43247b5babe3113eab2c9fb
I20251211 20:52:25.223886 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet bd8fd6c977204c90b558827fc0b15f43
I20251211 20:52:25.224428 1340 raft_consensus.cc:493] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:25.224596 1340 raft_consensus.cc:515] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.224606 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_287197890af94935922681dd3ac7d185"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:25.225191 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_287197890af94935922681dd3ac7d185 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:25.225216 1340 leader_election.cc:290] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:25.226042 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "33cce042f33b4be198d11fd39f5f8838" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:25.226568 982 leader_election.cc:343] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.227030 594 master_service.cc:954] ReplaceTablet: received request to replace tablet 8dff13e22899452c8d4f2731b8dac7d5 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:25.227523 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_45e60d0af4e64e4c9913708346ea407b" } new_table_name: "default.Table_502b11716c454b5dae33c715fd69dc1f" modify_external_catalogs: true
I20251211 20:52:25.228400 1168 tablet_bootstrap.cc:654] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.231170 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_01bf09a9b23a450493cea3c5bf9bdfe2" } new_table_name: "default.Table_8f7e347b9ac64793b1073dfa955ebe21" modify_external_catalogs: true
I20251211 20:52:25.233673 750 tablet_service.cc:1552] Processing DeleteTablet for tablet bd8fd6c977204c90b558827fc0b15f43 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:25.234010 1160 tablet_bootstrap.cc:492] T 353da9106578450987a4f6544df8d7e6 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:25.234193 1160 ts_tablet_manager.cc:1403] T 353da9106578450987a4f6544df8d7e6 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.000s
I20251211 20:52:25.234642 739 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a206b37e43247b5babe3113eab2c9fb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:25.234659 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ddf1bd1e18f4187bf050d6f7767b722 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:25.234932 1163 tablet_replica.cc:333] T 4ddf1bd1e18f4187bf050d6f7767b722 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:25.235005 1160 raft_consensus.cc:359] T 353da9106578450987a4f6544df8d7e6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.235221 1163 raft_consensus.cc:2243] T 4ddf1bd1e18f4187bf050d6f7767b722 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.235390 1163 raft_consensus.cc:2272] T 4ddf1bd1e18f4187bf050d6f7767b722 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.235661 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet bd8fd6c977204c90b558827fc0b15f43 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:25.236080 1160 raft_consensus.cc:385] T 353da9106578450987a4f6544df8d7e6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.236183 1160 raft_consensus.cc:740] T 353da9106578450987a4f6544df8d7e6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.236443 1163 ts_tablet_manager.cc:1916] T 4ddf1bd1e18f4187bf050d6f7767b722 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.236413 1160 consensus_queue.cc:260] T 353da9106578450987a4f6544df8d7e6 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.237079 1160 ts_tablet_manager.cc:1434] T 353da9106578450987a4f6544df8d7e6 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20251211 20:52:25.237375 1160 tablet_bootstrap.cc:492] T 7f086c7c56fe4abb8df7d3c0978cce6e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:25.237493 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a206b37e43247b5babe3113eab2c9fb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:25.239009 880 tablet_service.cc:1552] Processing DeleteTablet for tablet bd8fd6c977204c90b558827fc0b15f43 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:25.239408 1160 tablet_bootstrap.cc:654] T 7f086c7c56fe4abb8df7d3c0978cce6e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.240538 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a206b37e43247b5babe3113eab2c9fb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:25.241315 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet bd8fd6c977204c90b558827fc0b15f43 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd8fd6c977204c90b558827fc0b15f43 already in progress: creating tablet
I20251211 20:52:25.244110 1163 ts_tablet_manager.cc:1929] T 4ddf1bd1e18f4187bf050d6f7767b722 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:25.242817 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet e69d53ac90a7463aa36ff15dadad7e0b (table default.Table_2645aa306b5c44f6bad5e2e67a91ebec [id=0ec7cf10eb8a47aca499b141913b38cf]) successfully deleted
I20251211 20:52:25.244272 1163 log.cc:1199] T 4ddf1bd1e18f4187bf050d6f7767b722 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/4ddf1bd1e18f4187bf050d6f7767b722
I20251211 20:52:25.244568 1163 ts_tablet_manager.cc:1950] T 4ddf1bd1e18f4187bf050d6f7767b722 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
W20251211 20:52:25.244565 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet bd8fd6c977204c90b558827fc0b15f43 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd8fd6c977204c90b558827fc0b15f43 already in progress: creating tablet
W20251211 20:52:25.244905 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 6a206b37e43247b5babe3113eab2c9fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a206b37e43247b5babe3113eab2c9fb already in progress: creating tablet
W20251211 20:52:25.245261 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 6a206b37e43247b5babe3113eab2c9fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a206b37e43247b5babe3113eab2c9fb already in progress: creating tablet
I20251211 20:52:25.252683 889 tablet_service.cc:1505] Processing CreateTablet for tablet 8139acb0ae324d0f973ce877edc1cd04 (DEFAULT_TABLE table=default.Table_d282d0b8f49e4f44998af55480ad3cb8 [id=959101c0af144f52a5680b0c9d4cbd20]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.253293 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8139acb0ae324d0f973ce877edc1cd04. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.254774 1398 log.cc:826] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a: Log is configured to *not* fsync() on all Append() calls
I20251211 20:52:25.245559 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 4ddf1bd1e18f4187bf050d6f7767b722 (table default.Table_307027e1160747b98c2fb2ea31eb8745 [id=3ade2e0cb4924ec38113bf929b5f554b]) successfully deleted
I20251211 20:52:25.261749 750 tablet_service.cc:1505] Processing CreateTablet for tablet f7f52d2547534ef99255c5d3f59410c8 (DEFAULT_TABLE table=default.Table_b3e8496af4d340218ce858c81796a29c [id=ecb6d90c41224c2d8f771a78922eaf05]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.262224 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7f52d2547534ef99255c5d3f59410c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.263897 1160 tablet_bootstrap.cc:492] T 7f086c7c56fe4abb8df7d3c0978cce6e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:25.264060 1160 ts_tablet_manager.cc:1403] T 7f086c7c56fe4abb8df7d3c0978cce6e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.005s
I20251211 20:52:25.264952 1160 raft_consensus.cc:359] T 7f086c7c56fe4abb8df7d3c0978cce6e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.265154 1160 raft_consensus.cc:385] T 7f086c7c56fe4abb8df7d3c0978cce6e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.265246 1160 raft_consensus.cc:740] T 7f086c7c56fe4abb8df7d3c0978cce6e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.265514 1160 consensus_queue.cc:260] T 7f086c7c56fe4abb8df7d3c0978cce6e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.266155 1160 ts_tablet_manager.cc:1434] T 7f086c7c56fe4abb8df7d3c0978cce6e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251211 20:52:25.266479 1160 tablet_bootstrap.cc:492] T 049cdecf12304211adbc1c1008dba989 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:25.268826 1023 tablet_service.cc:1505] Processing CreateTablet for tablet f7f52d2547534ef99255c5d3f59410c8 (DEFAULT_TABLE table=default.Table_b3e8496af4d340218ce858c81796a29c [id=ecb6d90c41224c2d8f771a78922eaf05]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.269306 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7f52d2547534ef99255c5d3f59410c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.276347 739 tablet_service.cc:1505] Processing CreateTablet for tablet 8139acb0ae324d0f973ce877edc1cd04 (DEFAULT_TABLE table=default.Table_d282d0b8f49e4f44998af55480ad3cb8 [id=959101c0af144f52a5680b0c9d4cbd20]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.276623 1160 tablet_bootstrap.cc:654] T 049cdecf12304211adbc1c1008dba989 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.276974 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8139acb0ae324d0f973ce877edc1cd04. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.280025 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 8139acb0ae324d0f973ce877edc1cd04 (DEFAULT_TABLE table=default.Table_d282d0b8f49e4f44998af55480ad3cb8 [id=959101c0af144f52a5680b0c9d4cbd20]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.280480 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8139acb0ae324d0f973ce877edc1cd04. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.282569 738 tablet_service.cc:1505] Processing CreateTablet for tablet 05385f20e88c46108e09a78814811209 (DEFAULT_TABLE table=default.Table_638eea7803484ee9965a1058a3526993 [id=c06795252c09401b99e58c702574ac05]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.282967 738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05385f20e88c46108e09a78814811209. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.285970 1160 tablet_bootstrap.cc:492] T 049cdecf12304211adbc1c1008dba989 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:25.286172 1160 ts_tablet_manager.cc:1403] T 049cdecf12304211adbc1c1008dba989 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.004s
I20251211 20:52:25.287703 1160 raft_consensus.cc:359] T 049cdecf12304211adbc1c1008dba989 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.287930 1160 raft_consensus.cc:385] T 049cdecf12304211adbc1c1008dba989 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.288009 1160 raft_consensus.cc:740] T 049cdecf12304211adbc1c1008dba989 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.288260 1160 consensus_queue.cc:260] T 049cdecf12304211adbc1c1008dba989 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.288844 1160 ts_tablet_manager.cc:1434] T 049cdecf12304211adbc1c1008dba989 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20251211 20:52:25.289186 1160 tablet_bootstrap.cc:492] T 1951a67964834152afb31f95741d981a P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:25.290213 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 55c40b2464f8444ca6c9fa0400f7f7f2 (DEFAULT_TABLE table=default.Table_069bb219e15e43be995ff011e2289a84 [id=bc0b945e86cf43fdb99d56658ca1576c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.290423 1398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251211 20:52:25.290880 1160 tablet_bootstrap.cc:654] T 1951a67964834152afb31f95741d981a P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.291168 1398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a: Bootstrap complete.
I20251211 20:52:25.291085 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55c40b2464f8444ca6c9fa0400f7f7f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.291976 750 tablet_service.cc:1505] Processing CreateTablet for tablet 55c40b2464f8444ca6c9fa0400f7f7f2 (DEFAULT_TABLE table=default.Table_069bb219e15e43be995ff011e2289a84 [id=bc0b945e86cf43fdb99d56658ca1576c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.292459 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55c40b2464f8444ca6c9fa0400f7f7f2. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:25.303709 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet bd8fd6c977204c90b558827fc0b15f43 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd8fd6c977204c90b558827fc0b15f43 already in progress: creating tablet
W20251211 20:52:25.304107 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 6a206b37e43247b5babe3113eab2c9fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a206b37e43247b5babe3113eab2c9fb already in progress: creating tablet
I20251211 20:52:25.304613 738 tablet_service.cc:1505] Processing CreateTablet for tablet 340730a17a834a77bbc20b95aba98e2c (DEFAULT_TABLE table=default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.305079 738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 340730a17a834a77bbc20b95aba98e2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.305473 1166 tablet_bootstrap.cc:492] T 4ef506349e5b434db7a78809dc44014e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:25.305660 1166 ts_tablet_manager.cc:1403] T 4ef506349e5b434db7a78809dc44014e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.199s user 0.024s sys 0.021s
I20251211 20:52:25.308514 1166 raft_consensus.cc:359] T 4ef506349e5b434db7a78809dc44014e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.308763 1166 raft_consensus.cc:385] T 4ef506349e5b434db7a78809dc44014e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.308843 1166 raft_consensus.cc:740] T 4ef506349e5b434db7a78809dc44014e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.309157 1166 consensus_queue.cc:260] T 4ef506349e5b434db7a78809dc44014e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.309231 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 05385f20e88c46108e09a78814811209 (DEFAULT_TABLE table=default.Table_638eea7803484ee9965a1058a3526993 [id=c06795252c09401b99e58c702574ac05]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.309887 1166 ts_tablet_manager.cc:1434] T 4ef506349e5b434db7a78809dc44014e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251211 20:52:25.310802 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05385f20e88c46108e09a78814811209. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.311010 1166 tablet_bootstrap.cc:492] T 2433068ec179490dac3e91d046a42b70 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:25.311355 1094 heartbeater.cc:344] Connected to a master server at 127.0.104.62:41123
I20251211 20:52:25.314443 1166 tablet_bootstrap.cc:654] T 2433068ec179490dac3e91d046a42b70 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.315172 1160 tablet_bootstrap.cc:492] T 1951a67964834152afb31f95741d981a P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:25.315330 1160 ts_tablet_manager.cc:1403] T 1951a67964834152afb31f95741d981a P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.000s
I20251211 20:52:25.314445 1020 tablet_service.cc:1505] Processing CreateTablet for tablet ef4fada332cf4b5c9da26ba5f908b4c6 (DEFAULT_TABLE table=default.Table_ddb3c1574c924ada922b623f332a3613 [id=ef058448a4ff49f59d0e943e11ed64b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.315960 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef4fada332cf4b5c9da26ba5f908b4c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.316143 1160 raft_consensus.cc:359] T 1951a67964834152afb31f95741d981a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.316356 1160 raft_consensus.cc:385] T 1951a67964834152afb31f95741d981a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.316457 1160 raft_consensus.cc:740] T 1951a67964834152afb31f95741d981a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.316661 1160 consensus_queue.cc:260] T 1951a67964834152afb31f95741d981a P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.317315 1160 ts_tablet_manager.cc:1434] T 1951a67964834152afb31f95741d981a P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:25.317608 1160 tablet_bootstrap.cc:492] T acdb7dddf3e74ce1bc92f232b6180cca P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:25.320832 1160 tablet_bootstrap.cc:654] T acdb7dddf3e74ce1bc92f232b6180cca P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.322395 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "33cce042f33b4be198d11fd39f5f8838" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:25.324261 880 tablet_service.cc:1505] Processing CreateTablet for tablet f7f52d2547534ef99255c5d3f59410c8 (DEFAULT_TABLE table=default.Table_b3e8496af4d340218ce858c81796a29c [id=ecb6d90c41224c2d8f771a78922eaf05]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.324656 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7f52d2547534ef99255c5d3f59410c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.324085 1398 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:25.327778 1362 raft_consensus.cc:493] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:25.327797 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "694518e6c1ae4032b06b99e9bde2e5dd" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:25.327950 1362 raft_consensus.cc:515] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.328378 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "694518e6c1ae4032b06b99e9bde2e5dd" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:25.328634 1362 leader_election.cc:290] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:25.329763 710 leader_election.cc:343] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 049cdecf12304211adbc1c1008dba989
I20251211 20:52:25.330186 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1951a67964834152afb31f95741d981a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:25.330395 913 raft_consensus.cc:2468] T 1951a67964834152afb31f95741d981a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
W20251211 20:52:25.330771 983 leader_election.cc:343] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.330910 1160 tablet_bootstrap.cc:492] T acdb7dddf3e74ce1bc92f232b6180cca P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:25.330906 983 leader_election.cc:304] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:25.331081 1160 ts_tablet_manager.cc:1403] T acdb7dddf3e74ce1bc92f232b6180cca P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.001s
I20251211 20:52:25.331317 1340 raft_consensus.cc:2749] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:25.331741 711 leader_election.cc:343] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.331872 1160 raft_consensus.cc:359] T acdb7dddf3e74ce1bc92f232b6180cca P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.332060 1160 raft_consensus.cc:385] T acdb7dddf3e74ce1bc92f232b6180cca P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.332054 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "049cdecf12304211adbc1c1008dba989" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:25.332109 711 leader_election.cc:343] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.332167 1160 raft_consensus.cc:740] T acdb7dddf3e74ce1bc92f232b6180cca P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.332262 913 raft_consensus.cc:2468] T 049cdecf12304211adbc1c1008dba989 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:25.332383 1160 consensus_queue.cc:260] T acdb7dddf3e74ce1bc92f232b6180cca P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.332403 711 leader_election.cc:304] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters: e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:25.332805 711 leader_election.cc:304] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters: e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:25.332844 1160 ts_tablet_manager.cc:1434] T acdb7dddf3e74ce1bc92f232b6180cca P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251211 20:52:25.333130 1160 tablet_bootstrap.cc:492] T f4db14d082d34aef857e106992bb8950 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:25.333132 1362 raft_consensus.cc:2804] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:25.333220 1362 raft_consensus.cc:493] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:25.333317 1362 raft_consensus.cc:3060] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:25.334750 1160 tablet_bootstrap.cc:654] T f4db14d082d34aef857e106992bb8950 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.336092 1362 raft_consensus.cc:515] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
W20251211 20:52:25.337185 583 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 5fd2a6cf80a04f20aef8ff9c84bc88db (table default.Table_15a4631cb9ae4a8d892505a404f973f1 [id=10fd0368198c447c95b692b1f3e6a023], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5fd2a6cf80a04f20aef8ff9c84bc88db consensus state has no leader
W20251211 20:52:25.337790 710 leader_election.cc:343] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 1951a67964834152afb31f95741d981a
I20251211 20:52:25.338002 959 heartbeater.cc:344] Connected to a master server at 127.0.104.62:41123
I20251211 20:52:25.340749 1362 leader_election.cc:290] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:25.340958 1362 raft_consensus.cc:2804] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:25.341063 1362 raft_consensus.cc:493] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:25.341143 1362 raft_consensus.cc:3060] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:25.341944 1398 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1830b0c58cfe495989f3d911dda0409a, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.342624 1398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:25.345039 1340 raft_consensus.cc:493] T f92d26bcde3b410aa56bc23a61b1ac35 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:25.344887 1422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } } }
I20251211 20:52:25.345186 1422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [sys.catalog]: This master's current role is: FOLLOWER
I20251211 20:52:25.345229 1340 raft_consensus.cc:515] T f92d26bcde3b410aa56bc23a61b1ac35 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.345912 1340 leader_election.cc:290] T f92d26bcde3b410aa56bc23a61b1ac35 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:25.346329 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f92d26bcde3b410aa56bc23a61b1ac35" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:25.346475 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f92d26bcde3b410aa56bc23a61b1ac35" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:25.346804 983 leader_election.cc:343] T f92d26bcde3b410aa56bc23a61b1ac35 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:25.346990 982 leader_election.cc:343] T f92d26bcde3b410aa56bc23a61b1ac35 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.347098 982 leader_election.cc:304] T f92d26bcde3b410aa56bc23a61b1ac35 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:25.347373 1340 raft_consensus.cc:2749] T f92d26bcde3b410aa56bc23a61b1ac35 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:25.348171 1398 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [sys.catalog]: configured and running, proceeding with master startup.
I20251211 20:52:25.350170 1362 raft_consensus.cc:515] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.350817 1362 leader_election.cc:290] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:25.351872 710 leader_election.cc:343] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 049cdecf12304211adbc1c1008dba989
I20251211 20:52:25.368568 880 tablet_service.cc:1505] Processing CreateTablet for tablet 55c40b2464f8444ca6c9fa0400f7f7f2 (DEFAULT_TABLE table=default.Table_069bb219e15e43be995ff011e2289a84 [id=bc0b945e86cf43fdb99d56658ca1576c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.369167 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55c40b2464f8444ca6c9fa0400f7f7f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.371877 1340 raft_consensus.cc:493] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:25.372114 1340 raft_consensus.cc:515] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.372812 1340 leader_election.cc:290] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:25.373538 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4d98967714184fe6a3e38dd4ca7979dc" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:25.373673 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4d98967714184fe6a3e38dd4ca7979dc" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:25.374207 982 leader_election.cc:343] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:25.374539 983 leader_election.cc:343] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.374647 983 leader_election.cc:304] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:25.375123 1340 raft_consensus.cc:2749] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:25.378609 711 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 45816us
I20251211 20:52:25.379446 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1951a67964834152afb31f95741d981a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:25.379707 913 raft_consensus.cc:3060] T 1951a67964834152afb31f95741d981a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:25.382520 913 raft_consensus.cc:2468] T 1951a67964834152afb31f95741d981a P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:25.384609 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "049cdecf12304211adbc1c1008dba989" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:25.384907 913 raft_consensus.cc:3060] T 049cdecf12304211adbc1c1008dba989 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:25.388227 913 raft_consensus.cc:2468] T 049cdecf12304211adbc1c1008dba989 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:25.389434 711 leader_election.cc:304] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters: e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:25.390213 711 leader_election.cc:304] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters: e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:25.390712 1362 raft_consensus.cc:2804] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:25.391022 1362 raft_consensus.cc:697] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:25.391351 1362 consensus_queue.cc:237] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.393435 889 tablet_service.cc:1505] Processing CreateTablet for tablet 05385f20e88c46108e09a78814811209 (DEFAULT_TABLE table=default.Table_638eea7803484ee9965a1058a3526993 [id=c06795252c09401b99e58c702574ac05]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.393694 894 tablet_service.cc:1505] Processing CreateTablet for tablet 340730a17a834a77bbc20b95aba98e2c (DEFAULT_TABLE table=default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.393904 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05385f20e88c46108e09a78814811209. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.394137 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 340730a17a834a77bbc20b95aba98e2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.406320 1362 raft_consensus.cc:2804] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:25.406587 1362 raft_consensus.cc:697] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:25.406893 1362 consensus_queue.cc:237] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.432934 1160 tablet_bootstrap.cc:492] T f4db14d082d34aef857e106992bb8950 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:25.433127 1160 ts_tablet_manager.cc:1403] T f4db14d082d34aef857e106992bb8950 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.100s user 0.005s sys 0.027s
I20251211 20:52:25.433924 1160 raft_consensus.cc:359] T f4db14d082d34aef857e106992bb8950 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.434127 1160 raft_consensus.cc:385] T f4db14d082d34aef857e106992bb8950 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.434212 1160 raft_consensus.cc:740] T f4db14d082d34aef857e106992bb8950 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.434294 1352 raft_consensus.cc:1217] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.294] Dedup: 1.5->[1.6-1.294]
I20251211 20:52:25.434429 1160 consensus_queue.cc:260] T f4db14d082d34aef857e106992bb8950 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.434909 1160 ts_tablet_manager.cc:1434] T f4db14d082d34aef857e106992bb8950 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:25.435774 1166 tablet_bootstrap.cc:492] T 2433068ec179490dac3e91d046a42b70 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:25.435941 1166 ts_tablet_manager.cc:1403] T 2433068ec179490dac3e91d046a42b70 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.125s user 0.005s sys 0.033s
I20251211 20:52:25.436266 1168 tablet_bootstrap.cc:492] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:25.436443 1168 ts_tablet_manager.cc:1403] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.215s user 0.041s sys 0.045s
I20251211 20:52:25.436731 1166 raft_consensus.cc:359] T 2433068ec179490dac3e91d046a42b70 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.436908 1166 raft_consensus.cc:385] T 2433068ec179490dac3e91d046a42b70 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.437004 1166 raft_consensus.cc:740] T 2433068ec179490dac3e91d046a42b70 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.437235 1166 consensus_queue.cc:260] T 2433068ec179490dac3e91d046a42b70 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.437254 1168 raft_consensus.cc:359] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.437415 1168 raft_consensus.cc:385] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.437520 1168 raft_consensus.cc:740] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.437780 1168 consensus_queue.cc:260] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.438007 1166 ts_tablet_manager.cc:1434] T 2433068ec179490dac3e91d046a42b70 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:25.438257 1168 ts_tablet_manager.cc:1434] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:25.438290 1166 tablet_bootstrap.cc:492] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:25.438539 1168 tablet_bootstrap.cc:492] T 732c76659c0540bc8b6909b430ffb146 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:25.440088 1166 tablet_bootstrap.cc:654] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.442577 1160 tablet_bootstrap.cc:492] T af3ca9ac988e458cb7068c10c006d587 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:25.446046 1160 tablet_bootstrap.cc:654] T af3ca9ac988e458cb7068c10c006d587 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.446419 592 catalog_manager.cc:5654] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2). New cstate: current_term: 1 leader_uuid: "eb0f1933d9a74fccaa4618047fc05c41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:25.446866 592 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 01ada50100bc48e39d1bc67082da750e (table default.Table_18d19a317a9c4656b4c764df4a64dbea [id=a24d8c58b9d84cdea2b278ae9c6e0ba5]). Expected version 2 got 0
I20251211 20:52:25.449935 590 catalog_manager.cc:5654] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } }
I20251211 20:52:25.450832 1160 tablet_bootstrap.cc:492] T af3ca9ac988e458cb7068c10c006d587 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:25.451035 1160 ts_tablet_manager.cc:1403] T af3ca9ac988e458cb7068c10c006d587 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.003s
I20251211 20:52:25.451874 1160 raft_consensus.cc:359] T af3ca9ac988e458cb7068c10c006d587 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.452080 1160 raft_consensus.cc:385] T af3ca9ac988e458cb7068c10c006d587 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.452157 1160 raft_consensus.cc:740] T af3ca9ac988e458cb7068c10c006d587 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.452533 1340 raft_consensus.cc:493] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:25.452728 1340 raft_consensus.cc:515] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.453368 1340 leader_election.cc:290] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:25.454046 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d19975cc513e414b80f478a6c5eb09de" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:25.454756 982 leader_election.cc:343] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.455260 590 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 694518e6c1ae4032b06b99e9bde2e5dd (table default.Table_86450313ee0040bb87db3b340566266b [id=d49d5216afba41d68497f42fc72b1bf0]). Expected version 2 got 0
I20251211 20:52:25.455435 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d19975cc513e414b80f478a6c5eb09de" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:25.455960 983 leader_election.cc:343] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.455929 1160 consensus_queue.cc:260] T af3ca9ac988e458cb7068c10c006d587 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.456092 983 leader_election.cc:304] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:25.456568 1340 raft_consensus.cc:2749] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:25.457597 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ef506349e5b434db7a78809dc44014e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:25.458191 1163 tablet_replica.cc:333] T 4ef506349e5b434db7a78809dc44014e P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:25.458376 1163 raft_consensus.cc:2243] T 4ef506349e5b434db7a78809dc44014e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.458536 1163 raft_consensus.cc:2272] T 4ef506349e5b434db7a78809dc44014e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.459342 1163 ts_tablet_manager.cc:1916] T 4ef506349e5b434db7a78809dc44014e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.460134 1340 raft_consensus.cc:493] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:25.460327 1340 raft_consensus.cc:515] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.460947 1027 tablet_service.cc:1505] Processing CreateTablet for tablet bc32a778e5424850b23ccaf74bb9df07 (DEFAULT_TABLE table=default.Table_880a8408f1fb4d68a3f9d30b26acffa5 [id=db38368266194271a02ad265fd991439]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.461377 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc32a778e5424850b23ccaf74bb9df07. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.461463 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "33e1d8a576cd41b1a34778b1458df396" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:25.462033 982 leader_election.cc:343] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.460948 1340 leader_election.cc:290] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:25.462975 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "33e1d8a576cd41b1a34778b1458df396" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:25.463483 983 leader_election.cc:343] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.463618 983 leader_election.cc:304] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:25.464169 1340 raft_consensus.cc:2749] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:25.464812 738 tablet_service.cc:1505] Processing CreateTablet for tablet bc32a778e5424850b23ccaf74bb9df07 (DEFAULT_TABLE table=default.Table_880a8408f1fb4d68a3f9d30b26acffa5 [id=db38368266194271a02ad265fd991439]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.465281 738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc32a778e5424850b23ccaf74bb9df07. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.466166 1163 ts_tablet_manager.cc:1929] T 4ef506349e5b434db7a78809dc44014e P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:25.466341 1163 log.cc:1199] T 4ef506349e5b434db7a78809dc44014e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/4ef506349e5b434db7a78809dc44014e
I20251211 20:52:25.466756 1163 ts_tablet_manager.cc:1950] T 4ef506349e5b434db7a78809dc44014e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:25.467911 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 4ef506349e5b434db7a78809dc44014e (table default.Table_fd4577173dc44659b95db8e463e0131c [id=143faf67e1204e7bb2015f06392e48a6]) successfully deleted
I20251211 20:52:25.470512 1160 ts_tablet_manager.cc:1434] T af3ca9ac988e458cb7068c10c006d587 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.019s user 0.000s sys 0.003s
I20251211 20:52:25.471939 1160 tablet_bootstrap.cc:492] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:25.477705 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 340730a17a834a77bbc20b95aba98e2c (DEFAULT_TABLE table=default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.477918 1160 tablet_bootstrap.cc:654] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.479542 880 tablet_service.cc:1505] Processing CreateTablet for tablet bc32a778e5424850b23ccaf74bb9df07 (DEFAULT_TABLE table=default.Table_880a8408f1fb4d68a3f9d30b26acffa5 [id=db38368266194271a02ad265fd991439]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.480126 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc32a778e5424850b23ccaf74bb9df07. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.470038 1168 tablet_bootstrap.cc:654] T 732c76659c0540bc8b6909b430ffb146 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.480962 1166 tablet_bootstrap.cc:492] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:25.490751 1166 ts_tablet_manager.cc:1403] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.053s user 0.000s sys 0.008s
I20251211 20:52:25.491016 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 340730a17a834a77bbc20b95aba98e2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.491707 1166 raft_consensus.cc:359] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.491921 1166 raft_consensus.cc:385] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.492002 1166 raft_consensus.cc:740] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.492281 1166 consensus_queue.cc:260] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.492858 1166 ts_tablet_manager.cc:1434] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251211 20:52:25.493647 1160 tablet_bootstrap.cc:492] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:25.493816 1160 ts_tablet_manager.cc:1403] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.006s
I20251211 20:52:25.494633 1160 raft_consensus.cc:359] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:25.494825 1160 raft_consensus.cc:385] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.494930 1160 raft_consensus.cc:740] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.495167 1160 consensus_queue.cc:260] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:25.495694 1166 tablet_bootstrap.cc:492] T 8dff13e22899452c8d4f2731b8dac7d5 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:25.497385 1166 tablet_bootstrap.cc:654] T 8dff13e22899452c8d4f2731b8dac7d5 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.497741 1168 tablet_bootstrap.cc:492] T 732c76659c0540bc8b6909b430ffb146 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:25.497917 1168 ts_tablet_manager.cc:1403] T 732c76659c0540bc8b6909b430ffb146 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.059s user 0.009s sys 0.000s
I20251211 20:52:25.498682 1168 raft_consensus.cc:359] T 732c76659c0540bc8b6909b430ffb146 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.498867 1168 raft_consensus.cc:385] T 732c76659c0540bc8b6909b430ffb146 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.498944 1168 raft_consensus.cc:740] T 732c76659c0540bc8b6909b430ffb146 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.499230 1166 tablet_bootstrap.cc:492] T 8dff13e22899452c8d4f2731b8dac7d5 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:25.499200 1168 consensus_queue.cc:260] T 732c76659c0540bc8b6909b430ffb146 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.499378 1166 ts_tablet_manager.cc:1403] T 8dff13e22899452c8d4f2731b8dac7d5 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251211 20:52:25.499789 1168 ts_tablet_manager.cc:1434] T 732c76659c0540bc8b6909b430ffb146 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:25.500140 1168 tablet_bootstrap.cc:492] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:25.501996 1160 ts_tablet_manager.cc:1434] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20251211 20:52:25.502292 1160 tablet_bootstrap.cc:492] T 73aa2a3fb2f44bbca870fb7f4cf6087c P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:25.502380 1168 tablet_bootstrap.cc:654] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.503863 1160 tablet_bootstrap.cc:654] T 73aa2a3fb2f44bbca870fb7f4cf6087c P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.504557 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet c4b76a7e84d344268567c551236ce772
I20251211 20:52:25.505055 595 catalog_manager.cc:6724] ReplaceTablet: tablet c4b76a7e84d344268567c551236ce772 deleted and replaced by tablet 68be21a25c6743f5b86d518d98fbc35e
I20251211 20:52:25.507799 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet e41f052151624df39a3c83744870c94e
I20251211 20:52:25.508195 589 catalog_manager.cc:6724] ReplaceTablet: tablet e41f052151624df39a3c83744870c94e deleted and replaced by tablet b33fd6c238e049b382d8ec9da910fe92
I20251211 20:52:25.508519 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_8b8ed719985f42128e56bc26105e7a9c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:25.509111 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8b8ed719985f42128e56bc26105e7a9c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:25.509198 1160 tablet_bootstrap.cc:492] T 73aa2a3fb2f44bbca870fb7f4cf6087c P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:25.509334 1160 ts_tablet_manager.cc:1403] T 73aa2a3fb2f44bbca870fb7f4cf6087c P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20251211 20:52:25.510077 1160 raft_consensus.cc:359] T 73aa2a3fb2f44bbca870fb7f4cf6087c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.510267 1160 raft_consensus.cc:385] T 73aa2a3fb2f44bbca870fb7f4cf6087c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.510322 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet f93006227cc6489e91f3593e11789b54
I20251211 20:52:25.510339 1160 raft_consensus.cc:740] T 73aa2a3fb2f44bbca870fb7f4cf6087c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.510563 1160 consensus_queue.cc:260] T 73aa2a3fb2f44bbca870fb7f4cf6087c P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.510933 588 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9eae16b0b8f141589dc1420df5181cb2" } new_table_name: "default.Table_a66a87006b684bbd87431e2aad969dcf" modify_external_catalogs: true
I20251211 20:52:25.512377 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f9014306b36e47b98c8e4c6070ea4a37" } modify_external_catalogs: true
I20251211 20:52:25.512530 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f9014306b36e47b98c8e4c6070ea4a37" } modify_external_catalogs: true
I20251211 20:52:25.513623 1160 ts_tablet_manager.cc:1434] T 73aa2a3fb2f44bbca870fb7f4cf6087c P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
W20251211 20:52:25.513767 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c85624ad640041c29bb87416e1992a9e consensus state has no leader
I20251211 20:52:25.513963 1160 tablet_bootstrap.cc:492] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:25.514153 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_bb77b0fc048e4501a80c73e72e428a52" } new_table_name: "default.Table_300858c884e34abe9becdb6113469673" modify_external_catalogs: true
I20251211 20:52:25.515607 1160 tablet_bootstrap.cc:654] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.507282 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_254d66657cb14d0f8bbf713f794806f6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:25.516754 1168 tablet_bootstrap.cc:492] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:25.516844 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet dcdcff8235814b58b3683104a2ff0cbf
I20251211 20:52:25.516920 1168 ts_tablet_manager.cc:1403] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.000s
I20251211 20:52:25.517756 1168 raft_consensus.cc:359] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:25.517849 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_1e40c473960c4fbebfc86c9680481ec1" } new_table_name: "default.Table_415c09e71243460aaa9acfd57742bcd7" modify_external_catalogs: true
I20251211 20:52:25.517946 1168 raft_consensus.cc:385] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.518028 1168 raft_consensus.cc:740] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.518237 1168 consensus_queue.cc:260] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:25.518718 1168 ts_tablet_manager.cc:1434] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:25.519143 1404 raft_consensus.cc:493] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:25.519287 1404 raft_consensus.cc:515] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:25.519964 1168 tablet_bootstrap.cc:492] T e41f052151624df39a3c83744870c94e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:25.520113 1404 leader_election.cc:290] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:25.520925 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ec7df7b11ce34b8489048e6acdcb4e94" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:25.521481 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_254d66657cb14d0f8bbf713f794806f6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:25.521662 1168 tablet_bootstrap.cc:654] T e41f052151624df39a3c83744870c94e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:25.522727 847 leader_election.cc:343] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.523447 1160 tablet_bootstrap.cc:492] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:25.523617 1160 ts_tablet_manager.cc:1403] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20251211 20:52:25.524377 1160 raft_consensus.cc:359] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.524560 1160 raft_consensus.cc:385] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.524657 1160 raft_consensus.cc:740] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.525818 822 heartbeater.cc:344] Connected to a master server at 127.0.104.62:41123
I20251211 20:52:25.527041 1160 consensus_queue.cc:260] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.527174 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_2a8eb0e48b1e42d59c215333429cac46"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:25.527611 1160 ts_tablet_manager.cc:1434] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
W20251211 20:52:25.527860 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2a8eb0e48b1e42d59c215333429cac46 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:25.529002 1160 tablet_bootstrap.cc:492] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:25.529577 959 heartbeater.cc:344] Connected to a master server at 127.0.104.62:41123
I20251211 20:52:25.536403 1160 tablet_bootstrap.cc:654] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:25.536676 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8229b704e7464b30b7d2902a938ce33b has no consensus state
I20251211 20:52:25.538251 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_60c74dd24cd34456b535dc8445e13383"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:25.538949 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_60c74dd24cd34456b535dc8445e13383 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:25.543162 744 tablet_service.cc:1552] Processing DeleteTablet for tablet dcdcff8235814b58b3683104a2ff0cbf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:25.543990 739 tablet_service.cc:1552] Processing DeleteTablet for tablet e41f052151624df39a3c83744870c94e with delete_type TABLET_DATA_DELETED (replaced by tablet b33fd6c238e049b382d8ec9da910fe92) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:25.544008 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet dcdcff8235814b58b3683104a2ff0cbf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dcdcff8235814b58b3683104a2ff0cbf already in progress: creating tablet
W20251211 20:52:25.544734 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet e41f052151624df39a3c83744870c94e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e41f052151624df39a3c83744870c94e already in progress: creating tablet
I20251211 20:52:25.544795 741 tablet_service.cc:1552] Processing DeleteTablet for tablet f93006227cc6489e91f3593e11789b54 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:25.545483 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet f93006227cc6489e91f3593e11789b54 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f93006227cc6489e91f3593e11789b54 already in progress: creating tablet
I20251211 20:52:25.545645 750 tablet_service.cc:1552] Processing DeleteTablet for tablet c4b76a7e84d344268567c551236ce772 with delete_type TABLET_DATA_DELETED (replaced by tablet 68be21a25c6743f5b86d518d98fbc35e) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:25.546286 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet c4b76a7e84d344268567c551236ce772 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c4b76a7e84d344268567c551236ce772 already in progress: creating tablet
I20251211 20:52:25.546303 1362 raft_consensus.cc:493] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:25.546442 1362 raft_consensus.cc:515] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.547113 1362 leader_election.cc:290] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:25.547782 750 tablet_service.cc:1505] Processing CreateTablet for tablet d762b4afb5ff447dac3468393826205a (DEFAULT_TABLE table=default.Table_a0c412985adf4382a7ea28c77654fd6c [id=4a74b9a60cbb4198b55cee630b2f4bef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.547974 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "af3ca9ac988e458cb7068c10c006d587" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:25.548712 913 raft_consensus.cc:2468] T af3ca9ac988e458cb7068c10c006d587 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:25.549211 1362 raft_consensus.cc:493] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:25.549352 741 tablet_service.cc:1505] Processing CreateTablet for tablet 3c578d81e8dd41aaa0f707c4e3fed225 (DEFAULT_TABLE table=default.Table_bb77b0fc048e4501a80c73e72e428a52 [id=c5be59a2c3934b3789641ceeb2d4d6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.549389 1362 raft_consensus.cc:515] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:25.550046 1362 leader_election.cc:290] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
W20251211 20:52:25.550216 582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.61:40749, user_credentials={real_user=slave}} blocked reactor thread for 50431.4us
I20251211 20:52:25.550237 739 tablet_service.cc:1505] Processing CreateTablet for tablet 40722aef4e194ff094d271a079bda9d9 (DEFAULT_TABLE table=default.Table_b7c88854437f4dfd9998ccb8246e6d0d [id=3309205faabf4a42acadc43766e4f57f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.550772 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d762b4afb5ff447dac3468393826205a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.550841 889 tablet_service.cc:1552] Processing DeleteTablet for tablet c4b76a7e84d344268567c551236ce772 with delete_type TABLET_DATA_DELETED (replaced by tablet 68be21a25c6743f5b86d518d98fbc35e) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:25.551379 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8a064789f89c424ca698e52333d73eea" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:25.551594 889 tablet_service.cc:1552] Processing DeleteTablet for tablet e41f052151624df39a3c83744870c94e with delete_type TABLET_DATA_DELETED (replaced by tablet b33fd6c238e049b382d8ec9da910fe92) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:25.551965 711 leader_election.cc:304] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters:
I20251211 20:52:25.552217 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40722aef4e194ff094d271a079bda9d9. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:25.552522 711 leader_election.cc:343] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.552896 1362 raft_consensus.cc:2804] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:25.552994 1362 raft_consensus.cc:493] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:25.553007 889 tablet_service.cc:1552] Processing DeleteTablet for tablet f93006227cc6489e91f3593e11789b54 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:25.553095 1362 raft_consensus.cc:3060] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:25.554472 889 tablet_service.cc:1552] Processing DeleteTablet for tablet dcdcff8235814b58b3683104a2ff0cbf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:25.556104 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c578d81e8dd41aaa0f707c4e3fed225. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.556411 1362 raft_consensus.cc:515] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.556542 889 tablet_service.cc:1505] Processing CreateTablet for tablet d762b4afb5ff447dac3468393826205a (DEFAULT_TABLE table=default.Table_a0c412985adf4382a7ea28c77654fd6c [id=4a74b9a60cbb4198b55cee630b2f4bef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.556990 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d762b4afb5ff447dac3468393826205a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.557619 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "af3ca9ac988e458cb7068c10c006d587" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:25.557837 913 raft_consensus.cc:3060] T af3ca9ac988e458cb7068c10c006d587 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:25.560721 913 raft_consensus.cc:2468] T af3ca9ac988e458cb7068c10c006d587 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:25.561475 711 leader_election.cc:304] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters:
I20251211 20:52:25.561872 1420 raft_consensus.cc:2804] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:25.562055 1420 raft_consensus.cc:697] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:25.563023 1420 consensus_queue.cc:237] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.564895 1362 leader_election.cc:290] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:25.565373 894 tablet_service.cc:1505] Processing CreateTablet for tablet 40722aef4e194ff094d271a079bda9d9 (DEFAULT_TABLE table=default.Table_b7c88854437f4dfd9998ccb8246e6d0d [id=3309205faabf4a42acadc43766e4f57f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.565866 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40722aef4e194ff094d271a079bda9d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.568922 1160 tablet_bootstrap.cc:492] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:25.569136 1160 ts_tablet_manager.cc:1403] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.040s user 0.003s sys 0.003s
I20251211 20:52:25.569998 1160 raft_consensus.cc:359] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.575026 890 tablet_service.cc:1505] Processing CreateTablet for tablet 9f8aa28c1c9b4fc58dfc128952c180ba (DEFAULT_TABLE table=default.Table_03555bd3f502404a928ffdf4ef934e49 [id=0327e686e7ed4475a3e8cda8721a3a3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.575503 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f8aa28c1c9b4fc58dfc128952c180ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.582772 1160 raft_consensus.cc:385] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.583105 1160 raft_consensus.cc:740] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.583521 1160 consensus_queue.cc:260] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.586488 1160 ts_tablet_manager.cc:1434] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.017s user 0.003s sys 0.000s
I20251211 20:52:25.588102 1160 tablet_bootstrap.cc:492] T 33e1d8a576cd41b1a34778b1458df396 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:25.587757 889 tablet_service.cc:1505] Processing CreateTablet for tablet 3c578d81e8dd41aaa0f707c4e3fed225 (DEFAULT_TABLE table=default.Table_bb77b0fc048e4501a80c73e72e428a52 [id=c5be59a2c3934b3789641ceeb2d4d6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.592844 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c578d81e8dd41aaa0f707c4e3fed225. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.599363 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_bf0900ef3fb8447c834a43e509caa105"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:25.600099 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bf0900ef3fb8447c834a43e509caa105 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:25.602079 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_70adbfea12154508bd1988d085007a3e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:25.602667 591 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 804a11c8f7414eb1a6bcff2952be9ae2 consensus state has no leader
I20251211 20:52:25.603085 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_9ee6894ba6364a54be16bb427ac7197b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:25.603710 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9ee6894ba6364a54be16bb427ac7197b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:25.605212 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3c578d81e8dd41aaa0f707c4e3fed225 consensus state has no leader
I20251211 20:52:25.607864 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 4b736a1b347d44b398980654caa21332
W20251211 20:52:25.608677 588 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 97ada0641165444a998db26796793507 has no consensus state
I20251211 20:52:25.608721 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_d282d0b8f49e4f44998af55480ad3cb8" } modify_external_catalogs: true
I20251211 20:52:25.608942 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_d282d0b8f49e4f44998af55480ad3cb8" } modify_external_catalogs: true
I20251211 20:52:25.609068 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_bcccedcd97444712a0f74e76db8cd74a" } modify_external_catalogs: true
I20251211 20:52:25.609242 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_bcccedcd97444712a0f74e76db8cd74a" } modify_external_catalogs: true
I20251211 20:52:25.610930 593 master_service.cc:954] ReplaceTablet: received request to replace tablet c5b164dc0ce1461fae98d950f7c7a726 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:25.611315 589 master_service.cc:954] ReplaceTablet: received request to replace tablet 732c76659c0540bc8b6909b430ffb146 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:25.611428 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b736a1b347d44b398980654caa21332 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:25.611611 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 1951a67964834152afb31f95741d981a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:25.611850 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 8a064789f89c424ca698e52333d73eea with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:25.611464 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 049cdecf12304211adbc1c1008dba989 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:25.611876 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 732c76659c0540bc8b6909b430ffb146 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:25.612044 739 tablet_service.cc:1552] Processing DeleteTablet for tablet af3ca9ac988e458cb7068c10c006d587 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:25.612133 1165 tablet_replica.cc:333] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:25.612344 1165 raft_consensus.cc:2243] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:25.612664 1165 pending_rounds.cc:70] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919: Trying to abort 1 pending ops.
I20251211 20:52:25.612746 1165 pending_rounds.cc:77] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432070725939200 op_type: NO_OP noop_request { }
I20251211 20:52:25.612866 1165 raft_consensus.cc:2889] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:25.612977 1165 raft_consensus.cc:2272] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:25.602676 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_70adbfea12154508bd1988d085007a3e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:25.613662 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 4b736a1b347d44b398980654caa21332 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4b736a1b347d44b398980654caa21332 already in progress: creating tablet
I20251211 20:52:25.614691 1165 ts_tablet_manager.cc:1916] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:25.618738 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet c4b76a7e84d344268567c551236ce772 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c4b76a7e84d344268567c551236ce772 already in progress: creating tablet
W20251211 20:52:25.619107 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet e41f052151624df39a3c83744870c94e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e41f052151624df39a3c83744870c94e already in progress: creating tablet
W20251211 20:52:25.619398 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet f93006227cc6489e91f3593e11789b54 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f93006227cc6489e91f3593e11789b54 already in progress: creating tablet
W20251211 20:52:25.619640 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet dcdcff8235814b58b3683104a2ff0cbf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dcdcff8235814b58b3683104a2ff0cbf already in progress: creating tablet
I20251211 20:52:25.620584 1236 raft_consensus.cc:493] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:25.620759 1236 raft_consensus.cc:515] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.621477 1236 leader_election.cc:290] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
W20251211 20:52:25.622442 847 leader_election.cc:343] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 297dc26ade384213b364a876c54bdd14
I20251211 20:52:25.623032 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b736a1b347d44b398980654caa21332 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:25.623680 1340 raft_consensus.cc:493] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:25.625145 1340 raft_consensus.cc:515] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.627053 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0129162969d74f5da3c08a1e6a27ab22" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:25.628243 1165 ts_tablet_manager.cc:1929] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:25.628455 1165 log.cc:1199] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/1951a67964834152afb31f95741d981a
I20251211 20:52:25.628932 1165 ts_tablet_manager.cc:1950] T 1951a67964834152afb31f95741d981a P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:25.630229 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 1951a67964834152afb31f95741d981a (table default.Table_39124c0c70034d00818c18db160416cf [id=016a0bf9d2704b74af4e64d2f9a23878]) successfully deleted
I20251211 20:52:25.630272 739 tablet_service.cc:1505] Processing CreateTablet for tablet 9f8aa28c1c9b4fc58dfc128952c180ba (DEFAULT_TABLE table=default.Table_03555bd3f502404a928ffdf4ef934e49 [id=0327e686e7ed4475a3e8cda8721a3a3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.630671 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f8aa28c1c9b4fc58dfc128952c180ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.631281 744 tablet_service.cc:1505] Processing CreateTablet for tablet 8feee01fc0aa4b32be926ff2b617b894 (DEFAULT_TABLE table=default.Table_265379fa17b74d0f954a2a2c4de113c8 [id=18f8d722faaa4293a112525ff0c96216]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.631788 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8feee01fc0aa4b32be926ff2b617b894. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.636412 749 tablet_service.cc:1505] Processing CreateTablet for tablet 822abda3c85848728205b71847ef7029 (DEFAULT_TABLE table=default.Table_4aad4cf01eff415f86b8446c29977007 [id=01282adb7b834261a7750c350ad0b250]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.636873 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 822abda3c85848728205b71847ef7029. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.637583 1168 tablet_bootstrap.cc:492] T e41f052151624df39a3c83744870c94e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:25.637773 1168 ts_tablet_manager.cc:1403] T e41f052151624df39a3c83744870c94e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.118s user 0.024s sys 0.051s
I20251211 20:52:25.638618 1168 raft_consensus.cc:359] T e41f052151624df39a3c83744870c94e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.638839 1168 raft_consensus.cc:385] T e41f052151624df39a3c83744870c94e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.638918 1168 raft_consensus.cc:740] T e41f052151624df39a3c83744870c94e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.640600 1168 consensus_queue.cc:260] T e41f052151624df39a3c83744870c94e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.641171 1168 ts_tablet_manager.cc:1434] T e41f052151624df39a3c83744870c94e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:25.641572 1168 tablet_bootstrap.cc:492] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:25.642349 739 tablet_service.cc:1505] Processing CreateTablet for tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d (DEFAULT_TABLE table=default.Table_ae7f102bdee3470990e863235f08d636 [id=d555d1fc6aef4a21b4d3f2804264d030]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.643278 741 tablet_service.cc:1505] Processing CreateTablet for tablet 28ef13be3b84410d9c786c43502a3478 (DEFAULT_TABLE table=default.Table_88dcd36b4447481487bc918a05511862 [id=d9453cea23a8430c98a3317856b3cec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.643689 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28ef13be3b84410d9c786c43502a3478. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.644589 1168 tablet_bootstrap.cc:654] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.645196 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.651077 1166 raft_consensus.cc:359] T 8dff13e22899452c8d4f2731b8dac7d5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:25.651481 1404 raft_consensus.cc:493] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:25.652065 1404 raft_consensus.cc:515] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.652825 1404 leader_election.cc:290] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:25.656464 1019 tablet_service.cc:1552] Processing DeleteTablet for tablet f93006227cc6489e91f3593e11789b54 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:25.656720 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 2433068ec179490dac3e91d046a42b70 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:25.659044 595 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 01ada50100bc48e39d1bc67082da750e (table default.Table_18d19a317a9c4656b4c764df4a64dbea [id=a24d8c58b9d84cdea2b278ae9c6e0ba5]). Expected version 2 got 0
I20251211 20:52:25.659204 1021 tablet_service.cc:1505] Processing CreateTablet for tablet 3c578d81e8dd41aaa0f707c4e3fed225 (DEFAULT_TABLE table=default.Table_bb77b0fc048e4501a80c73e72e428a52 [id=c5be59a2c3934b3789641ceeb2d4d6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.659374 595 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet a5eeb56709f046d9be92ac600ebc330e (table default.Table_f2ce490e0c5d445ab84989aae77c2f19 [id=f78e1da479af4d23a552af1c4fc55f81]). Expected version 1 got 0
I20251211 20:52:25.660033 1340 leader_election.cc:290] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:25.661046 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet e41f052151624df39a3c83744870c94e with delete_type TABLET_DATA_DELETED (replaced by tablet b33fd6c238e049b382d8ec9da910fe92) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:25.661619 1163 tablet_replica.cc:333] T 2433068ec179490dac3e91d046a42b70 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:25.663005 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b736a1b347d44b398980654caa21332 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:25.663213 983 leader_election.cc:343] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.663442 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet dcdcff8235814b58b3683104a2ff0cbf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:25.664825 1028 tablet_service.cc:1505] Processing CreateTablet for tablet 8feee01fc0aa4b32be926ff2b617b894 (DEFAULT_TABLE table=default.Table_265379fa17b74d0f954a2a2c4de113c8 [id=18f8d722faaa4293a112525ff0c96216]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:25.665706 847 leader_election.cc:343] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: ec7df7b11ce34b8489048e6acdcb4e94
I20251211 20:52:25.665879 847 leader_election.cc:304] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:25.662976 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 9f8aa28c1c9b4fc58dfc128952c180ba (DEFAULT_TABLE table=default.Table_03555bd3f502404a928ffdf4ef934e49 [id=0327e686e7ed4475a3e8cda8721a3a3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.667644 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet c4b76a7e84d344268567c551236ce772 with delete_type TABLET_DATA_DELETED (replaced by tablet 68be21a25c6743f5b86d518d98fbc35e) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:25.667682 1166 raft_consensus.cc:385] T 8dff13e22899452c8d4f2731b8dac7d5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.667987 1166 raft_consensus.cc:740] T 8dff13e22899452c8d4f2731b8dac7d5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.668262 1166 consensus_queue.cc:260] T 8dff13e22899452c8d4f2731b8dac7d5 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:25.668882 1165 tablet_replica.cc:333] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
W20251211 20:52:25.669898 847 leader_election.cc:343] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 297dc26ade384213b364a876c54bdd14
I20251211 20:52:25.670068 847 leader_election.cc:304] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
W20251211 20:52:25.670593 847 leader_election.cc:343] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: e69d53ac90a7463aa36ff15dadad7e0b
W20251211 20:52:25.670775 847 leader_election.cc:343] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: e69d53ac90a7463aa36ff15dadad7e0b
I20251211 20:52:25.670890 847 leader_election.cc:304] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:25.671159 1401 raft_consensus.cc:2749] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:25.671252 1236 raft_consensus.cc:2749] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:25.671365 1401 raft_consensus.cc:2749] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:25.674095 710 leader_election.cc:343] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: af3ca9ac988e458cb7068c10c006d587
I20251211 20:52:25.677006 1160 tablet_bootstrap.cc:654] T 33e1d8a576cd41b1a34778b1458df396 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.684924 894 tablet_service.cc:1505] Processing CreateTablet for tablet 822abda3c85848728205b71847ef7029 (DEFAULT_TABLE table=default.Table_4aad4cf01eff415f86b8446c29977007 [id=01282adb7b834261a7750c350ad0b250]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.685452 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 822abda3c85848728205b71847ef7029. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:25.688907 710 leader_election.cc:343] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: af3ca9ac988e458cb7068c10c006d587
W20251211 20:52:25.689424 710 leader_election.cc:343] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 8a064789f89c424ca698e52333d73eea
I20251211 20:52:25.689798 710 leader_election.cc:304] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:25.690485 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0129162969d74f5da3c08a1e6a27ab22" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:25.692456 890 tablet_service.cc:1505] Processing CreateTablet for tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d (DEFAULT_TABLE table=default.Table_ae7f102bdee3470990e863235f08d636 [id=d555d1fc6aef4a21b4d3f2804264d030]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.692906 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.694543 1165 raft_consensus.cc:2243] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.694819 1420 raft_consensus.cc:2706] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Received pre-election callback for term 1 while not running: Illegal state: RaftConsensus is not running: State = Stopping
I20251211 20:52:25.695461 1165 raft_consensus.cc:2272] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.697093 1165 ts_tablet_manager.cc:1916] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.700888 878 tablet_service.cc:1552] Processing DeleteTablet for tablet 7f086c7c56fe4abb8df7d3c0978cce6e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:25.701233 1164 tablet_replica.cc:333] T 7f086c7c56fe4abb8df7d3c0978cce6e P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:25.701449 1164 raft_consensus.cc:2243] T 7f086c7c56fe4abb8df7d3c0978cce6e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.701650 1164 raft_consensus.cc:2272] T 7f086c7c56fe4abb8df7d3c0978cce6e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.702416 1164 ts_tablet_manager.cc:1916] T 7f086c7c56fe4abb8df7d3c0978cce6e P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.702469 878 tablet_service.cc:1552] Processing DeleteTablet for tablet 1951a67964834152afb31f95741d981a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:25.703097 878 tablet_service.cc:1552] Processing DeleteTablet for tablet 5ed0a0141051462eb45ca3a66d41d009 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:25.703714 878 tablet_service.cc:1552] Processing DeleteTablet for tablet 297dc26ade384213b364a876c54bdd14 with delete_type TABLET_DATA_DELETED (replaced by tablet 8dff13e22899452c8d4f2731b8dac7d5) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:25.704293 878 tablet_service.cc:1552] Processing DeleteTablet for tablet 353da9106578450987a4f6544df8d7e6 with delete_type TABLET_DATA_DELETED (replaced by tablet d762b4afb5ff447dac3468393826205a) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:25.705050 878 tablet_service.cc:1552] Processing DeleteTablet for tablet acdb7dddf3e74ce1bc92f232b6180cca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:25.705138 1168 tablet_bootstrap.cc:492] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:25.705288 1168 ts_tablet_manager.cc:1403] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.064s user 0.009s sys 0.018s
I20251211 20:52:25.705626 878 tablet_service.cc:1552] Processing DeleteTablet for tablet e69d53ac90a7463aa36ff15dadad7e0b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:25.705888 890 tablet_service.cc:1552] Processing DeleteTablet for tablet 049cdecf12304211adbc1c1008dba989 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:25.706220 1168 raft_consensus.cc:359] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.706220 1019 tablet_service.cc:1505] Processing CreateTablet for tablet d762b4afb5ff447dac3468393826205a (DEFAULT_TABLE table=default.Table_a0c412985adf4382a7ea28c77654fd6c [id=4a74b9a60cbb4198b55cee630b2f4bef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.706404 1168 raft_consensus.cc:385] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.706496 1168 raft_consensus.cc:740] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.706362 889 tablet_service.cc:1505] Processing CreateTablet for tablet 28ef13be3b84410d9c786c43502a3478 (DEFAULT_TABLE table=default.Table_88dcd36b4447481487bc918a05511862 [id=d9453cea23a8430c98a3317856b3cec3]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:25.706606 982 leader_election.cc:343] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.706871 1166 ts_tablet_manager.cc:1434] T 8dff13e22899452c8d4f2731b8dac7d5 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.169s user 0.000s sys 0.003s
I20251211 20:52:25.706947 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c578d81e8dd41aaa0f707c4e3fed225. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.706980 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28ef13be3b84410d9c786c43502a3478. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.707257 1166 tablet_bootstrap.cc:492] T c5b164dc0ce1461fae98d950f7c7a726 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:25.708750 1164 ts_tablet_manager.cc:1929] T 7f086c7c56fe4abb8df7d3c0978cce6e P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:25.708868 1164 log.cc:1199] T 7f086c7c56fe4abb8df7d3c0978cce6e P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/7f086c7c56fe4abb8df7d3c0978cce6e
I20251211 20:52:25.708972 880 tablet_service.cc:1505] Processing CreateTablet for tablet 8feee01fc0aa4b32be926ff2b617b894 (DEFAULT_TABLE table=default.Table_265379fa17b74d0f954a2a2c4de113c8 [id=18f8d722faaa4293a112525ff0c96216]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.709084 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d762b4afb5ff447dac3468393826205a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.709280 1164 ts_tablet_manager.cc:1950] T 7f086c7c56fe4abb8df7d3c0978cce6e P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:25.709347 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8feee01fc0aa4b32be926ff2b617b894. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.710166 1164 tablet_replica.cc:333] T 1951a67964834152afb31f95741d981a P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:25.710374 1164 raft_consensus.cc:2243] T 1951a67964834152afb31f95741d981a P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.710534 1164 raft_consensus.cc:2272] T 1951a67964834152afb31f95741d981a P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.711328 1164 ts_tablet_manager.cc:1916] T 1951a67964834152afb31f95741d981a P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.712605 1024 tablet_service.cc:1505] Processing CreateTablet for tablet 40722aef4e194ff094d271a079bda9d9 (DEFAULT_TABLE table=default.Table_b7c88854437f4dfd9998ccb8246e6d0d [id=3309205faabf4a42acadc43766e4f57f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.712744 1340 raft_consensus.cc:493] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:25.712905 1340 raft_consensus.cc:515] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.713042 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40722aef4e194ff094d271a079bda9d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.713546 1340 leader_election.cc:290] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:25.713769 1340 raft_consensus.cc:493] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:25.713946 1340 raft_consensus.cc:515] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.714476 1340 leader_election.cc:290] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:25.714643 1163 raft_consensus.cc:2243] T 2433068ec179490dac3e91d046a42b70 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.714847 1163 raft_consensus.cc:2272] T 2433068ec179490dac3e91d046a42b70 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.715060 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d (DEFAULT_TABLE table=default.Table_ae7f102bdee3470990e863235f08d636 [id=d555d1fc6aef4a21b4d3f2804264d030]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.715480 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.716893 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2260850f32b541aab1128818f144b5a7" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:25.716979 1163 ts_tablet_manager.cc:1916] T 2433068ec179490dac3e91d046a42b70 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.717240 1164 ts_tablet_manager.cc:1929] T 1951a67964834152afb31f95741d981a P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20251211 20:52:25.717731 983 leader_election.cc:343] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.719674 1016 tablet_service.cc:1505] Processing CreateTablet for tablet 28ef13be3b84410d9c786c43502a3478 (DEFAULT_TABLE table=default.Table_88dcd36b4447481487bc918a05511862 [id=d9453cea23a8430c98a3317856b3cec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.720347 1016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28ef13be3b84410d9c786c43502a3478. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.722132 1164 log.cc:1199] T 1951a67964834152afb31f95741d981a P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/1951a67964834152afb31f95741d981a
I20251211 20:52:25.717319 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c58241f7e1404a8c8db92c4f9a47d143" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:25.722568 1164 ts_tablet_manager.cc:1950] T 1951a67964834152afb31f95741d981a P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
W20251211 20:52:25.722771 983 leader_election.cc:343] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.723601 1164 tablet_replica.cc:333] T 5ed0a0141051462eb45ca3a66d41d009 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:25.723665 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 822abda3c85848728205b71847ef7029 (DEFAULT_TABLE table=default.Table_4aad4cf01eff415f86b8446c29977007 [id=01282adb7b834261a7750c350ad0b250]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:25.723793 1164 raft_consensus.cc:2243] T 5ed0a0141051462eb45ca3a66d41d009 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.723934 1163 ts_tablet_manager.cc:1929] T 2433068ec179490dac3e91d046a42b70 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:25.723963 1164 raft_consensus.cc:2272] T 5ed0a0141051462eb45ca3a66d41d009 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.724071 1163 log.cc:1199] T 2433068ec179490dac3e91d046a42b70 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/2433068ec179490dac3e91d046a42b70
I20251211 20:52:25.724079 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 822abda3c85848728205b71847ef7029. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.724435 1163 ts_tablet_manager.cc:1950] T 2433068ec179490dac3e91d046a42b70 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:25.725401 1163 tablet_replica.cc:333] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:25.725603 1163 raft_consensus.cc:2243] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.725806 1163 raft_consensus.cc:2272] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.726352 1163 ts_tablet_manager.cc:1916] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.727056 1164 ts_tablet_manager.cc:1916] T 5ed0a0141051462eb45ca3a66d41d009 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.727259 1168 consensus_queue.cc:260] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.727972 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ef506349e5b434db7a78809dc44014e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:25.728040 1168 ts_tablet_manager.cc:1434] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.023s user 0.001s sys 0.001s
I20251211 20:52:25.728209 982 leader_election.cc:304] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:25.728367 1168 tablet_bootstrap.cc:492] T 034f47d8564f4094a9d76aee2fea3119 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:25.728518 890 tablet_service.cc:1552] Processing DeleteTablet for tablet af3ca9ac988e458cb7068c10c006d587 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:25.729506 890 tablet_service.cc:1552] Processing DeleteTablet for tablet f4db14d082d34aef857e106992bb8950 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:25.729508 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c58241f7e1404a8c8db92c4f9a47d143" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:25.730079 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2260850f32b541aab1128818f144b5a7" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:25.732730 1451 raft_consensus.cc:2749] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:25.733245 1404 raft_consensus.cc:493] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:25.733389 1404 raft_consensus.cc:515] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.733671 588 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet b33fd6c238e049b382d8ec9da910fe92
I20251211 20:52:25.733776 1168 tablet_bootstrap.cc:654] T 034f47d8564f4094a9d76aee2fea3119 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.734041 1404 leader_election.cc:290] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
W20251211 20:52:25.730723 982 leader_election.cc:343] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.734463 982 leader_election.cc:304] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:25.734876 982 leader_election.cc:343] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.734997 982 leader_election.cc:304] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:25.735397 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 8139acb0ae324d0f973ce877edc1cd04
I20251211 20:52:25.735519 1340 raft_consensus.cc:2749] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:25.735723 1340 raft_consensus.cc:2749] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:25.736507 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a5eeb56709f046d9be92ac600ebc330e" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:25.737057 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8feee01fc0aa4b32be926ff2b617b894. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.739761 890 tablet_service.cc:1552] Processing DeleteTablet for tablet 73aa2a3fb2f44bbca870fb7f4cf6087c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:25.740279 894 tablet_service.cc:1552] Processing DeleteTablet for tablet ec7df7b11ce34b8489048e6acdcb4e94 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:25.741832 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_1d1c44cc71394478bc3f93b23ca3dad4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:25.742446 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1d1c44cc71394478bc3f93b23ca3dad4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:25.743121 847 leader_election.cc:343] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.743923 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a5eeb56709f046d9be92ac600ebc330e" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:25.744158 1049 raft_consensus.cc:2468] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 0.
W20251211 20:52:25.745144 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 4b736a1b347d44b398980654caa21332 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4b736a1b347d44b398980654caa21332 already in progress: creating tablet
I20251211 20:52:25.745369 1166 tablet_bootstrap.cc:654] T c5b164dc0ce1461fae98d950f7c7a726 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.745755 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 7f086c7c56fe4abb8df7d3c0978cce6e (table default.Table_e5a162d6463e4ff5a4edaaae5a4d03cb [id=872ff521abd740ee924f22e9c0ff8585]) successfully deleted
I20251211 20:52:25.745952 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 1951a67964834152afb31f95741d981a (table default.Table_39124c0c70034d00818c18db160416cf [id=016a0bf9d2704b74af4e64d2f9a23878]) successfully deleted
I20251211 20:52:25.746447 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_383cabcd12a74a14bc3d7dec7c236a19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:25.747011 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_383cabcd12a74a14bc3d7dec7c236a19 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:25.748524 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f8aa28c1c9b4fc58dfc128952c180ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:25.756349 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 8139acb0ae324d0f973ce877edc1cd04 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:25.757225 1165 ts_tablet_manager.cc:1929] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:25.757383 1165 log.cc:1199] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/8a064789f89c424ca698e52333d73eea
I20251211 20:52:25.757975 1165 ts_tablet_manager.cc:1950] T 8a064789f89c424ca698e52333d73eea P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:25.758986 1165 tablet_replica.cc:333] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:25.759160 1165 raft_consensus.cc:2243] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:25.759428 1165 pending_rounds.cc:70] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919: Trying to abort 1 pending ops.
I20251211 20:52:25.759485 1165 pending_rounds.cc:77] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432070795849728 op_type: NO_OP noop_request { }
I20251211 20:52:25.769896 1165 raft_consensus.cc:2889] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:25.770081 1165 raft_consensus.cc:2272] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.771196 1165 ts_tablet_manager.cc:1916] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.771412 1160 tablet_bootstrap.cc:492] T 33e1d8a576cd41b1a34778b1458df396 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:25.771601 1160 ts_tablet_manager.cc:1403] T 33e1d8a576cd41b1a34778b1458df396 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.184s user 0.002s sys 0.001s
I20251211 20:52:25.772398 1160 raft_consensus.cc:359] T 33e1d8a576cd41b1a34778b1458df396 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.772593 1160 raft_consensus.cc:385] T 33e1d8a576cd41b1a34778b1458df396 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.772681 1160 raft_consensus.cc:740] T 33e1d8a576cd41b1a34778b1458df396 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.772945 1160 consensus_queue.cc:260] T 33e1d8a576cd41b1a34778b1458df396 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.774492 1160 ts_tablet_manager.cc:1434] T 33e1d8a576cd41b1a34778b1458df396 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251211 20:52:25.774796 1160 tablet_bootstrap.cc:492] T 694518e6c1ae4032b06b99e9bde2e5dd P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:25.776535 1160 tablet_bootstrap.cc:654] T 694518e6c1ae4032b06b99e9bde2e5dd P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.780483 847 leader_election.cc:304] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919
I20251211 20:52:25.780623 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_03555bd3f502404a928ffdf4ef934e49" } new_table_name: "default.Table_bf7d402a36be44cfb6b79041906d26a1" modify_external_catalogs: true
I20251211 20:52:25.781098 1164 ts_tablet_manager.cc:1929] T 5ed0a0141051462eb45ca3a66d41d009 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:25.781250 1164 log.cc:1199] T 5ed0a0141051462eb45ca3a66d41d009 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/5ed0a0141051462eb45ca3a66d41d009
I20251211 20:52:25.782313 1404 raft_consensus.cc:2804] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:25.782462 1404 raft_consensus.cc:493] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:25.782550 1404 raft_consensus.cc:3060] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:25.783253 1164 ts_tablet_manager.cc:1950] T 5ed0a0141051462eb45ca3a66d41d009 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:25.783444 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_70a00e0b1675435d9553c29cf9b0d3ea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:25.784013 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_70a00e0b1675435d9553c29cf9b0d3ea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:25.784389 1164 tablet_replica.cc:333] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:25.784590 1164 raft_consensus.cc:2243] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.784762 1164 raft_consensus.cc:2272] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.785292 1404 raft_consensus.cc:515] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.785962 1404 leader_election.cc:290] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Requested vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:25.786336 1164 ts_tablet_manager.cc:1916] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.786319 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_75c7933bd82449a1a24a7481815ee23b" } modify_external_catalogs: true
I20251211 20:52:25.786339 1422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9230f94a6a1a49cead51aeea9f069087. Latest consensus state: current_term: 1 leader_uuid: "9230f94a6a1a49cead51aeea9f069087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } } }
I20251211 20:52:25.786489 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_75c7933bd82449a1a24a7481815ee23b" } modify_external_catalogs: true
I20251211 20:52:25.786561 1422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [sys.catalog]: This master's current role is: FOLLOWER
I20251211 20:52:25.787155 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_415c09e71243460aaa9acfd57742bcd7" } modify_external_catalogs: true
I20251211 20:52:25.787346 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_415c09e71243460aaa9acfd57742bcd7" } modify_external_catalogs: true
I20251211 20:52:25.787761 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a5eeb56709f046d9be92ac600ebc330e" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:25.788098 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f26ebdd2202a43c58d2e43dcf381d0aa" } new_table_name: "default.Table_786e1caf9f674d48a6aad3ef7edc5b01" modify_external_catalogs: true
I20251211 20:52:25.793155 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 5ed0a0141051462eb45ca3a66d41d009 (table default.Table_40256d3973c74566a09b1d235062d284 [id=1f4650c549d94e2c9ef753ff96670aa6]) successfully deleted
I20251211 20:52:25.795231 1165 ts_tablet_manager.cc:1929] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:25.795380 1165 log.cc:1199] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/049cdecf12304211adbc1c1008dba989
I20251211 20:52:25.800450 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 8a064789f89c424ca698e52333d73eea (table default.Table_f439572c5b1d4997af54e0ab3c036023 [id=cda4c0ad8a0347be88a0518491e6af49]) successfully deleted
I20251211 20:52:25.801304 739 tablet_service.cc:1552] Processing DeleteTablet for tablet e41f052151624df39a3c83744870c94e with delete_type TABLET_DATA_DELETED (replaced by tablet b33fd6c238e049b382d8ec9da910fe92) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:25.801692 739 tablet_service.cc:1552] Processing DeleteTablet for tablet 8139acb0ae324d0f973ce877edc1cd04 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:25.804807 1165 ts_tablet_manager.cc:1950] T 049cdecf12304211adbc1c1008dba989 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
W20251211 20:52:25.807454 847 leader_election.cc:343] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:25.808074 1164 ts_tablet_manager.cc:1929] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:25.808228 1164 log.cc:1199] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/297dc26ade384213b364a876c54bdd14
I20251211 20:52:25.809132 1164 ts_tablet_manager.cc:1950] T 297dc26ade384213b364a876c54bdd14 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:25.809345 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 049cdecf12304211adbc1c1008dba989 (table default.Table_e6a96166bf33442989e15129d4d44631 [id=34edf86667d448c0b87e55627c67a704]) successfully deleted
I20251211 20:52:25.809580 594 master_service.cc:954] ReplaceTablet: received request to replace tablet d824bc02b25842d18f28f711eef6669f from {username='slave'} at 127.0.0.1:44772
W20251211 20:52:25.809684 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet f93006227cc6489e91f3593e11789b54 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f93006227cc6489e91f3593e11789b54 already in progress: creating tablet
I20251211 20:52:25.809897 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 2433068ec179490dac3e91d046a42b70 (table default.Table_26b3b7e5db73457da38332d4e3d943f6 [id=18edd3deb100406cb5f421ee06216e6b]) successfully deleted
I20251211 20:52:25.810212 1165 tablet_replica.cc:333] T 732c76659c0540bc8b6909b430ffb146 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:25.810402 1165 raft_consensus.cc:2243] T 732c76659c0540bc8b6909b430ffb146 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.810611 1165 raft_consensus.cc:2272] T 732c76659c0540bc8b6909b430ffb146 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.810820 594 master_service.cc:954] ReplaceTablet: received request to replace tablet 0246f1df99684d0da43d97acbf1182a0 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:25.812618 1165 ts_tablet_manager.cc:1916] T 732c76659c0540bc8b6909b430ffb146 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.813498 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 297dc26ade384213b364a876c54bdd14 (table default.Table_f572100667d148e8a69e7930596f0f52 [id=cb97ccf262fd4c41a180267fd0678cec]) successfully deleted
I20251211 20:52:25.813576 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a5eeb56709f046d9be92ac600ebc330e" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:25.813702 1160 tablet_bootstrap.cc:492] T 694518e6c1ae4032b06b99e9bde2e5dd P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:25.813870 1160 ts_tablet_manager.cc:1403] T 694518e6c1ae4032b06b99e9bde2e5dd P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.039s user 0.000s sys 0.003s
I20251211 20:52:25.814682 1160 raft_consensus.cc:359] T 694518e6c1ae4032b06b99e9bde2e5dd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.814879 1160 raft_consensus.cc:385] T 694518e6c1ae4032b06b99e9bde2e5dd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.814957 1160 raft_consensus.cc:740] T 694518e6c1ae4032b06b99e9bde2e5dd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.815514 1049 raft_consensus.cc:3060] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:25.817034 1160 consensus_queue.cc:260] T 694518e6c1ae4032b06b99e9bde2e5dd P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.817646 1168 tablet_bootstrap.cc:492] T 034f47d8564f4094a9d76aee2fea3119 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:25.817724 1160 ts_tablet_manager.cc:1434] T 694518e6c1ae4032b06b99e9bde2e5dd P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20251211 20:52:25.817823 1168 ts_tablet_manager.cc:1403] T 034f47d8564f4094a9d76aee2fea3119 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.090s user 0.006s sys 0.006s
I20251211 20:52:25.818583 1168 raft_consensus.cc:359] T 034f47d8564f4094a9d76aee2fea3119 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.818784 1168 raft_consensus.cc:385] T 034f47d8564f4094a9d76aee2fea3119 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.818856 1168 raft_consensus.cc:740] T 034f47d8564f4094a9d76aee2fea3119 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.819119 1168 consensus_queue.cc:260] T 034f47d8564f4094a9d76aee2fea3119 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.819247 1160 tablet_bootstrap.cc:492] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:25.821360 1168 ts_tablet_manager.cc:1434] T 034f47d8564f4094a9d76aee2fea3119 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:25.821588 1160 tablet_bootstrap.cc:654] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.821698 1168 tablet_bootstrap.cc:492] T f4db14d082d34aef857e106992bb8950 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:25.824368 1168 tablet_bootstrap.cc:654] T f4db14d082d34aef857e106992bb8950 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.828670 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 0129162969d74f5da3c08a1e6a27ab22 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:25.829116 594 master_service.cc:954] ReplaceTablet: received request to replace tablet 55c40b2464f8444ca6c9fa0400f7f7f2 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:25.830158 1164 tablet_replica.cc:333] T 353da9106578450987a4f6544df8d7e6 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:25.830376 1164 raft_consensus.cc:2243] T 353da9106578450987a4f6544df8d7e6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.830593 1164 raft_consensus.cc:2272] T 353da9106578450987a4f6544df8d7e6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.831115 1164 ts_tablet_manager.cc:1916] T 353da9106578450987a4f6544df8d7e6 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.834443 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 8dff13e22899452c8d4f2731b8dac7d5 with delete_type TABLET_DATA_DELETED (replaced by tablet dc5de12701714fcd81b9dba6973a28b5) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:25.837198 1458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9230f94a6a1a49cead51aeea9f069087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } } }
I20251211 20:52:25.838490 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_6bf819d1a1794e62b4cf65ffe5d2c2b5" } new_table_name: "default.Table_c607094f0e4d4af48d2e54d4d728ae13" modify_external_catalogs: true
I20251211 20:52:25.839201 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4358d588f1664e98accbfc761378c61f" } new_table_name: "default.Table_b70392ad5fb6451482100395256494bd" modify_external_catalogs: true
I20251211 20:52:25.845530 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet 8139acb0ae324d0f973ce877edc1cd04 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:25.845815 1165 ts_tablet_manager.cc:1929] T 732c76659c0540bc8b6909b430ffb146 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:25.845963 1165 log.cc:1199] T 732c76659c0540bc8b6909b430ffb146 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/732c76659c0540bc8b6909b430ffb146
I20251211 20:52:25.846318 1165 ts_tablet_manager.cc:1950] T 732c76659c0540bc8b6909b430ffb146 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:25.846625 1458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [sys.catalog]: This master's current role is: LEADER
I20251211 20:52:25.849125 1049 raft_consensus.cc:2468] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 1.
I20251211 20:52:25.850422 1165 tablet_replica.cc:333] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:25.850634 1165 raft_consensus.cc:2243] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:25.850881 1165 pending_rounds.cc:70] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919: Trying to abort 1 pending ops.
I20251211 20:52:25.850965 1165 pending_rounds.cc:77] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432071430131712 op_type: NO_OP noop_request { }
I20251211 20:52:25.851073 1165 raft_consensus.cc:2889] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:25.851182 1165 raft_consensus.cc:2272] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.851435 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_638eea7803484ee9965a1058a3526993" } new_table_name: "default.Table_2a5ff353268344dba84653a0fd41e4a0" modify_external_catalogs: true
W20251211 20:52:25.852839 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ed9ee283045a44978422cd080aa2cda5 consensus state has no leader
I20251211 20:52:25.853288 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_e5979cadea8746f4afd0ae1b2673b272"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:25.853819 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e5979cadea8746f4afd0ae1b2673b272 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:25.853878 1165 ts_tablet_manager.cc:1916] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:25.857856 591 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9f8aa28c1c9b4fc58dfc128952c180ba consensus state has no leader
I20251211 20:52:25.858219 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_88dcd36b4447481487bc918a05511862" } modify_external_catalogs: true
I20251211 20:52:25.858455 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_88dcd36b4447481487bc918a05511862" } modify_external_catalogs: true
I20251211 20:52:25.862016 1168 tablet_bootstrap.cc:492] T f4db14d082d34aef857e106992bb8950 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:25.862174 1168 ts_tablet_manager.cc:1403] T f4db14d082d34aef857e106992bb8950 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.041s user 0.001s sys 0.002s
I20251211 20:52:25.862973 1168 raft_consensus.cc:359] T f4db14d082d34aef857e106992bb8950 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.863153 1168 raft_consensus.cc:385] T f4db14d082d34aef857e106992bb8950 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.863225 1168 raft_consensus.cc:740] T f4db14d082d34aef857e106992bb8950 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.863444 1168 consensus_queue.cc:260] T f4db14d082d34aef857e106992bb8950 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.863962 1168 ts_tablet_manager.cc:1434] T f4db14d082d34aef857e106992bb8950 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:25.864281 1168 tablet_bootstrap.cc:492] T 9662efc97a53461d888a062463e80ee6 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:25.865113 1166 tablet_bootstrap.cc:492] T c5b164dc0ce1461fae98d950f7c7a726 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:25.865307 1166 ts_tablet_manager.cc:1403] T c5b164dc0ce1461fae98d950f7c7a726 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.158s user 0.004s sys 0.000s
I20251211 20:52:25.867717 1168 tablet_bootstrap.cc:654] T 9662efc97a53461d888a062463e80ee6 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.869190 1166 raft_consensus.cc:359] T c5b164dc0ce1461fae98d950f7c7a726 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.869402 1166 raft_consensus.cc:385] T c5b164dc0ce1461fae98d950f7c7a726 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.869470 1166 raft_consensus.cc:740] T c5b164dc0ce1461fae98d950f7c7a726 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.869709 1166 consensus_queue.cc:260] T c5b164dc0ce1461fae98d950f7c7a726 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.870195 1166 ts_tablet_manager.cc:1434] T c5b164dc0ce1461fae98d950f7c7a726 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
W20251211 20:52:25.873499 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a9b2aa21ab124b28aebc33adba0583d2 has no consensus state
I20251211 20:52:25.873958 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_0c2f5186505f4a14b5c9bba07ca14178"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:25.874802 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8b89ae724b7f41c98741a41d75592da7 consensus state has no leader
I20251211 20:52:25.875151 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_c7e035d673434c588da10ea5a480f541"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:25.875881 592 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 05385f20e88c46108e09a78814811209 consensus state has no leader
I20251211 20:52:25.876268 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_0a0fab7f167b4e34b48892ee705fd1e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:25.878535 1165 ts_tablet_manager.cc:1929] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:25.878703 1165 log.cc:1199] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/af3ca9ac988e458cb7068c10c006d587
I20251211 20:52:25.879184 1165 ts_tablet_manager.cc:1950] T af3ca9ac988e458cb7068c10c006d587 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:25.881112 1166 tablet_bootstrap.cc:492] T 1c04187b8ed146c289a6b17d62b07199 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:25.882413 1165 tablet_replica.cc:333] T e41f052151624df39a3c83744870c94e P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:25.882623 1165 raft_consensus.cc:2243] T e41f052151624df39a3c83744870c94e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.882788 1165 raft_consensus.cc:2272] T e41f052151624df39a3c83744870c94e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.883001 1166 tablet_bootstrap.cc:654] T 1c04187b8ed146c289a6b17d62b07199 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:25.891938 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c2f5186505f4a14b5c9bba07ca14178 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:25.892333 847 leader_election.cc:304] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919
I20251211 20:52:25.892406 1163 ts_tablet_manager.cc:1929] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:25.892537 1163 log.cc:1199] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/e41f052151624df39a3c83744870c94e
I20251211 20:52:25.892886 1163 ts_tablet_manager.cc:1950] T e41f052151624df39a3c83744870c94e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:25.893448 1404 raft_consensus.cc:2804] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:25.893600 1404 raft_consensus.cc:697] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [term 1 LEADER]: Becoming Leader. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Running, Role: LEADER
I20251211 20:52:25.894362 1404 consensus_queue.cc:237] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.870666 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 732c76659c0540bc8b6909b430ffb146 (table default.Table_a2a9717ed78046cab7c00077ac646665 [id=65dab4128d204227947dd4651a5aeb6c]) successfully deleted
W20251211 20:52:25.895727 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c7e035d673434c588da10ea5a480f541 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:25.895810 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet c5b164dc0ce1461fae98d950f7c7a726 with delete_type TABLET_DATA_DELETED (replaced by tablet 49a33084c0fc460eb9592224b6dfc279) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:25.896927 1166 tablet_bootstrap.cc:492] T 1c04187b8ed146c289a6b17d62b07199 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:25.897079 1166 ts_tablet_manager.cc:1403] T 1c04187b8ed146c289a6b17d62b07199 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.001s
I20251211 20:52:25.897861 1166 raft_consensus.cc:359] T 1c04187b8ed146c289a6b17d62b07199 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:25.898065 1166 raft_consensus.cc:385] T 1c04187b8ed146c289a6b17d62b07199 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20251211 20:52:25.898041 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0a0fab7f167b4e34b48892ee705fd1e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:25.898144 1166 raft_consensus.cc:740] T 1c04187b8ed146c289a6b17d62b07199 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.898353 1166 consensus_queue.cc:260] T 1c04187b8ed146c289a6b17d62b07199 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:25.899864 1164 ts_tablet_manager.cc:1929] T 353da9106578450987a4f6544df8d7e6 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:25.900008 1164 log.cc:1199] T 353da9106578450987a4f6544df8d7e6 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/353da9106578450987a4f6544df8d7e6
I20251211 20:52:25.900413 1164 ts_tablet_manager.cc:1950] T 353da9106578450987a4f6544df8d7e6 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:25.900357 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet af3ca9ac988e458cb7068c10c006d587 (table default.Table_a11e64052cf34ba6a6a31f4edb417c19 [id=0afc1e783888443eb3f3e7848a04c207]) successfully deleted
W20251211 20:52:25.900743 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 4b736a1b347d44b398980654caa21332 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4b736a1b347d44b398980654caa21332 already in progress: creating tablet
I20251211 20:52:25.900925 1164 tablet_replica.cc:333] T acdb7dddf3e74ce1bc92f232b6180cca P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:25.901116 1164 raft_consensus.cc:2243] T acdb7dddf3e74ce1bc92f232b6180cca P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.901289 1164 raft_consensus.cc:2272] T acdb7dddf3e74ce1bc92f232b6180cca P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.901835 1164 ts_tablet_manager.cc:1916] T acdb7dddf3e74ce1bc92f232b6180cca P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.903862 1094 heartbeater.cc:344] Connected to a master server at 127.0.104.62:41123
I20251211 20:52:25.912484 1166 ts_tablet_manager.cc:1434] T 1c04187b8ed146c289a6b17d62b07199 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.015s user 0.003s sys 0.000s
I20251211 20:52:25.912901 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c04187b8ed146c289a6b17d62b07199 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:25.914094 1168 tablet_bootstrap.cc:492] T 9662efc97a53461d888a062463e80ee6 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:25.914119 1163 tablet_replica.cc:333] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
W20251211 20:52:25.914238 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet dcdcff8235814b58b3683104a2ff0cbf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dcdcff8235814b58b3683104a2ff0cbf already in progress: creating tablet
I20251211 20:52:25.914294 1168 ts_tablet_manager.cc:1403] T 9662efc97a53461d888a062463e80ee6 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.050s user 0.005s sys 0.007s
W20251211 20:52:25.914577 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet c4b76a7e84d344268567c551236ce772 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c4b76a7e84d344268567c551236ce772 already in progress: creating tablet
W20251211 20:52:25.914925 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 4ef506349e5b434db7a78809dc44014e because the tablet was not found. No further retry: Not found: Tablet not found: 4ef506349e5b434db7a78809dc44014e
I20251211 20:52:25.915081 1168 raft_consensus.cc:359] T 9662efc97a53461d888a062463e80ee6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.915279 1168 raft_consensus.cc:385] T 9662efc97a53461d888a062463e80ee6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.915347 1168 raft_consensus.cc:740] T 9662efc97a53461d888a062463e80ee6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.915553 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 353da9106578450987a4f6544df8d7e6 (table default.Table_a0c412985adf4382a7ea28c77654fd6c [id=4a74b9a60cbb4198b55cee630b2f4bef]) successfully deleted
I20251211 20:52:25.915634 1168 consensus_queue.cc:260] T 9662efc97a53461d888a062463e80ee6 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:25.916121 1168 ts_tablet_manager.cc:1434] T 9662efc97a53461d888a062463e80ee6 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251211 20:52:25.916548 1168 tablet_bootstrap.cc:492] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:25.917474 1164 ts_tablet_manager.cc:1929] T acdb7dddf3e74ce1bc92f232b6180cca P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:25.917613 1164 log.cc:1199] T acdb7dddf3e74ce1bc92f232b6180cca P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/acdb7dddf3e74ce1bc92f232b6180cca
I20251211 20:52:25.917960 1164 ts_tablet_manager.cc:1950] T acdb7dddf3e74ce1bc92f232b6180cca P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:25.918280 1168 tablet_bootstrap.cc:654] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.921317 822 heartbeater.cc:344] Connected to a master server at 127.0.104.62:41123
I20251211 20:52:25.922062 1164 tablet_replica.cc:333] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:25.922267 1164 raft_consensus.cc:2243] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.923810 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet acdb7dddf3e74ce1bc92f232b6180cca (table default.Table_28b427311c9b4699a47c38ebb4d93e01 [id=b68aac5147654707b677d020a2cc37cd]) successfully deleted
I20251211 20:52:25.923933 1164 raft_consensus.cc:2272] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.924885 1165 ts_tablet_manager.cc:1916] T e41f052151624df39a3c83744870c94e P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.925567 1163 raft_consensus.cc:2243] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.925873 1163 raft_consensus.cc:2272] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.915915 1166 tablet_bootstrap.cc:492] T 5fc8d485cc234fd490c487a8bc881bf4 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:25.927436 1163 ts_tablet_manager.cc:1916] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.929113 1164 ts_tablet_manager.cc:1916] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.929302 1160 tablet_bootstrap.cc:492] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:25.929457 1160 ts_tablet_manager.cc:1403] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.110s user 0.006s sys 0.009s
I20251211 20:52:25.930541 1160 raft_consensus.cc:359] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:25.930768 1160 raft_consensus.cc:385] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.930872 1160 raft_consensus.cc:740] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.931061 1165 ts_tablet_manager.cc:1929] T e41f052151624df39a3c83744870c94e P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:25.931203 1165 log.cc:1199] T e41f052151624df39a3c83744870c94e P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/e41f052151624df39a3c83744870c94e
I20251211 20:52:25.931383 1166 tablet_bootstrap.cc:654] T 5fc8d485cc234fd490c487a8bc881bf4 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.936942 1164 ts_tablet_manager.cc:1929] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:25.937101 1164 log.cc:1199] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/e69d53ac90a7463aa36ff15dadad7e0b
I20251211 20:52:25.937490 1164 ts_tablet_manager.cc:1950] T e69d53ac90a7463aa36ff15dadad7e0b P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:25.938192 1164 tablet_replica.cc:333] T 049cdecf12304211adbc1c1008dba989 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:25.938395 1164 raft_consensus.cc:2243] T 049cdecf12304211adbc1c1008dba989 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.938577 1164 raft_consensus.cc:2272] T 049cdecf12304211adbc1c1008dba989 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.939152 1164 ts_tablet_manager.cc:1916] T 049cdecf12304211adbc1c1008dba989 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.940290 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet e69d53ac90a7463aa36ff15dadad7e0b (table default.Table_2645aa306b5c44f6bad5e2e67a91ebec [id=0ec7cf10eb8a47aca499b141913b38cf]) successfully deleted
I20251211 20:52:25.941686 1166 tablet_bootstrap.cc:492] T 5fc8d485cc234fd490c487a8bc881bf4 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:25.941934 1166 ts_tablet_manager.cc:1403] T 5fc8d485cc234fd490c487a8bc881bf4 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.026s user 0.006s sys 0.000s
I20251211 20:52:25.942824 1166 raft_consensus.cc:359] T 5fc8d485cc234fd490c487a8bc881bf4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:25.943115 1166 raft_consensus.cc:385] T 5fc8d485cc234fd490c487a8bc881bf4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.943230 1166 raft_consensus.cc:740] T 5fc8d485cc234fd490c487a8bc881bf4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.943713 1166 consensus_queue.cc:260] T 5fc8d485cc234fd490c487a8bc881bf4 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:25.948513 592 master_service.cc:954] ReplaceTablet: received request to replace tablet bb5c5523e175406d9f6f9cd2736a2045 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:25.949337 1166 ts_tablet_manager.cc:1434] T 5fc8d485cc234fd490c487a8bc881bf4 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:25.949911 1166 tablet_bootstrap.cc:492] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:25.953130 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_3570af5d8bff402ab3140ae51bae174c" } modify_external_catalogs: true
I20251211 20:52:25.953325 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_3570af5d8bff402ab3140ae51bae174c" } modify_external_catalogs: true
I20251211 20:52:25.954888 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_ae0290b97d234621bbd1ff141b5c2400"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:25.955487 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ae0290b97d234621bbd1ff141b5c2400 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:25.959564 1166 tablet_bootstrap.cc:654] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.961546 1163 ts_tablet_manager.cc:1929] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:25.961808 1164 ts_tablet_manager.cc:1929] T 049cdecf12304211adbc1c1008dba989 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:25.961958 1164 log.cc:1199] T 049cdecf12304211adbc1c1008dba989 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/049cdecf12304211adbc1c1008dba989
I20251211 20:52:25.962082 1163 log.cc:1199] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/0129162969d74f5da3c08a1e6a27ab22
I20251211 20:52:25.962602 1163 ts_tablet_manager.cc:1950] T 0129162969d74f5da3c08a1e6a27ab22 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:25.962829 1164 ts_tablet_manager.cc:1950] T 049cdecf12304211adbc1c1008dba989 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:25.962877 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a66a87006b684bbd87431e2aad969dcf" } modify_external_catalogs: true
I20251211 20:52:25.963114 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a66a87006b684bbd87431e2aad969dcf" } modify_external_catalogs: true
I20251211 20:52:25.964627 1164 tablet_replica.cc:333] T af3ca9ac988e458cb7068c10c006d587 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:25.964814 1164 raft_consensus.cc:2243] T af3ca9ac988e458cb7068c10c006d587 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.964993 1164 raft_consensus.cc:2272] T af3ca9ac988e458cb7068c10c006d587 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.965129 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 049cdecf12304211adbc1c1008dba989 (table default.Table_e6a96166bf33442989e15129d4d44631 [id=34edf86667d448c0b87e55627c67a704]) successfully deleted
I20251211 20:52:25.965520 1164 ts_tablet_manager.cc:1916] T af3ca9ac988e458cb7068c10c006d587 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.968380 1160 consensus_queue.cc:260] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:25.972232 1164 ts_tablet_manager.cc:1929] T af3ca9ac988e458cb7068c10c006d587 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:25.972368 1164 log.cc:1199] T af3ca9ac988e458cb7068c10c006d587 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/af3ca9ac988e458cb7068c10c006d587
I20251211 20:52:25.972716 1164 ts_tablet_manager.cc:1950] T af3ca9ac988e458cb7068c10c006d587 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:25.973762 1164 tablet_replica.cc:333] T f4db14d082d34aef857e106992bb8950 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:25.973982 1164 raft_consensus.cc:2243] T f4db14d082d34aef857e106992bb8950 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.974135 1164 raft_consensus.cc:2272] T f4db14d082d34aef857e106992bb8950 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.974674 1164 ts_tablet_manager.cc:1916] T f4db14d082d34aef857e106992bb8950 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.974859 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 171d5b17dfee49159925b476b08787a4
I20251211 20:52:25.975378 1160 ts_tablet_manager.cc:1434] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.046s user 0.001s sys 0.001s
I20251211 20:52:25.975600 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_4000fbd0bb614d0c902f539be2923421"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:25.976156 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4000fbd0bb614d0c902f539be2923421 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:25.978446 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet 171d5b17dfee49159925b476b08787a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:25.979168 1160 tablet_bootstrap.cc:492] T 732c76659c0540bc8b6909b430ffb146 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:25.980410 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 171d5b17dfee49159925b476b08787a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:25.981106 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet af3ca9ac988e458cb7068c10c006d587 (table default.Table_a11e64052cf34ba6a6a31f4edb417c19 [id=0afc1e783888443eb3f3e7848a04c207]) successfully deleted
I20251211 20:52:25.981875 1404 raft_consensus.cc:493] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:25.982095 1404 raft_consensus.cc:515] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:25.983500 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 28ef13be3b84410d9c786c43502a3478
I20251211 20:52:25.984252 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_740dd6a3832643c49f5f2bf34e924c41"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:25.984839 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_740dd6a3832643c49f5f2bf34e924c41 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:25.984979 1404 leader_election.cc:290] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:25.985675 847 leader_election.cc:343] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 8a064789f89c424ca698e52333d73eea
I20251211 20:52:25.986548 1337 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" instance_seqno: 1765486340578987) as {username='slave'} at 127.0.104.2:52485; Asking this server to re-register.
I20251211 20:52:25.982278 1164 ts_tablet_manager.cc:1929] T f4db14d082d34aef857e106992bb8950 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20251211 20:52:25.986754 847 leader_election.cc:343] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 8a064789f89c424ca698e52333d73eea
I20251211 20:52:25.986903 1164 log.cc:1199] T f4db14d082d34aef857e106992bb8950 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/f4db14d082d34aef857e106992bb8950
I20251211 20:52:25.986879 847 leader_election.cc:304] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:25.987771 959 heartbeater.cc:461] Registering TS with master...
I20251211 20:52:25.987973 589 master_service.cc:954] ReplaceTablet: received request to replace tablet d762b4afb5ff447dac3468393826205a from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:25.988040 1166 tablet_bootstrap.cc:492] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:25.988197 1166 ts_tablet_manager.cc:1403] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.038s user 0.004s sys 0.000s
I20251211 20:52:25.987301 1164 ts_tablet_manager.cc:1950] T f4db14d082d34aef857e106992bb8950 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:25.989228 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 0129162969d74f5da3c08a1e6a27ab22 (table default.Table_daeaf2f3dd24423e9060e4f09a5f4c99 [id=755ac98365be4def8589dcfd1f3e2cbf]) successfully deleted
I20251211 20:52:25.989400 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet 28ef13be3b84410d9c786c43502a3478 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:25.989562 1164 tablet_replica.cc:333] T 73aa2a3fb2f44bbca870fb7f4cf6087c P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:25.989754 1164 raft_consensus.cc:2243] T 73aa2a3fb2f44bbca870fb7f4cf6087c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.989920 1164 raft_consensus.cc:2272] T 73aa2a3fb2f44bbca870fb7f4cf6087c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.990582 1337 ts_manager.cc:194] Registered new tserver with Master: eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:25.991778 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fc8d485cc234fd490c487a8bc881bf4 with delete_type TABLET_DATA_DELETED (replaced by tablet 28ef13be3b84410d9c786c43502a3478) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:25.993265 1163 tablet_replica.cc:333] T 8dff13e22899452c8d4f2731b8dac7d5 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:25.993521 1163 raft_consensus.cc:2243] T 8dff13e22899452c8d4f2731b8dac7d5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:25.993541 1164 ts_tablet_manager.cc:1916] T 73aa2a3fb2f44bbca870fb7f4cf6087c P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:25.993810 1163 raft_consensus.cc:2272] T 8dff13e22899452c8d4f2731b8dac7d5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:25.994637 1160 tablet_bootstrap.cc:654] T 732c76659c0540bc8b6909b430ffb146 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:25.994877 1404 raft_consensus.cc:2749] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:25.994314 1166 raft_consensus.cc:359] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:25.995671 1166 raft_consensus.cc:385] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:25.995771 1166 raft_consensus.cc:740] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:25.995983 1166 consensus_queue.cc:260] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:25.997481 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 28ef13be3b84410d9c786c43502a3478 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:25.998888 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet f4db14d082d34aef857e106992bb8950 (table default.Table_9ac73058eb564b63871197be450065f0 [id=ffb336d92b05442b80d70cd1f419431c]) successfully deleted
I20251211 20:52:25.999955 1163 ts_tablet_manager.cc:1916] T 8dff13e22899452c8d4f2731b8dac7d5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.000000 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet f00f7c1f6a4e44ff89341b4a55cc93fb with delete_type TABLET_DATA_DELETED (replaced by tablet 88066d94d5d34e0fab20da9fb6ef178e) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:26.003940 1164 ts_tablet_manager.cc:1929] T 73aa2a3fb2f44bbca870fb7f4cf6087c P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.003978 1166 ts_tablet_manager.cc:1434] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.016s user 0.002s sys 0.000s
I20251211 20:52:26.004088 1164 log.cc:1199] T 73aa2a3fb2f44bbca870fb7f4cf6087c P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/73aa2a3fb2f44bbca870fb7f4cf6087c
I20251211 20:52:26.004449 1164 ts_tablet_manager.cc:1950] T 73aa2a3fb2f44bbca870fb7f4cf6087c P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:25.999401 588 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 33e1d8a576cd41b1a34778b1458df396 (table default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]). Expected version 2 got 0
I20251211 20:52:26.005332 1164 tablet_replica.cc:333] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:26.005540 1164 raft_consensus.cc:2243] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.005532 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_f12876c06fc04c63a2f23a6fe5b89f2b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:26.005929 1166 tablet_bootstrap.cc:492] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:26.006151 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f12876c06fc04c63a2f23a6fe5b89f2b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:26.007431 595 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 33e1d8a576cd41b1a34778b1458df396 (table default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]). Expected version 2 got 0
I20251211 20:52:26.008260 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_cedad3ae249f4796b8c3756c4f4d6f74"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:26.008850 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cedad3ae249f4796b8c3756c4f4d6f74 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:26.005700 1164 raft_consensus.cc:2272] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.010032 1166 tablet_bootstrap.cc:654] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.010025 1164 ts_tablet_manager.cc:1916] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.018105 1451 raft_consensus.cc:493] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:26.018438 1163 ts_tablet_manager.cc:1929] T 8dff13e22899452c8d4f2731b8dac7d5 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.018312 1451 raft_consensus.cc:515] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.019088 1451 leader_election.cc:290] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:26.019840 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f00f7c1f6a4e44ff89341b4a55cc93fb" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:26.020290 1164 ts_tablet_manager.cc:1929] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.020443 1164 log.cc:1199] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/ec7df7b11ce34b8489048e6acdcb4e94
W20251211 20:52:26.020422 983 leader_election.cc:343] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:26.020948 1163 log.cc:1199] T 8dff13e22899452c8d4f2731b8dac7d5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/8dff13e22899452c8d4f2731b8dac7d5
I20251211 20:52:26.021350 1164 ts_tablet_manager.cc:1950] T ec7df7b11ce34b8489048e6acdcb4e94 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:26.022441 1163 ts_tablet_manager.cc:1950] T 8dff13e22899452c8d4f2731b8dac7d5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:26.023684 1163 tablet_replica.cc:333] T c5b164dc0ce1461fae98d950f7c7a726 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:26.023715 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 8dff13e22899452c8d4f2731b8dac7d5 (table default.Table_f572100667d148e8a69e7930596f0f52 [id=cb97ccf262fd4c41a180267fd0678cec]) successfully deleted
I20251211 20:52:26.023988 1166 tablet_bootstrap.cc:492] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:26.024111 1163 raft_consensus.cc:2243] T c5b164dc0ce1461fae98d950f7c7a726 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
W20251211 20:52:26.024142 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 8139acb0ae324d0f973ce877edc1cd04 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8139acb0ae324d0f973ce877edc1cd04 already in progress: creating tablet
I20251211 20:52:26.024413 1163 raft_consensus.cc:2272] T c5b164dc0ce1461fae98d950f7c7a726 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.025485 1163 ts_tablet_manager.cc:1916] T c5b164dc0ce1461fae98d950f7c7a726 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.026645 1166 ts_tablet_manager.cc:1403] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.000s
I20251211 20:52:26.027499 1166 raft_consensus.cc:359] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.027843 1166 raft_consensus.cc:385] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.028138 1334 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" instance_seqno: 1765486341071617) as {username='slave'} at 127.0.104.3:47897; Asking this server to re-register.
I20251211 20:52:26.029323 1094 heartbeater.cc:461] Registering TS with master...
I20251211 20:52:26.029422 1404 consensus_queue.cc:1048] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:26.031879 1334 ts_manager.cc:194] Registered new tserver with Master: e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:26.032651 596 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 694518e6c1ae4032b06b99e9bde2e5dd (table default.Table_86450313ee0040bb87db3b340566266b [id=d49d5216afba41d68497f42fc72b1bf0]). Expected version 2 got 0
I20251211 20:52:26.033149 596 catalog_manager.cc:5654] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2). New cstate: current_term: 1 leader_uuid: "eb0f1933d9a74fccaa4618047fc05c41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:26.033689 596 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet a5eeb56709f046d9be92ac600ebc330e (table default.Table_f2ce490e0c5d445ab84989aae77c2f19 [id=f78e1da479af4d23a552af1c4fc55f81]). Expected version 1 got 0
I20251211 20:52:26.039011 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 73aa2a3fb2f44bbca870fb7f4cf6087c (table default.Table_3fca4e8f1b6a427d8d016e27f88ea1ce [id=4f93b23edd4548aeb6bcb8f92878907d]) successfully deleted
I20251211 20:52:26.044103 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet ec7df7b11ce34b8489048e6acdcb4e94 (table default.Table_02014bfadb7b434f960168f2db70b838 [id=14a0dbb8486246b09f3e06fc2120cd28]) successfully deleted
W20251211 20:52:26.044471 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 8139acb0ae324d0f973ce877edc1cd04 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8139acb0ae324d0f973ce877edc1cd04 already in progress: creating tablet
I20251211 20:52:26.051666 1466 catalog_manager.cc:1269] Loaded cluster ID: 521ed6ceec724e0ab1d4cde545c10f31
I20251211 20:52:26.051759 1466 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a: loading cluster ID for follower catalog manager: success
W20251211 20:52:26.054116 582 catalog_manager.cc:4952] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 171d5b17dfee49159925b476b08787a4 because the tablet was not found. No further retry: Not found: Tablet not found: 171d5b17dfee49159925b476b08787a4
I20251211 20:52:25.978435 739 tablet_service.cc:1552] Processing DeleteTablet for tablet 171d5b17dfee49159925b476b08787a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:25.983316 1168 tablet_bootstrap.cc:492] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:26.055830 1168 ts_tablet_manager.cc:1403] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.139s user 0.001s sys 0.012s
I20251211 20:52:26.056672 1168 raft_consensus.cc:359] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.056869 1168 raft_consensus.cc:385] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.056948 1168 raft_consensus.cc:740] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.057168 1168 consensus_queue.cc:260] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.057688 1168 ts_tablet_manager.cc:1434] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:26.057989 1168 tablet_bootstrap.cc:492] T c58241f7e1404a8c8db92c4f9a47d143 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:26.058790 1166 raft_consensus.cc:740] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.059113 1166 consensus_queue.cc:260] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.059718 1166 ts_tablet_manager.cc:1434] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.033s user 0.002s sys 0.000s
I20251211 20:52:25.989769 738 tablet_service.cc:1552] Processing DeleteTablet for tablet 28ef13be3b84410d9c786c43502a3478 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:26.013191 1420 consensus_queue.cc:1048] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.047s
I20251211 20:52:26.060253 1466 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a: acquiring CA information for follower catalog manager: success
I20251211 20:52:26.020148 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f00f7c1f6a4e44ff89341b4a55cc93fb" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:26.020442 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 9662efc97a53461d888a062463e80ee6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:26.021178 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 034f47d8564f4094a9d76aee2fea3119 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:26.020267 749 tablet_service.cc:1552] Processing DeleteTablet for tablet f4db14d082d34aef857e106992bb8950 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:26.062005 982 leader_election.cc:343] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:26.062842 982 leader_election.cc:304] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:26.063917 1168 tablet_bootstrap.cc:654] T c58241f7e1404a8c8db92c4f9a47d143 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.065114 1466 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251211 20:52:26.065802 1449 raft_consensus.cc:2749] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:25.931560 1165 ts_tablet_manager.cc:1950] T e41f052151624df39a3c83744870c94e P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:26.048035 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 55c40b2464f8444ca6c9fa0400f7f7f2
I20251211 20:52:26.067122 594 catalog_manager.cc:6724] ReplaceTablet: tablet 55c40b2464f8444ca6c9fa0400f7f7f2 deleted and replaced by tablet fb030a62796248869d46b682656eaeb0
I20251211 20:52:26.048309 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 804a11c8f7414eb1a6bcff2952be9ae2
I20251211 20:52:26.050166 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet bb5c5523e175406d9f6f9cd2736a2045
I20251211 20:52:26.068064 592 catalog_manager.cc:6724] ReplaceTablet: tablet bb5c5523e175406d9f6f9cd2736a2045 deleted and replaced by tablet cce44b80c97948e1af7ac68e77cccfd6
I20251211 20:52:26.050971 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 97ada0641165444a998db26796793507
I20251211 20:52:26.042789 665 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 171d5b17dfee49159925b476b08787a4
I20251211 20:52:26.066520 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet e41f052151624df39a3c83744870c94e (table default.Table_bcccedcd97444712a0f74e76db8cd74a [id=3de8f4cc61014e96aa64e7d4dd768096]) successfully deleted
I20251211 20:52:26.069593 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 55c40b2464f8444ca6c9fa0400f7f7f2 with delete_type TABLET_DATA_DELETED (replaced by tablet fb030a62796248869d46b682656eaeb0) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:26.069926 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 804a11c8f7414eb1a6bcff2952be9ae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:26.070431 749 tablet_service.cc:1552] Processing DeleteTablet for tablet bb5c5523e175406d9f6f9cd2736a2045 with delete_type TABLET_DATA_DELETED (replaced by tablet cce44b80c97948e1af7ac68e77cccfd6) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:26.071417 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 97ada0641165444a998db26796793507 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:26.071463 1016 tablet_service.cc:1552] Processing DeleteTablet for tablet bb5c5523e175406d9f6f9cd2736a2045 with delete_type TABLET_DATA_DELETED (replaced by tablet cce44b80c97948e1af7ac68e77cccfd6) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:26.071743 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet 804a11c8f7414eb1a6bcff2952be9ae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:26.072008 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 55c40b2464f8444ca6c9fa0400f7f7f2 with delete_type TABLET_DATA_DELETED (replaced by tablet fb030a62796248869d46b682656eaeb0) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:26.072192 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet 97ada0641165444a998db26796793507 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:26.072438 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet 27466459d8e6411ab87ac474fb617735 with delete_type TABLET_DATA_DELETED (replaced by tablet 8feee01fc0aa4b32be926ff2b617b894) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:26.072984 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 171d5b17dfee49159925b476b08787a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:26.073774 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet 171d5b17dfee49159925b476b08787a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:26.074831 749 tablet_service.cc:1505] Processing CreateTablet for tablet 3fc173c396ff4af5a8589cff18165e85 (DEFAULT_TABLE table=default.Table_2a8eb0e48b1e42d59c215333429cac46 [id=a455f6ef64ae44f78972d14b2ea9654c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.075265 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fc173c396ff4af5a8589cff18165e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.077421 1166 tablet_bootstrap.cc:492] T d824bc02b25842d18f28f711eef6669f P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:26.082108 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 28ef13be3b84410d9c786c43502a3478 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 28ef13be3b84410d9c786c43502a3478 already in progress: creating tablet
I20251211 20:52:26.084525 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 3fc173c396ff4af5a8589cff18165e85 (DEFAULT_TABLE table=default.Table_2a8eb0e48b1e42d59c215333429cac46 [id=a455f6ef64ae44f78972d14b2ea9654c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.084950 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fc173c396ff4af5a8589cff18165e85. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:26.088692 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 8139acb0ae324d0f973ce877edc1cd04 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8139acb0ae324d0f973ce877edc1cd04 already in progress: creating tablet
W20251211 20:52:26.091919 581 catalog_manager.cc:4952] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 171d5b17dfee49159925b476b08787a4 because the tablet was not found. No further retry: Not found: Tablet not found: 171d5b17dfee49159925b476b08787a4
I20251211 20:52:26.096974 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 8a064789f89c424ca698e52333d73eea with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:26.097950 1164 tablet_replica.cc:333] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:26.098176 1164 raft_consensus.cc:2243] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.098381 1164 raft_consensus.cc:2272] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.099097 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet c58241f7e1404a8c8db92c4f9a47d143
I20251211 20:52:26.099184 1164 ts_tablet_manager.cc:1916] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.100893 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet c58241f7e1404a8c8db92c4f9a47d143 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:26.102013 749 tablet_service.cc:1552] Processing DeleteTablet for tablet c58241f7e1404a8c8db92c4f9a47d143 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:26.102056 912 raft_consensus.cc:3060] T 694518e6c1ae4032b06b99e9bde2e5dd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:26.103753 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 28ef13be3b84410d9c786c43502a3478 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 28ef13be3b84410d9c786c43502a3478 already in progress: creating tablet
I20251211 20:52:26.104494 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 55c40b2464f8444ca6c9fa0400f7f7f2 with delete_type TABLET_DATA_DELETED (replaced by tablet fb030a62796248869d46b682656eaeb0) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:26.105574 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 804a11c8f7414eb1a6bcff2952be9ae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:26.105840 1420 consensus_queue.cc:1048] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:26.106026 1165 tablet_replica.cc:333] T 9662efc97a53461d888a062463e80ee6 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:26.106189 1165 raft_consensus.cc:2243] T 9662efc97a53461d888a062463e80ee6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.106381 1165 raft_consensus.cc:2272] T 9662efc97a53461d888a062463e80ee6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.105584 1164 ts_tablet_manager.cc:1929] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.106897 1164 log.cc:1199] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/8a064789f89c424ca698e52333d73eea
I20251211 20:52:26.107276 1164 ts_tablet_manager.cc:1950] T 8a064789f89c424ca698e52333d73eea P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:26.107738 889 tablet_service.cc:1552] Processing DeleteTablet for tablet bb5c5523e175406d9f6f9cd2736a2045 with delete_type TABLET_DATA_DELETED (replaced by tablet cce44b80c97948e1af7ac68e77cccfd6) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:26.109417 749 tablet_service.cc:1505] Processing CreateTablet for tablet b072b759d1254859b0b8d401e3078792 (DEFAULT_TABLE table=default.Table_287197890af94935922681dd3ac7d185 [id=a137f0ddf52145a99c62444602ab3fdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.109889 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b072b759d1254859b0b8d401e3078792. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.111289 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet d762b4afb5ff447dac3468393826205a
I20251211 20:52:26.111788 589 catalog_manager.cc:6724] ReplaceTablet: tablet d762b4afb5ff447dac3468393826205a deleted and replaced by tablet 2197822e0f11484f9679c4858626ac2c
I20251211 20:52:26.112227 1165 ts_tablet_manager.cc:1916] T 9662efc97a53461d888a062463e80ee6 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.113111 741 tablet_service.cc:1552] Processing DeleteTablet for tablet d762b4afb5ff447dac3468393826205a with delete_type TABLET_DATA_DELETED (replaced by tablet 2197822e0f11484f9679c4858626ac2c) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:26.114187 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet d762b4afb5ff447dac3468393826205a with delete_type TABLET_DATA_DELETED (replaced by tablet 2197822e0f11484f9679c4858626ac2c) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:26.117471 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 97ada0641165444a998db26796793507 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:26.122052 1023 tablet_service.cc:1505] Processing CreateTablet for tablet b072b759d1254859b0b8d401e3078792 (DEFAULT_TABLE table=default.Table_287197890af94935922681dd3ac7d185 [id=a137f0ddf52145a99c62444602ab3fdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.122591 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b072b759d1254859b0b8d401e3078792. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.123032 1163 ts_tablet_manager.cc:1929] T c5b164dc0ce1461fae98d950f7c7a726 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.123174 1163 log.cc:1199] T c5b164dc0ce1461fae98d950f7c7a726 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/c5b164dc0ce1461fae98d950f7c7a726
I20251211 20:52:26.124145 1165 ts_tablet_manager.cc:1929] T 9662efc97a53461d888a062463e80ee6 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.124284 1165 log.cc:1199] T 9662efc97a53461d888a062463e80ee6 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/9662efc97a53461d888a062463e80ee6
I20251211 20:52:26.124637 1165 ts_tablet_manager.cc:1950] T 9662efc97a53461d888a062463e80ee6 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:26.124821 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 171d5b17dfee49159925b476b08787a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:26.125223 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 9662efc97a53461d888a062463e80ee6 (table default.Table_9d6a8e006ed241819eaecb22446277f5 [id=655f70b9f7e0400aa42179b922af2fa2]) successfully deleted
I20251211 20:52:26.125623 1165 tablet_replica.cc:333] T 034f47d8564f4094a9d76aee2fea3119 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:26.125833 1165 raft_consensus.cc:2243] T 034f47d8564f4094a9d76aee2fea3119 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.126003 1165 raft_consensus.cc:2272] T 034f47d8564f4094a9d76aee2fea3119 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.126214 589 master_service.cc:954] ReplaceTablet: received request to replace tablet bb5c5523e175406d9f6f9cd2736a2045 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:26.126755 1165 ts_tablet_manager.cc:1916] T 034f47d8564f4094a9d76aee2fea3119 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.129496 589 master_service.cc:954] ReplaceTablet: received request to replace tablet f93006227cc6489e91f3593e11789b54 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:26.130317 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_8adf3dfe2fb0445385c730198775d1e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:26.130692 741 tablet_service.cc:1505] Processing CreateTablet for tablet 5bd801124df84a6c96f4912993f1bdfd (DEFAULT_TABLE table=default.Table_87e2d779c1a74a57a8eb93d7163c1660 [id=44f2f2bf53af4b038ced294f20db9e48]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.131112 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bd801124df84a6c96f4912993f1bdfd. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:26.130904 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8adf3dfe2fb0445385c730198775d1e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:26.135054 590 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 694518e6c1ae4032b06b99e9bde2e5dd (table default.Table_86450313ee0040bb87db3b340566266b [id=d49d5216afba41d68497f42fc72b1bf0]). Expected version 2 got 0
I20251211 20:52:26.143193 1168 tablet_bootstrap.cc:492] T c58241f7e1404a8c8db92c4f9a47d143 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:26.143416 1168 ts_tablet_manager.cc:1403] T c58241f7e1404a8c8db92c4f9a47d143 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.085s user 0.002s sys 0.007s
I20251211 20:52:26.145273 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_943bd9d0a2d848e2bb986aee9bc1d718" } new_table_name: "default.Table_4a0345a7a9eb4dfc89d48517fbe15ad1" modify_external_catalogs: true
I20251211 20:52:26.146047 1165 ts_tablet_manager.cc:1929] T 034f47d8564f4094a9d76aee2fea3119 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.146190 1165 log.cc:1199] T 034f47d8564f4094a9d76aee2fea3119 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/034f47d8564f4094a9d76aee2fea3119
I20251211 20:52:26.146517 1165 ts_tablet_manager.cc:1950] T 034f47d8564f4094a9d76aee2fea3119 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:26.146973 592 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 694518e6c1ae4032b06b99e9bde2e5dd (table default.Table_86450313ee0040bb87db3b340566266b [id=d49d5216afba41d68497f42fc72b1bf0]). Expected version 2 got 0
I20251211 20:52:26.147082 1165 tablet_replica.cc:333] T f4db14d082d34aef857e106992bb8950 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:26.147253 1165 raft_consensus.cc:2243] T f4db14d082d34aef857e106992bb8950 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.147480 1165 raft_consensus.cc:2272] T f4db14d082d34aef857e106992bb8950 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.147989 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 034f47d8564f4094a9d76aee2fea3119 (table default.Table_b5a3abc502a64d6083d78344400b5eeb [id=c58ec33f45c348329be4acee3d5f4f6f]) successfully deleted
I20251211 20:52:26.151384 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_51f7abf29f7247eb8a83e8b9ef1d26c5" } new_table_name: "default.Table_5e65e4dae9454b3e993a46b2255d955d" modify_external_catalogs: true
I20251211 20:52:26.153319 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_1d1c44cc71394478bc3f93b23ca3dad4" } new_table_name: "default.Table_0ceac14066b444dfa440c23e489e574a" modify_external_catalogs: true
I20251211 20:52:26.156392 1165 ts_tablet_manager.cc:1916] T f4db14d082d34aef857e106992bb8950 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.156397 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_383cabcd12a74a14bc3d7dec7c236a19" } new_table_name: "default.Table_c27589191ac249f4a9460f5bbae2611b" modify_external_catalogs: true
I20251211 20:52:26.158546 1168 raft_consensus.cc:359] T c58241f7e1404a8c8db92c4f9a47d143 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.162252 592 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 694518e6c1ae4032b06b99e9bde2e5dd (table default.Table_86450313ee0040bb87db3b340566266b [id=d49d5216afba41d68497f42fc72b1bf0]). Expected version 2 got 0
I20251211 20:52:26.163040 1168 raft_consensus.cc:385] T c58241f7e1404a8c8db92c4f9a47d143 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.163305 1168 raft_consensus.cc:740] T c58241f7e1404a8c8db92c4f9a47d143 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.163610 590 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 694518e6c1ae4032b06b99e9bde2e5dd (table default.Table_86450313ee0040bb87db3b340566266b [id=d49d5216afba41d68497f42fc72b1bf0]). Expected version 2 got 0
I20251211 20:52:26.163784 1168 consensus_queue.cc:260] T c58241f7e1404a8c8db92c4f9a47d143 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.138130 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 5bd801124df84a6c96f4912993f1bdfd (DEFAULT_TABLE table=default.Table_87e2d779c1a74a57a8eb93d7163c1660 [id=44f2f2bf53af4b038ced294f20db9e48]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.164920 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bd801124df84a6c96f4912993f1bdfd. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:26.170213 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a2058b67ddf1413db338bfa2f64e7a77 has no consensus state
W20251211 20:52:26.170714 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bb0fc179a337469eae056c5bb3815e6b has no consensus state
I20251211 20:52:26.152328 1166 tablet_bootstrap.cc:654] T d824bc02b25842d18f28f711eef6669f P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.176676 594 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 01ada50100bc48e39d1bc67082da750e (table default.Table_18d19a317a9c4656b4c764df4a64dbea [id=a24d8c58b9d84cdea2b278ae9c6e0ba5]). Expected version 2 got 0
I20251211 20:52:26.176846 1334 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" instance_seqno: 1765486340106790) as {username='slave'} at 127.0.104.1:44799; Asking this server to re-register.
I20251211 20:52:26.180626 597 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 01ada50100bc48e39d1bc67082da750e (table default.Table_18d19a317a9c4656b4c764df4a64dbea [id=a24d8c58b9d84cdea2b278ae9c6e0ba5]). Expected version 2 got 0
I20251211 20:52:26.181993 822 heartbeater.cc:461] Registering TS with master...
I20251211 20:52:26.182507 1404 consensus_queue.cc:1048] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:26.123564 1163 ts_tablet_manager.cc:1950] T c5b164dc0ce1461fae98d950f7c7a726 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:26.183670 1334 ts_manager.cc:194] Registered new tserver with Master: 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:26.184680 594 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 01ada50100bc48e39d1bc67082da750e (table default.Table_18d19a317a9c4656b4c764df4a64dbea [id=a24d8c58b9d84cdea2b278ae9c6e0ba5]). Expected version 2 got 0
I20251211 20:52:26.184973 594 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet a5eeb56709f046d9be92ac600ebc330e (table default.Table_f2ce490e0c5d445ab84989aae77c2f19 [id=f78e1da479af4d23a552af1c4fc55f81]). Expected version 1 got 0
I20251211 20:52:26.185261 594 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 694518e6c1ae4032b06b99e9bde2e5dd (table default.Table_86450313ee0040bb87db3b340566266b [id=d49d5216afba41d68497f42fc72b1bf0]). Expected version 2 got 0
I20251211 20:52:26.188468 1168 ts_tablet_manager.cc:1434] T c58241f7e1404a8c8db92c4f9a47d143 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.021s user 0.000s sys 0.002s
I20251211 20:52:26.188509 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet c5b164dc0ce1461fae98d950f7c7a726 (table default.Table_b06a3971c3be404e9787e05466931423 [id=9c24845e4efe475fa792a51ce8a66be6]) successfully deleted
I20251211 20:52:26.188900 1168 tablet_bootstrap.cc:492] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:26.191179 1168 tablet_bootstrap.cc:654] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.191794 1163 tablet_replica.cc:333] T 1c04187b8ed146c289a6b17d62b07199 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:26.191999 1163 raft_consensus.cc:2243] T 1c04187b8ed146c289a6b17d62b07199 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.192201 1163 raft_consensus.cc:2272] T 1c04187b8ed146c289a6b17d62b07199 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.193192 1163 ts_tablet_manager.cc:1916] T 1c04187b8ed146c289a6b17d62b07199 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.198174 1166 tablet_bootstrap.cc:492] T d824bc02b25842d18f28f711eef6669f P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:26.198345 1166 ts_tablet_manager.cc:1403] T d824bc02b25842d18f28f711eef6669f P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.121s user 0.003s sys 0.000s
I20251211 20:52:26.198750 1165 ts_tablet_manager.cc:1929] T f4db14d082d34aef857e106992bb8950 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.198911 1165 log.cc:1199] T f4db14d082d34aef857e106992bb8950 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/f4db14d082d34aef857e106992bb8950
I20251211 20:52:26.200004 1165 ts_tablet_manager.cc:1950] T f4db14d082d34aef857e106992bb8950 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:26.201179 741 tablet_service.cc:1505] Processing CreateTablet for tablet 60d966c0397c4a4ea3718445e87e9703 (DEFAULT_TABLE table=default.Table_33d44c934d124375827a2987a3ec3d3d [id=cae3f6884694487e82fc81328e9527d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.201272 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet f4db14d082d34aef857e106992bb8950 (table default.Table_9ac73058eb564b63871197be450065f0 [id=ffb336d92b05442b80d70cd1f419431c]) successfully deleted
W20251211 20:52:26.201547 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 55c40b2464f8444ca6c9fa0400f7f7f2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55c40b2464f8444ca6c9fa0400f7f7f2 already in progress: creating tablet
I20251211 20:52:26.201622 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60d966c0397c4a4ea3718445e87e9703. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.201639 1166 raft_consensus.cc:359] T d824bc02b25842d18f28f711eef6669f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.201821 1166 raft_consensus.cc:385] T d824bc02b25842d18f28f711eef6669f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.201911 1166 raft_consensus.cc:740] T d824bc02b25842d18f28f711eef6669f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.202148 1166 consensus_queue.cc:260] T d824bc02b25842d18f28f711eef6669f P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.202745 1166 ts_tablet_manager.cc:1434] T d824bc02b25842d18f28f711eef6669f P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:26.203023 1166 tablet_bootstrap.cc:492] T dd425601d89148eb95525f796a3abf1e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:26.203372 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 773cf53745a64f62817113b33e3182e7 consensus state has no leader
W20251211 20:52:26.203811 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c4e048ca28824329b924845ccf5067cf consensus state has no leader
I20251211 20:52:26.204561 595 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_502b11716c454b5dae33c715fd69dc1f" } new_table_name: "default.Table_ea25e2f3ec124ff28f04f72f6f7121e1" modify_external_catalogs: true
I20251211 20:52:26.205843 1166 tablet_bootstrap.cc:654] T dd425601d89148eb95525f796a3abf1e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.206573 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_03b10e1dfcd94a08a73fd396ce0b6f98"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:26.207134 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_03b10e1dfcd94a08a73fd396ce0b6f98 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:26.208640 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b0e00b07650849958a1a4486eeb3c660"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:26.209144 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b0e00b07650849958a1a4486eeb3c660 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:26.209944 1165 tablet_replica.cc:333] T c58241f7e1404a8c8db92c4f9a47d143 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:26.210146 1165 raft_consensus.cc:2243] T c58241f7e1404a8c8db92c4f9a47d143 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.210312 1165 raft_consensus.cc:2272] T c58241f7e1404a8c8db92c4f9a47d143 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:26.210314 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 804a11c8f7414eb1a6bcff2952be9ae2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 804a11c8f7414eb1a6bcff2952be9ae2 already in progress: creating tablet
W20251211 20:52:26.210541 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet bb5c5523e175406d9f6f9cd2736a2045 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb5c5523e175406d9f6f9cd2736a2045 already in progress: creating tablet
W20251211 20:52:26.210734 581 catalog_manager.cc:4952] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 97ada0641165444a998db26796793507 because the tablet was not found. No further retry: Not found: Tablet not found: 97ada0641165444a998db26796793507
W20251211 20:52:26.210932 581 catalog_manager.cc:4952] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 171d5b17dfee49159925b476b08787a4 because the tablet was not found. No further retry: Not found: Tablet not found: 171d5b17dfee49159925b476b08787a4
I20251211 20:52:26.211426 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_8b66a5217c494c8ebfa10a2b986e5e78"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:26.211721 1165 ts_tablet_manager.cc:1916] T c58241f7e1404a8c8db92c4f9a47d143 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:26.212036 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8b66a5217c494c8ebfa10a2b986e5e78 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:26.213423 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_63e0e08eb63c40b9aece6be45b6d4944"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:26.214211 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_63e0e08eb63c40b9aece6be45b6d4944 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:26.216192 1163 ts_tablet_manager.cc:1929] T 1c04187b8ed146c289a6b17d62b07199 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.216352 1163 log.cc:1199] T 1c04187b8ed146c289a6b17d62b07199 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/1c04187b8ed146c289a6b17d62b07199
I20251211 20:52:26.216737 1163 ts_tablet_manager.cc:1950] T 1c04187b8ed146c289a6b17d62b07199 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:26.216918 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 60d966c0397c4a4ea3718445e87e9703 (DEFAULT_TABLE table=default.Table_33d44c934d124375827a2987a3ec3d3d [id=cae3f6884694487e82fc81328e9527d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.217321 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60d966c0397c4a4ea3718445e87e9703. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.219041 1165 ts_tablet_manager.cc:1929] T c58241f7e1404a8c8db92c4f9a47d143 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.219188 1165 log.cc:1199] T c58241f7e1404a8c8db92c4f9a47d143 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/c58241f7e1404a8c8db92c4f9a47d143
I20251211 20:52:26.220356 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 97ada0641165444a998db26796793507 (DEFAULT_TABLE table=default.Table_a66a87006b684bbd87431e2aad969dcf [id=021234dec4df4bcb9ec0fee4cb1f887e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.220804 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97ada0641165444a998db26796793507. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.221483 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 3cd7a465249246c28f07b8fb2e42675f (DEFAULT_TABLE table=default.Table_227fbf051d5643528315ca00194583bc [id=1515561ab78d46f6b5aa37a20d90d3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.221890 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cd7a465249246c28f07b8fb2e42675f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.222828 1166 tablet_bootstrap.cc:492] T dd425601d89148eb95525f796a3abf1e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:26.222996 1166 ts_tablet_manager.cc:1403] T dd425601d89148eb95525f796a3abf1e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.000s
I20251211 20:52:26.223892 1166 raft_consensus.cc:359] T dd425601d89148eb95525f796a3abf1e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.223932 1016 tablet_service.cc:1505] Processing CreateTablet for tablet c8e17645a26947ab8c01f9b093fa422c (DEFAULT_TABLE table=default.Table_45aae1bb922143a2b0fb90dd4b385f0c [id=5d98f157d4eb47bf825a3ecd427eb2e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.224107 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_dd84a1d3fe964c76b146f798f6781baa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:26.224309 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 1c04187b8ed146c289a6b17d62b07199 (table default.Table_2bdd381fdf6443da869b647ddb92699b [id=70ed017f68a140ceaa39557fbe86aa0b]) successfully deleted
W20251211 20:52:26.224720 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dd84a1d3fe964c76b146f798f6781baa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:26.225337 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 171d5b17dfee49159925b476b08787a4 because the tablet was not found. No further retry: Not found: Tablet not found: 171d5b17dfee49159925b476b08787a4
I20251211 20:52:26.225849 1024 tablet_service.cc:1505] Processing CreateTablet for tablet fb2d30147cd74d1893783be1e1694d1f (DEFAULT_TABLE table=default.Table_1bf756c0a806499fbfd10745831a653f [id=c5b5110e2f1f4eb29d233b6a9eef4737]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.226629 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_fdd4d94139714e3c954d6daf3c82a649" } modify_external_catalogs: true
I20251211 20:52:26.226711 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_cc48404bbbf24a7caae7641a8d42c247" } modify_external_catalogs: true
I20251211 20:52:26.226820 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_fdd4d94139714e3c954d6daf3c82a649" } modify_external_catalogs: true
I20251211 20:52:26.226882 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_cc48404bbbf24a7caae7641a8d42c247" } modify_external_catalogs: true
I20251211 20:52:26.228427 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_31d573e7af144e9997dfe480f8466902"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:26.228919 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_31d573e7af144e9997dfe480f8466902 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:26.230660 889 tablet_service.cc:1505] Processing CreateTablet for tablet 3fc173c396ff4af5a8589cff18165e85 (DEFAULT_TABLE table=default.Table_2a8eb0e48b1e42d59c215333429cac46 [id=a455f6ef64ae44f78972d14b2ea9654c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.230902 1016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8e17645a26947ab8c01f9b093fa422c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.231119 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fc173c396ff4af5a8589cff18165e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.231127 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb2d30147cd74d1893783be1e1694d1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.230867 1166 raft_consensus.cc:385] T dd425601d89148eb95525f796a3abf1e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.231365 1166 raft_consensus.cc:740] T dd425601d89148eb95525f796a3abf1e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
W20251211 20:52:26.232482 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 28ef13be3b84410d9c786c43502a3478 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 28ef13be3b84410d9c786c43502a3478 already in progress: creating tablet
I20251211 20:52:26.235213 1166 consensus_queue.cc:260] T dd425601d89148eb95525f796a3abf1e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.236320 1166 ts_tablet_manager.cc:1434] T dd425601d89148eb95525f796a3abf1e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.013s user 0.003s sys 0.000s
I20251211 20:52:26.236663 1166 tablet_bootstrap.cc:492] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:26.238090 1163 tablet_replica.cc:333] T 5fc8d485cc234fd490c487a8bc881bf4 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:26.238294 1163 raft_consensus.cc:2243] T 5fc8d485cc234fd490c487a8bc881bf4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.239099 1160 tablet_bootstrap.cc:492] T 732c76659c0540bc8b6909b430ffb146 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:26.239236 1163 raft_consensus.cc:2272] T 5fc8d485cc234fd490c487a8bc881bf4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.239332 1160 ts_tablet_manager.cc:1403] T 732c76659c0540bc8b6909b430ffb146 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.260s user 0.009s sys 0.020s
I20251211 20:52:26.240149 1160 raft_consensus.cc:359] T 732c76659c0540bc8b6909b430ffb146 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.240358 1160 raft_consensus.cc:385] T 732c76659c0540bc8b6909b430ffb146 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.240432 1160 raft_consensus.cc:740] T 732c76659c0540bc8b6909b430ffb146 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.240576 889 tablet_service.cc:1552] Processing DeleteTablet for tablet c58241f7e1404a8c8db92c4f9a47d143 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:26.240945 1160 consensus_queue.cc:260] T 732c76659c0540bc8b6909b430ffb146 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.245033 1160 ts_tablet_manager.cc:1434] T 732c76659c0540bc8b6909b430ffb146 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20251211 20:52:26.245278 889 tablet_service.cc:1552] Processing DeleteTablet for tablet d762b4afb5ff447dac3468393826205a with delete_type TABLET_DATA_DELETED (replaced by tablet 2197822e0f11484f9679c4858626ac2c) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:26.248675 1166 tablet_bootstrap.cc:654] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.248760 1160 tablet_bootstrap.cc:492] T 9662efc97a53461d888a062463e80ee6 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:26.248986 889 tablet_service.cc:1505] Processing CreateTablet for tablet b072b759d1254859b0b8d401e3078792 (DEFAULT_TABLE table=default.Table_287197890af94935922681dd3ac7d185 [id=a137f0ddf52145a99c62444602ab3fdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.249506 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b072b759d1254859b0b8d401e3078792. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.250890 1160 tablet_bootstrap.cc:654] T 9662efc97a53461d888a062463e80ee6 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.256578 880 tablet_service.cc:1505] Processing CreateTablet for tablet 5bd801124df84a6c96f4912993f1bdfd (DEFAULT_TABLE table=default.Table_87e2d779c1a74a57a8eb93d7163c1660 [id=44f2f2bf53af4b038ced294f20db9e48]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.256994 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bd801124df84a6c96f4912993f1bdfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.261533 1163 ts_tablet_manager.cc:1916] T 5fc8d485cc234fd490c487a8bc881bf4 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.263751 744 tablet_service.cc:1505] Processing CreateTablet for tablet c8e17645a26947ab8c01f9b093fa422c (DEFAULT_TABLE table=default.Table_45aae1bb922143a2b0fb90dd4b385f0c [id=5d98f157d4eb47bf825a3ecd427eb2e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.264199 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8e17645a26947ab8c01f9b093fa422c. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:26.268297 595 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8229b704e7464b30b7d2902a938ce33b consensus state has no leader
I20251211 20:52:26.268682 1165 ts_tablet_manager.cc:1950] T c58241f7e1404a8c8db92c4f9a47d143 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:26.269673 739 tablet_service.cc:1505] Processing CreateTablet for tablet fb2d30147cd74d1893783be1e1694d1f (DEFAULT_TABLE table=default.Table_1bf756c0a806499fbfd10745831a653f [id=c5b5110e2f1f4eb29d233b6a9eef4737]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.270071 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet c58241f7e1404a8c8db92c4f9a47d143 (table default.Table_3570af5d8bff402ab3140ae51bae174c [id=90602e87d8b6425295bb04a5fca84ba7]) successfully deleted
I20251211 20:52:26.270200 1166 tablet_bootstrap.cc:492] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:26.270500 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb2d30147cd74d1893783be1e1694d1f. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:26.270973 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet d762b4afb5ff447dac3468393826205a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d762b4afb5ff447dac3468393826205a already in progress: creating tablet
I20251211 20:52:26.273018 1166 ts_tablet_manager.cc:1403] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.005s
I20251211 20:52:26.274232 1160 tablet_bootstrap.cc:492] T 9662efc97a53461d888a062463e80ee6 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:26.274384 1160 ts_tablet_manager.cc:1403] T 9662efc97a53461d888a062463e80ee6 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.000s
I20251211 20:52:26.274101 738 tablet_service.cc:1505] Processing CreateTablet for tablet 3cd7a465249246c28f07b8fb2e42675f (DEFAULT_TABLE table=default.Table_227fbf051d5643528315ca00194583bc [id=1515561ab78d46f6b5aa37a20d90d3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.274569 880 tablet_service.cc:1505] Processing CreateTablet for tablet 60d966c0397c4a4ea3718445e87e9703 (DEFAULT_TABLE table=default.Table_33d44c934d124375827a2987a3ec3d3d [id=cae3f6884694487e82fc81328e9527d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.274986 738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cd7a465249246c28f07b8fb2e42675f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.274997 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60d966c0397c4a4ea3718445e87e9703. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.275188 1160 raft_consensus.cc:359] T 9662efc97a53461d888a062463e80ee6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.275385 1160 raft_consensus.cc:385] T 9662efc97a53461d888a062463e80ee6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.275485 1160 raft_consensus.cc:740] T 9662efc97a53461d888a062463e80ee6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.275789 1160 consensus_queue.cc:260] T 9662efc97a53461d888a062463e80ee6 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.277609 1166 raft_consensus.cc:359] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.277814 1166 raft_consensus.cc:385] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.277901 1166 raft_consensus.cc:740] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.279764 1166 consensus_queue.cc:260] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.279958 1160 ts_tablet_manager.cc:1434] T 9662efc97a53461d888a062463e80ee6 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251211 20:52:26.283495 741 tablet_service.cc:1505] Processing CreateTablet for tablet 97ada0641165444a998db26796793507 (DEFAULT_TABLE table=default.Table_a66a87006b684bbd87431e2aad969dcf [id=021234dec4df4bcb9ec0fee4cb1f887e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.284554 894 tablet_service.cc:1505] Processing CreateTablet for tablet 97ada0641165444a998db26796793507 (DEFAULT_TABLE table=default.Table_a66a87006b684bbd87431e2aad969dcf [id=021234dec4df4bcb9ec0fee4cb1f887e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.284972 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97ada0641165444a998db26796793507. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.285243 1166 ts_tablet_manager.cc:1434] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20251211 20:52:26.286041 1160 tablet_bootstrap.cc:492] T 034f47d8564f4094a9d76aee2fea3119 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:26.287834 1160 tablet_bootstrap.cc:654] T 034f47d8564f4094a9d76aee2fea3119 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.288480 1166 tablet_bootstrap.cc:492] T 91602f30b4464e35b16fa3dcac445b49 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:26.288856 596 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 01ada50100bc48e39d1bc67082da750e (table default.Table_18d19a317a9c4656b4c764df4a64dbea [id=a24d8c58b9d84cdea2b278ae9c6e0ba5]). Expected version 2 got 0
I20251211 20:52:26.295118 890 tablet_service.cc:1505] Processing CreateTablet for tablet 3cd7a465249246c28f07b8fb2e42675f (DEFAULT_TABLE table=default.Table_227fbf051d5643528315ca00194583bc [id=1515561ab78d46f6b5aa37a20d90d3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.295599 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cd7a465249246c28f07b8fb2e42675f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.301019 596 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 01ada50100bc48e39d1bc67082da750e (table default.Table_18d19a317a9c4656b4c764df4a64dbea [id=a24d8c58b9d84cdea2b278ae9c6e0ba5]). Expected version 2 got 0
I20251211 20:52:26.301265 596 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet dd425601d89148eb95525f796a3abf1e (table default.Table_d4afd6ed615a41ac8f6714c8aac27857 [id=c842792b01cb4c689fcfd3d796151508]). Expected version 1 got 0
I20251211 20:52:26.301409 596 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 8308ad46551447439a248d20ed1e5922 (table default.Table_5d36b67ed10545eb97ac64602292ca7b [id=17b2359416c04980857565ab4df594bd]). Expected version 2 got 0
I20251211 20:52:26.302744 894 tablet_service.cc:1505] Processing CreateTablet for tablet c8e17645a26947ab8c01f9b093fa422c (DEFAULT_TABLE table=default.Table_45aae1bb922143a2b0fb90dd4b385f0c [id=5d98f157d4eb47bf825a3ecd427eb2e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.303232 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8e17645a26947ab8c01f9b093fa422c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.303452 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet d824bc02b25842d18f28f711eef6669f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:26.307210 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97ada0641165444a998db26796793507. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.310189 1166 tablet_bootstrap.cc:654] T 91602f30b4464e35b16fa3dcac445b49 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.316455 1163 ts_tablet_manager.cc:1929] T 5fc8d485cc234fd490c487a8bc881bf4 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.316780 1163 log.cc:1199] T 5fc8d485cc234fd490c487a8bc881bf4 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/5fc8d485cc234fd490c487a8bc881bf4
I20251211 20:52:26.317186 1163 ts_tablet_manager.cc:1950] T 5fc8d485cc234fd490c487a8bc881bf4 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:26.318272 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 5fc8d485cc234fd490c487a8bc881bf4 (table default.Table_88dcd36b4447481487bc918a05511862 [id=d9453cea23a8430c98a3317856b3cec3]) successfully deleted
I20251211 20:52:26.318348 1163 tablet_replica.cc:333] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:26.318543 1163 raft_consensus.cc:2243] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.318730 1163 raft_consensus.cc:2272] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.319345 1163 ts_tablet_manager.cc:1916] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.332444 1168 tablet_bootstrap.cc:492] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:26.332605 1168 ts_tablet_manager.cc:1403] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.144s user 0.021s sys 0.004s
I20251211 20:52:26.333330 1168 raft_consensus.cc:359] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.333511 1168 raft_consensus.cc:385] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.333609 1168 raft_consensus.cc:740] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.333882 1168 consensus_queue.cc:260] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.334357 1168 ts_tablet_manager.cc:1434] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:26.335003 1168 tablet_bootstrap.cc:492] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:26.335367 1160 tablet_bootstrap.cc:492] T 034f47d8564f4094a9d76aee2fea3119 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:26.335546 1160 ts_tablet_manager.cc:1403] T 034f47d8564f4094a9d76aee2fea3119 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.050s user 0.007s sys 0.000s
I20251211 20:52:26.336338 1160 raft_consensus.cc:359] T 034f47d8564f4094a9d76aee2fea3119 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.336563 1160 raft_consensus.cc:385] T 034f47d8564f4094a9d76aee2fea3119 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.336630 1160 raft_consensus.cc:740] T 034f47d8564f4094a9d76aee2fea3119 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.336702 1168 tablet_bootstrap.cc:654] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.336871 1160 consensus_queue.cc:260] T 034f47d8564f4094a9d76aee2fea3119 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.337350 1160 ts_tablet_manager.cc:1434] T 034f47d8564f4094a9d76aee2fea3119 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:26.337690 1160 tablet_bootstrap.cc:492] T 33cce042f33b4be198d11fd39f5f8838 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:26.339988 581 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet bb0fc179a337469eae056c5bb3815e6b (table default.Table_c27589191ac249f4a9460f5bbae2611b [id=461d63ce3b4f4d12b5f395bb8c1cdbb8], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet bb0fc179a337469eae056c5bb3815e6b has no consensus state
I20251211 20:52:26.340448 1163 ts_tablet_manager.cc:1929] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.340587 1163 log.cc:1199] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/f00f7c1f6a4e44ff89341b4a55cc93fb
I20251211 20:52:26.341017 1163 ts_tablet_manager.cc:1950] T f00f7c1f6a4e44ff89341b4a55cc93fb P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:26.342280 1166 tablet_bootstrap.cc:492] T 91602f30b4464e35b16fa3dcac445b49 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:26.342446 1166 ts_tablet_manager.cc:1403] T 91602f30b4464e35b16fa3dcac445b49 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.054s user 0.003s sys 0.004s
I20251211 20:52:26.342780 596 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet f92d26bcde3b410aa56bc23a61b1ac35 (table default.Table_7c42321f3e494dc8835cc2db68e653e1 [id=bee5829ed7d44761a9ae57d3134d38dc]). Expected version 1 got 0
I20251211 20:52:26.343232 1166 raft_consensus.cc:359] T 91602f30b4464e35b16fa3dcac445b49 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.343415 1166 raft_consensus.cc:385] T 91602f30b4464e35b16fa3dcac445b49 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.343482 1166 raft_consensus.cc:740] T 91602f30b4464e35b16fa3dcac445b49 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.343842 1451 raft_consensus.cc:493] T dd425601d89148eb95525f796a3abf1e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:26.343972 1451 raft_consensus.cc:515] T dd425601d89148eb95525f796a3abf1e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.344604 1451 leader_election.cc:290] T dd425601d89148eb95525f796a3abf1e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:26.344998 1166 consensus_queue.cc:260] T 91602f30b4464e35b16fa3dcac445b49 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.345700 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dd425601d89148eb95525f796a3abf1e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:26.345939 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dd425601d89148eb95525f796a3abf1e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:26.346488 982 leader_election.cc:343] T dd425601d89148eb95525f796a3abf1e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:26.346669 983 leader_election.cc:343] T dd425601d89148eb95525f796a3abf1e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:26.346803 983 leader_election.cc:304] T dd425601d89148eb95525f796a3abf1e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:26.347203 1451 raft_consensus.cc:2749] T dd425601d89148eb95525f796a3abf1e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:26.349205 1166 ts_tablet_manager.cc:1434] T 91602f30b4464e35b16fa3dcac445b49 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20251211 20:52:26.349485 593 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet f92d26bcde3b410aa56bc23a61b1ac35 (table default.Table_7c42321f3e494dc8835cc2db68e653e1 [id=bee5829ed7d44761a9ae57d3134d38dc]). Expected version 1 got 0
I20251211 20:52:26.349552 1166 tablet_bootstrap.cc:492] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:26.352692 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 8a064789f89c424ca698e52333d73eea (table default.Table_f439572c5b1d4997af54e0ab3c036023 [id=cda4c0ad8a0347be88a0518491e6af49]) successfully deleted
W20251211 20:52:26.352957 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 55c40b2464f8444ca6c9fa0400f7f7f2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55c40b2464f8444ca6c9fa0400f7f7f2 already in progress: creating tablet
W20251211 20:52:26.353159 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 804a11c8f7414eb1a6bcff2952be9ae2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 804a11c8f7414eb1a6bcff2952be9ae2 already in progress: creating tablet
W20251211 20:52:26.353308 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet bb5c5523e175406d9f6f9cd2736a2045 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb5c5523e175406d9f6f9cd2736a2045 already in progress: creating tablet
W20251211 20:52:26.353477 582 catalog_manager.cc:4952] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 97ada0641165444a998db26796793507 because the tablet was not found. No further retry: Not found: Tablet not found: 97ada0641165444a998db26796793507
W20251211 20:52:26.353621 582 catalog_manager.cc:4952] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 171d5b17dfee49159925b476b08787a4 because the tablet was not found. No further retry: Not found: Tablet not found: 171d5b17dfee49159925b476b08787a4
W20251211 20:52:26.353889 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet c58241f7e1404a8c8db92c4f9a47d143 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c58241f7e1404a8c8db92c4f9a47d143 already in progress: creating tablet
I20251211 20:52:26.354647 894 tablet_service.cc:1505] Processing CreateTablet for tablet fb2d30147cd74d1893783be1e1694d1f (DEFAULT_TABLE table=default.Table_1bf756c0a806499fbfd10745831a653f [id=c5b5110e2f1f4eb29d233b6a9eef4737]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.354966 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet 91602f30b4464e35b16fa3dcac445b49 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:26.355018 1451 raft_consensus.cc:493] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:26.355211 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb2d30147cd74d1893783be1e1694d1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.355152 1451 raft_consensus.cc:515] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.356626 1451 leader_election.cc:290] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:26.356822 1166 tablet_bootstrap.cc:654] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.357017 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "27466459d8e6411ab87ac474fb617735" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:26.357200 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "27466459d8e6411ab87ac474fb617735" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:26.357616 983 leader_election.cc:343] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:26.357818 982 leader_election.cc:343] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:26.357916 982 leader_election.cc:304] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:26.358428 1451 raft_consensus.cc:2749] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:26.359213 1160 tablet_bootstrap.cc:654] T 33cce042f33b4be198d11fd39f5f8838 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.361649 1168 tablet_bootstrap.cc:492] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:26.361835 1168 ts_tablet_manager.cc:1403] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.027s user 0.005s sys 0.001s
I20251211 20:52:26.363095 1168 raft_consensus.cc:359] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.363332 1168 raft_consensus.cc:385] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.363404 1168 raft_consensus.cc:740] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.363703 1168 consensus_queue.cc:260] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.364195 1168 ts_tablet_manager.cc:1434] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:26.364526 1168 tablet_bootstrap.cc:492] T ec7df7b11ce34b8489048e6acdcb4e94 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:26.366242 1168 tablet_bootstrap.cc:654] T ec7df7b11ce34b8489048e6acdcb4e94 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.368887 1163 tablet_replica.cc:333] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:26.369155 1163 raft_consensus.cc:2243] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.369351 1163 raft_consensus.cc:2272] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.373612 1160 tablet_bootstrap.cc:492] T 33cce042f33b4be198d11fd39f5f8838 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:26.373930 1160 ts_tablet_manager.cc:1403] T 33cce042f33b4be198d11fd39f5f8838 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.036s user 0.003s sys 0.006s
I20251211 20:52:26.374845 1163 ts_tablet_manager.cc:1916] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.374800 1160 raft_consensus.cc:359] T 33cce042f33b4be198d11fd39f5f8838 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.375092 1160 raft_consensus.cc:385] T 33cce042f33b4be198d11fd39f5f8838 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.375221 1160 raft_consensus.cc:740] T 33cce042f33b4be198d11fd39f5f8838 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.375538 1160 consensus_queue.cc:260] T 33cce042f33b4be198d11fd39f5f8838 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.380718 1160 ts_tablet_manager.cc:1434] T 33cce042f33b4be198d11fd39f5f8838 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20251211 20:52:26.381203 1166 tablet_bootstrap.cc:492] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:26.381361 1166 ts_tablet_manager.cc:1403] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.002s
I20251211 20:52:26.382153 1166 raft_consensus.cc:359] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.382378 1166 raft_consensus.cc:385] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.382477 1166 raft_consensus.cc:740] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.382735 1166 consensus_queue.cc:260] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.383241 1166 ts_tablet_manager.cc:1434] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:26.384608 1166 tablet_bootstrap.cc:492] T f19a83b557e24f929b172d926996ae64 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:26.385680 1451 raft_consensus.cc:493] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:26.385803 1451 raft_consensus.cc:515] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.386391 1451 leader_election.cc:290] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:26.387071 1168 tablet_bootstrap.cc:492] T ec7df7b11ce34b8489048e6acdcb4e94 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:26.387210 1168 ts_tablet_manager.cc:1403] T ec7df7b11ce34b8489048e6acdcb4e94 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.002s
I20251211 20:52:26.387187 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8308ad46551447439a248d20ed1e5922" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:26.363125 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet f00f7c1f6a4e44ff89341b4a55cc93fb (table default.Table_b06f9587f78645bb9712c3b080f3c0d4 [id=ea12bdaa894f446da22635f95d97ac2a]) successfully deleted
W20251211 20:52:26.388182 983 leader_election.cc:343] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:26.388404 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8308ad46551447439a248d20ed1e5922" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:26.388432 1168 raft_consensus.cc:359] T ec7df7b11ce34b8489048e6acdcb4e94 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.388615 1168 raft_consensus.cc:385] T ec7df7b11ce34b8489048e6acdcb4e94 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20251211 20:52:26.388918 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet bb5c5523e175406d9f6f9cd2736a2045 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb5c5523e175406d9f6f9cd2736a2045 already in progress: creating tablet
I20251211 20:52:26.389037 1168 raft_consensus.cc:740] T ec7df7b11ce34b8489048e6acdcb4e94 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
W20251211 20:52:26.389153 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 804a11c8f7414eb1a6bcff2952be9ae2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 804a11c8f7414eb1a6bcff2952be9ae2 already in progress: creating tablet
W20251211 20:52:26.389338 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 55c40b2464f8444ca6c9fa0400f7f7f2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55c40b2464f8444ca6c9fa0400f7f7f2 already in progress: creating tablet
I20251211 20:52:26.389382 1168 consensus_queue.cc:260] T ec7df7b11ce34b8489048e6acdcb4e94 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.389410 1016 tablet_service.cc:1505] Processing CreateTablet for tablet a9b2aa21ab124b28aebc33adba0583d2 (DEFAULT_TABLE table=default.Table_b70392ad5fb6451482100395256494bd [id=eafa1350ab384c8cba766017840d59a6]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:26.389516 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 97ada0641165444a998db26796793507 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 97ada0641165444a998db26796793507 already in progress: creating tablet
I20251211 20:52:26.389633 1024 tablet_service.cc:1505] Processing CreateTablet for tablet 171d5b17dfee49159925b476b08787a4 (DEFAULT_TABLE table=default.Table_75c7933bd82449a1a24a7481815ee23b [id=4f5722552e534a00874e4113c0dfbaf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.389853 1016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9b2aa21ab124b28aebc33adba0583d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.389920 1168 ts_tablet_manager.cc:1434] T ec7df7b11ce34b8489048e6acdcb4e94 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:26.390132 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 648f43e95e034f2188a20e1b837f2818 (DEFAULT_TABLE table=default.Table_60c74dd24cd34456b535dc8445e13383 [id=75a5a167f3f84710afbf197ea1eff3cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.390221 1168 tablet_bootstrap.cc:492] T 33cce042f33b4be198d11fd39f5f8838 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:26.390460 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 648f43e95e034f2188a20e1b837f2818. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:26.392319 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet d762b4afb5ff447dac3468393826205a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d762b4afb5ff447dac3468393826205a already in progress: creating tablet
I20251211 20:52:26.394340 738 tablet_service.cc:1505] Processing CreateTablet for tablet 68be21a25c6743f5b86d518d98fbc35e (DEFAULT_TABLE table=default.Table_afd3c9ee35ad41caa1717a299443409e [id=b0ebaab8cd5a4deb9a5ea1e7bd3ccec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.394798 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 773cf53745a64f62817113b33e3182e7 (DEFAULT_TABLE table=default.Table_5e65e4dae9454b3e993a46b2255d955d [id=6bbfcc2561f349519a69ae1f56016cc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.394932 752 tablet_service.cc:1505] Processing CreateTablet for tablet 33addf3d158941098f240cfcdec45d28 (DEFAULT_TABLE table=default.Table_aa53337337ff420e8030af1bb87dd3f7 [id=f75813add54e46189ff1bbed82d83ba7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.395166 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 773cf53745a64f62817113b33e3182e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.397449 890 tablet_service.cc:1552] Processing DeleteTablet for tablet 732c76659c0540bc8b6909b430ffb146 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:26.397858 1164 tablet_replica.cc:333] T 732c76659c0540bc8b6909b430ffb146 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:26.398056 1164 raft_consensus.cc:2243] T 732c76659c0540bc8b6909b430ffb146 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.398191 1164 raft_consensus.cc:2272] T 732c76659c0540bc8b6909b430ffb146 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.398744 1164 ts_tablet_manager.cc:1916] T 732c76659c0540bc8b6909b430ffb146 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.399941 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 68be21a25c6743f5b86d518d98fbc35e (DEFAULT_TABLE table=default.Table_afd3c9ee35ad41caa1717a299443409e [id=b0ebaab8cd5a4deb9a5ea1e7bd3ccec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.400296 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68be21a25c6743f5b86d518d98fbc35e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.400849 1160 tablet_bootstrap.cc:492] T c58241f7e1404a8c8db92c4f9a47d143 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:26.402603 1160 tablet_bootstrap.cc:654] T c58241f7e1404a8c8db92c4f9a47d143 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.404318 1164 ts_tablet_manager.cc:1929] T 732c76659c0540bc8b6909b430ffb146 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.404415 1164 log.cc:1199] T 732c76659c0540bc8b6909b430ffb146 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/732c76659c0540bc8b6909b430ffb146
I20251211 20:52:26.404676 1164 ts_tablet_manager.cc:1950] T 732c76659c0540bc8b6909b430ffb146 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
W20251211 20:52:26.404814 982 leader_election.cc:343] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:26.404960 982 leader_election.cc:304] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:26.405308 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 171d5b17dfee49159925b476b08787a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.405413 1451 raft_consensus.cc:2749] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:26.406231 739 tablet_service.cc:1505] Processing CreateTablet for tablet 648f43e95e034f2188a20e1b837f2818 (DEFAULT_TABLE table=default.Table_60c74dd24cd34456b535dc8445e13383 [id=75a5a167f3f84710afbf197ea1eff3cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.407112 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 33addf3d158941098f240cfcdec45d28 (DEFAULT_TABLE table=default.Table_aa53337337ff420e8030af1bb87dd3f7 [id=f75813add54e46189ff1bbed82d83ba7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.407495 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33addf3d158941098f240cfcdec45d28. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.408938 741 tablet_service.cc:1505] Processing CreateTablet for tablet a9b2aa21ab124b28aebc33adba0583d2 (DEFAULT_TABLE table=default.Table_b70392ad5fb6451482100395256494bd [id=eafa1350ab384c8cba766017840d59a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.409305 738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68be21a25c6743f5b86d518d98fbc35e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.412647 744 tablet_service.cc:1505] Processing CreateTablet for tablet 773cf53745a64f62817113b33e3182e7 (DEFAULT_TABLE table=default.Table_5e65e4dae9454b3e993a46b2255d955d [id=6bbfcc2561f349519a69ae1f56016cc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.413298 750 tablet_service.cc:1505] Processing CreateTablet for tablet 0f046ea2637e4a5583b7969e19a605d5 (DEFAULT_TABLE table=default.Table_6669f84cc5f04b8fb74c1d226054f2b8 [id=e96fd7db496847a5b1e0b79546a336d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.414407 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 648f43e95e034f2188a20e1b837f2818. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.415364 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33addf3d158941098f240cfcdec45d28. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.419353 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f046ea2637e4a5583b7969e19a605d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.419654 1168 tablet_bootstrap.cc:654] T 33cce042f33b4be198d11fd39f5f8838 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.420631 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 773cf53745a64f62817113b33e3182e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.422613 1166 tablet_bootstrap.cc:654] T f19a83b557e24f929b172d926996ae64 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.424355 890 tablet_service.cc:1552] Processing DeleteTablet for tablet 9662efc97a53461d888a062463e80ee6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:26.424474 1160 tablet_bootstrap.cc:492] T c58241f7e1404a8c8db92c4f9a47d143 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:26.424701 1160 ts_tablet_manager.cc:1403] T c58241f7e1404a8c8db92c4f9a47d143 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.001s
I20251211 20:52:26.424849 1164 tablet_replica.cc:333] T 9662efc97a53461d888a062463e80ee6 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:26.425083 1164 raft_consensus.cc:2243] T 9662efc97a53461d888a062463e80ee6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.425321 1164 raft_consensus.cc:2272] T 9662efc97a53461d888a062463e80ee6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.425930 1160 raft_consensus.cc:359] T c58241f7e1404a8c8db92c4f9a47d143 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.426127 1160 raft_consensus.cc:385] T c58241f7e1404a8c8db92c4f9a47d143 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.426206 1160 raft_consensus.cc:740] T c58241f7e1404a8c8db92c4f9a47d143 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.426267 1164 ts_tablet_manager.cc:1916] T 9662efc97a53461d888a062463e80ee6 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.426910 1160 consensus_queue.cc:260] T c58241f7e1404a8c8db92c4f9a47d143 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.433312 1160 ts_tablet_manager.cc:1434] T c58241f7e1404a8c8db92c4f9a47d143 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20251211 20:52:26.432168 749 tablet_service.cc:1505] Processing CreateTablet for tablet 171d5b17dfee49159925b476b08787a4 (DEFAULT_TABLE table=default.Table_75c7933bd82449a1a24a7481815ee23b [id=4f5722552e534a00874e4113c0dfbaf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.434226 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9b2aa21ab124b28aebc33adba0583d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.434325 1160 tablet_bootstrap.cc:492] T 4870da0224e048ed82f227d7ba676a94 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:26.437417 1160 tablet_bootstrap.cc:654] T 4870da0224e048ed82f227d7ba676a94 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.439838 1163 ts_tablet_manager.cc:1929] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.439996 1163 log.cc:1199] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/27466459d8e6411ab87ac474fb617735
I20251211 20:52:26.440335 1163 ts_tablet_manager.cc:1950] T 27466459d8e6411ab87ac474fb617735 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:26.446862 1166 tablet_bootstrap.cc:492] T f19a83b557e24f929b172d926996ae64 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:26.447045 1166 ts_tablet_manager.cc:1403] T f19a83b557e24f929b172d926996ae64 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.062s user 0.002s sys 0.002s
I20251211 20:52:26.447978 1166 raft_consensus.cc:359] T f19a83b557e24f929b172d926996ae64 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:26.448200 1166 raft_consensus.cc:385] T f19a83b557e24f929b172d926996ae64 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.448279 1166 raft_consensus.cc:740] T f19a83b557e24f929b172d926996ae64 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.448567 1166 consensus_queue.cc:260] T f19a83b557e24f929b172d926996ae64 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:26.448874 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 171d5b17dfee49159925b476b08787a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.449317 1166 ts_tablet_manager.cc:1434] T f19a83b557e24f929b172d926996ae64 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251211 20:52:26.449752 1164 ts_tablet_manager.cc:1929] T 9662efc97a53461d888a062463e80ee6 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.449910 1164 log.cc:1199] T 9662efc97a53461d888a062463e80ee6 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/9662efc97a53461d888a062463e80ee6
I20251211 20:52:26.450341 1164 ts_tablet_manager.cc:1950] T 9662efc97a53461d888a062463e80ee6 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:26.462819 1166 tablet_bootstrap.cc:492] T bd8fd6c977204c90b558827fc0b15f43 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:26.465586 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 27466459d8e6411ab87ac474fb617735 (table default.Table_265379fa17b74d0f954a2a2c4de113c8 [id=18f8d722faaa4293a112525ff0c96216]) successfully deleted
W20251211 20:52:26.467671 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 171d5b17dfee49159925b476b08787a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 171d5b17dfee49159925b476b08787a4 already in progress: creating tablet
I20251211 20:52:26.468914 890 tablet_service.cc:1552] Processing DeleteTablet for tablet 034f47d8564f4094a9d76aee2fea3119 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:26.469259 1164 tablet_replica.cc:333] T 034f47d8564f4094a9d76aee2fea3119 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:26.469491 1164 raft_consensus.cc:2243] T 034f47d8564f4094a9d76aee2fea3119 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.469666 1164 raft_consensus.cc:2272] T 034f47d8564f4094a9d76aee2fea3119 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.470307 890 tablet_service.cc:1505] Processing CreateTablet for tablet 171d5b17dfee49159925b476b08787a4 (DEFAULT_TABLE table=default.Table_75c7933bd82449a1a24a7481815ee23b [id=4f5722552e534a00874e4113c0dfbaf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.470439 1164 ts_tablet_manager.cc:1916] T 034f47d8564f4094a9d76aee2fea3119 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.470842 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 171d5b17dfee49159925b476b08787a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.473289 1163 tablet_replica.cc:333] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:26.473567 1163 raft_consensus.cc:2243] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.475569 1163 raft_consensus.cc:2272] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.476483 1163 ts_tablet_manager.cc:1916] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.480293 889 tablet_service.cc:1505] Processing CreateTablet for tablet a9b2aa21ab124b28aebc33adba0583d2 (DEFAULT_TABLE table=default.Table_b70392ad5fb6451482100395256494bd [id=eafa1350ab384c8cba766017840d59a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.480742 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9b2aa21ab124b28aebc33adba0583d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.475920 1166 tablet_bootstrap.cc:654] T bd8fd6c977204c90b558827fc0b15f43 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.483529 1164 ts_tablet_manager.cc:1929] T 034f47d8564f4094a9d76aee2fea3119 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.489229 1164 log.cc:1199] T 034f47d8564f4094a9d76aee2fea3119 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/034f47d8564f4094a9d76aee2fea3119
I20251211 20:52:26.489815 1164 ts_tablet_manager.cc:1950] T 034f47d8564f4094a9d76aee2fea3119 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:26.491526 1166 tablet_bootstrap.cc:492] T bd8fd6c977204c90b558827fc0b15f43 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:26.491724 1166 ts_tablet_manager.cc:1403] T bd8fd6c977204c90b558827fc0b15f43 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.029s user 0.004s sys 0.000s
I20251211 20:52:26.491741 1168 tablet_bootstrap.cc:492] T 33cce042f33b4be198d11fd39f5f8838 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:26.491910 1168 ts_tablet_manager.cc:1403] T 33cce042f33b4be198d11fd39f5f8838 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.102s user 0.008s sys 0.010s
I20251211 20:52:26.492370 878 tablet_service.cc:1505] Processing CreateTablet for tablet 773cf53745a64f62817113b33e3182e7 (DEFAULT_TABLE table=default.Table_5e65e4dae9454b3e993a46b2255d955d [id=6bbfcc2561f349519a69ae1f56016cc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.492529 1166 raft_consensus.cc:359] T bd8fd6c977204c90b558827fc0b15f43 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.492720 1166 raft_consensus.cc:385] T bd8fd6c977204c90b558827fc0b15f43 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.492801 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 773cf53745a64f62817113b33e3182e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.492801 1166 raft_consensus.cc:740] T bd8fd6c977204c90b558827fc0b15f43 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.492782 1168 raft_consensus.cc:359] T 33cce042f33b4be198d11fd39f5f8838 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.492975 1168 raft_consensus.cc:385] T 33cce042f33b4be198d11fd39f5f8838 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.493037 1168 raft_consensus.cc:740] T 33cce042f33b4be198d11fd39f5f8838 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.493083 1166 consensus_queue.cc:260] T bd8fd6c977204c90b558827fc0b15f43 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.493636 1166 ts_tablet_manager.cc:1434] T bd8fd6c977204c90b558827fc0b15f43 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:26.493880 1168 consensus_queue.cc:260] T 33cce042f33b4be198d11fd39f5f8838 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.495724 1163 ts_tablet_manager.cc:1929] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.495837 1163 log.cc:1199] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/c58241f7e1404a8c8db92c4f9a47d143
I20251211 20:52:26.496191 1163 ts_tablet_manager.cc:1950] T c58241f7e1404a8c8db92c4f9a47d143 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:26.496690 1166 tablet_bootstrap.cc:492] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:26.498582 1166 tablet_bootstrap.cc:654] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.500993 889 tablet_service.cc:1505] Processing CreateTablet for tablet 648f43e95e034f2188a20e1b837f2818 (DEFAULT_TABLE table=default.Table_60c74dd24cd34456b535dc8445e13383 [id=75a5a167f3f84710afbf197ea1eff3cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.501497 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 648f43e95e034f2188a20e1b837f2818. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.502218 1163 tablet_replica.cc:333] T d824bc02b25842d18f28f711eef6669f P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:26.502452 1163 raft_consensus.cc:2243] T d824bc02b25842d18f28f711eef6669f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.502677 1163 raft_consensus.cc:2272] T d824bc02b25842d18f28f711eef6669f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.503047 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet c58241f7e1404a8c8db92c4f9a47d143 (table default.Table_3570af5d8bff402ab3140ae51bae174c [id=90602e87d8b6425295bb04a5fca84ba7]) successfully deleted
W20251211 20:52:26.503346 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet d762b4afb5ff447dac3468393826205a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d762b4afb5ff447dac3468393826205a already in progress: creating tablet
I20251211 20:52:26.505831 1168 ts_tablet_manager.cc:1434] T 33cce042f33b4be198d11fd39f5f8838 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
I20251211 20:52:26.509163 1168 tablet_bootstrap.cc:492] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:26.510870 1168 tablet_bootstrap.cc:654] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.511476 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 6c64fd8dd49b4e5d91c1ffb0fc2559e0
I20251211 20:52:26.513373 596 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 4870da0224e048ed82f227d7ba676a94 (table default.Table_5cb009a3a8b24126af4e28df1795d049 [id=28f3ebe9b7a64b5e89404b0407b0e5ae]). Expected version 1 got 0
I20251211 20:52:26.513403 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet 6c64fd8dd49b4e5d91c1ffb0fc2559e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:26.513706 1163 ts_tablet_manager.cc:1916] T d824bc02b25842d18f28f711eef6669f P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.515323 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 6c64fd8dd49b4e5d91c1ffb0fc2559e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:26.516036 581 catalog_manager.cc:4952] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 6c64fd8dd49b4e5d91c1ffb0fc2559e0 because the tablet was not found. No further retry: Not found: Tablet not found: 6c64fd8dd49b4e5d91c1ffb0fc2559e0
I20251211 20:52:26.518792 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 33cce042f33b4be198d11fd39f5f8838
I20251211 20:52:26.520124 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet 33cce042f33b4be198d11fd39f5f8838 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:26.525660 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet bd8fd6c977204c90b558827fc0b15f43 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:26.525923 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet f19a83b557e24f929b172d926996ae64 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:26.526937 750 tablet_service.cc:1552] Processing DeleteTablet for tablet ec7df7b11ce34b8489048e6acdcb4e94 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:22 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:26.527311 1165 tablet_replica.cc:333] T ec7df7b11ce34b8489048e6acdcb4e94 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:26.527523 1165 raft_consensus.cc:2243] T ec7df7b11ce34b8489048e6acdcb4e94 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.527748 1165 raft_consensus.cc:2272] T ec7df7b11ce34b8489048e6acdcb4e94 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.528610 1165 ts_tablet_manager.cc:1916] T ec7df7b11ce34b8489048e6acdcb4e94 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.531148 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_7aa604df1f25488f9f06c6b0e6daaf88"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:26.531729 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7aa604df1f25488f9f06c6b0e6daaf88 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:26.533242 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_60175b4e1c9744e0b6aefa1dece53aa0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:26.533785 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_60175b4e1c9744e0b6aefa1dece53aa0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:26.534976 1163 ts_tablet_manager.cc:1929] T d824bc02b25842d18f28f711eef6669f P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.535127 1163 log.cc:1199] T d824bc02b25842d18f28f711eef6669f P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/d824bc02b25842d18f28f711eef6669f
I20251211 20:52:26.535156 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_cbe68d84efef4745822f6aef5a723f11" } modify_external_catalogs: true
I20251211 20:52:26.535310 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_cbe68d84efef4745822f6aef5a723f11" } modify_external_catalogs: true
I20251211 20:52:26.535488 1163 ts_tablet_manager.cc:1950] T d824bc02b25842d18f28f711eef6669f P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:26.536460 1163 tablet_replica.cc:333] T 91602f30b4464e35b16fa3dcac445b49 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:26.536654 1163 raft_consensus.cc:2243] T 91602f30b4464e35b16fa3dcac445b49 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.536842 1163 raft_consensus.cc:2272] T 91602f30b4464e35b16fa3dcac445b49 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.536830 891 tablet_service.cc:1505] Processing CreateTablet for tablet 0f046ea2637e4a5583b7969e19a605d5 (DEFAULT_TABLE table=default.Table_6669f84cc5f04b8fb74c1d226054f2b8 [id=e96fd7db496847a5b1e0b79546a336d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.537024 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet d824bc02b25842d18f28f711eef6669f (table default.Table_984c6c545e3a4a2ca496d202198a177b [id=861123c9df6d4841a078e84897b4ce35]) successfully deleted
I20251211 20:52:26.537259 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f046ea2637e4a5583b7969e19a605d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.538450 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_6bfc3963d4e346858acdfd9b9016245d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:26.538964 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6bfc3963d4e346858acdfd9b9016245d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:26.543825 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 33cce042f33b4be198d11fd39f5f8838 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:26.544628 878 tablet_service.cc:1505] Processing CreateTablet for tablet 33addf3d158941098f240cfcdec45d28 (DEFAULT_TABLE table=default.Table_aa53337337ff420e8030af1bb87dd3f7 [id=f75813add54e46189ff1bbed82d83ba7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.545106 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33addf3d158941098f240cfcdec45d28. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.545210 1165 ts_tablet_manager.cc:1929] T ec7df7b11ce34b8489048e6acdcb4e94 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.545353 1165 log.cc:1199] T ec7df7b11ce34b8489048e6acdcb4e94 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/ec7df7b11ce34b8489048e6acdcb4e94
I20251211 20:52:26.545770 1165 ts_tablet_manager.cc:1950] T ec7df7b11ce34b8489048e6acdcb4e94 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:26.546963 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet ec7df7b11ce34b8489048e6acdcb4e94 (table default.Table_02014bfadb7b434f960168f2db70b838 [id=14a0dbb8486246b09f3e06fc2120cd28]) successfully deleted
I20251211 20:52:26.549839 1163 ts_tablet_manager.cc:1916] T 91602f30b4464e35b16fa3dcac445b49 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.550092 1165 tablet_replica.cc:333] T 33cce042f33b4be198d11fd39f5f8838 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:26.550309 1165 raft_consensus.cc:2243] T 33cce042f33b4be198d11fd39f5f8838 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.550508 1165 raft_consensus.cc:2272] T 33cce042f33b4be198d11fd39f5f8838 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.551014 1166 tablet_bootstrap.cc:492] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:26.551201 1166 ts_tablet_manager.cc:1403] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.055s user 0.017s sys 0.000s
I20251211 20:52:26.552986 1165 ts_tablet_manager.cc:1916] T 33cce042f33b4be198d11fd39f5f8838 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.554579 880 tablet_service.cc:1505] Processing CreateTablet for tablet 68be21a25c6743f5b86d518d98fbc35e (DEFAULT_TABLE table=default.Table_afd3c9ee35ad41caa1717a299443409e [id=b0ebaab8cd5a4deb9a5ea1e7bd3ccec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.555104 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68be21a25c6743f5b86d518d98fbc35e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.556327 1166 raft_consensus.cc:359] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.556556 1166 raft_consensus.cc:385] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.556639 1166 raft_consensus.cc:740] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.558884 1166 consensus_queue.cc:260] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.559507 1166 ts_tablet_manager.cc:1434] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20251211 20:52:26.576407 1166 tablet_bootstrap.cc:492] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:26.576592 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 0f046ea2637e4a5583b7969e19a605d5 (DEFAULT_TABLE table=default.Table_6669f84cc5f04b8fb74c1d226054f2b8 [id=e96fd7db496847a5b1e0b79546a336d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.577301 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 6a2442a4ebc940a281f9ea1a394f6621
I20251211 20:52:26.578236 1166 tablet_bootstrap.cc:654] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.578781 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f046ea2637e4a5583b7969e19a605d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.579551 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a2442a4ebc940a281f9ea1a394f6621 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:26.580015 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a2442a4ebc940a281f9ea1a394f6621 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:26.580297 596 master_service.cc:954] ReplaceTablet: received request to replace tablet 6a206b37e43247b5babe3113eab2c9fb from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:26.580680 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c607094f0e4d4af48d2e54d4d728ae13" } modify_external_catalogs: true
I20251211 20:52:26.580832 594 master_service.cc:954] ReplaceTablet: received request to replace tablet 27a08a64d92342d08ab4406e8006a97b from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:26.580875 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c607094f0e4d4af48d2e54d4d728ae13" } modify_external_catalogs: true
I20251211 20:52:26.582060 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_786e1caf9f674d48a6aad3ef7edc5b01" } modify_external_catalogs: true
I20251211 20:52:26.582235 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_786e1caf9f674d48a6aad3ef7edc5b01" } modify_external_catalogs: true
I20251211 20:52:26.582306 593 master_service.cc:954] ReplaceTablet: received request to replace tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:26.583393 596 master_service.cc:954] ReplaceTablet: received request to replace tablet 340730a17a834a77bbc20b95aba98e2c from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:26.583549 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_86e5b1e5b4214b48a21f398642d97640" } new_table_name: "default.Table_fbe890176f3d42768b00d57c928f38c4" modify_external_catalogs: true
I20251211 20:52:26.583699 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a81b3b9368cc4a118087cd5131a50534" } new_table_name: "default.Table_063289c4a3514468a2c1fa058dde23f3" modify_external_catalogs: true
I20251211 20:52:26.585492 595 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0ceac14066b444dfa440c23e489e574a" } new_table_name: "default.Table_906e264983df4cbcb0dd73a1edd16c7f" modify_external_catalogs: true
I20251211 20:52:26.588158 1165 ts_tablet_manager.cc:1929] T 33cce042f33b4be198d11fd39f5f8838 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.588805 1420 raft_consensus.cc:493] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:26.588912 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_e98180a63672423688b0de80af514e8b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:26.588963 1420 raft_consensus.cc:515] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.589144 1163 ts_tablet_manager.cc:1929] T 91602f30b4464e35b16fa3dcac445b49 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.589263 1163 log.cc:1199] T 91602f30b4464e35b16fa3dcac445b49 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/91602f30b4464e35b16fa3dcac445b49
W20251211 20:52:26.589402 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e98180a63672423688b0de80af514e8b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:26.589623 1163 ts_tablet_manager.cc:1950] T 91602f30b4464e35b16fa3dcac445b49 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:26.589609 1420 leader_election.cc:290] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:26.590617 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 91602f30b4464e35b16fa3dcac445b49 (table default.Table_afc299e5a13c43878cf6c746698af917 [id=6534b4b4514f4763bcff70c04d93ac7c]) successfully deleted
I20251211 20:52:26.590596 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a5eeb56709f046d9be92ac600ebc330e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:26.590639 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a5eeb56709f046d9be92ac600ebc330e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:26.590911 1047 raft_consensus.cc:2393] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6f8d784fd1cd492bbe3809403ec34919 in current term 1: Already voted for candidate eb0f1933d9a74fccaa4618047fc05c41 in this term.
W20251211 20:52:26.591038 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 6c64fd8dd49b4e5d91c1ffb0fc2559e0 because the tablet was not found. No further retry: Not found: Tablet not found: 6c64fd8dd49b4e5d91c1ffb0fc2559e0
I20251211 20:52:26.591555 710 leader_election.cc:304] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:26.592598 1420 raft_consensus.cc:3060] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:26.593686 891 tablet_service.cc:1552] Processing DeleteTablet for tablet c58241f7e1404a8c8db92c4f9a47d143 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:26.593946 1164 tablet_replica.cc:333] T c58241f7e1404a8c8db92c4f9a47d143 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:26.594108 1164 raft_consensus.cc:2243] T c58241f7e1404a8c8db92c4f9a47d143 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.594776 1164 raft_consensus.cc:2272] T c58241f7e1404a8c8db92c4f9a47d143 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.595274 1165 log.cc:1199] T 33cce042f33b4be198d11fd39f5f8838 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/33cce042f33b4be198d11fd39f5f8838
I20251211 20:52:26.595708 1165 ts_tablet_manager.cc:1950] T 33cce042f33b4be198d11fd39f5f8838 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:26.596429 1420 raft_consensus.cc:2749] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:26.596748 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 33cce042f33b4be198d11fd39f5f8838 (table default.Table_fdd4d94139714e3c954d6daf3c82a649 [id=5a7b1e562b084b109569a8733fd470a3]) successfully deleted
W20251211 20:52:26.597165 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 6a2442a4ebc940a281f9ea1a394f6621 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a2442a4ebc940a281f9ea1a394f6621 already in progress: creating tablet
I20251211 20:52:26.590713 1163 tablet_replica.cc:333] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:26.599903 1163 raft_consensus.cc:2243] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.600085 1163 raft_consensus.cc:2272] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.600639 1163 ts_tablet_manager.cc:1916] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.603930 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 6c64fd8dd49b4e5d91c1ffb0fc2559e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:26.605115 890 tablet_service.cc:1552] Processing DeleteTablet for tablet 33cce042f33b4be198d11fd39f5f8838 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:26.605700 1164 ts_tablet_manager.cc:1916] T c58241f7e1404a8c8db92c4f9a47d143 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.606986 1163 ts_tablet_manager.cc:1929] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.607142 1163 log.cc:1199] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/33cce042f33b4be198d11fd39f5f8838
I20251211 20:52:26.607503 1163 ts_tablet_manager.cc:1950] T 33cce042f33b4be198d11fd39f5f8838 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:26.608639 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 33cce042f33b4be198d11fd39f5f8838 (table default.Table_fdd4d94139714e3c954d6daf3c82a649 [id=5a7b1e562b084b109569a8733fd470a3]) successfully deleted
I20251211 20:52:26.608728 749 tablet_service.cc:1505] Processing CreateTablet for tablet e28d66ce4711430cb09a9650e99fbe9e (DEFAULT_TABLE table=default.Table_8b8ed719985f42128e56bc26105e7a9c [id=401822f4521f46eeb815147f643a6fcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.612625 1017 tablet_service.cc:1505] Processing CreateTablet for tablet e28d66ce4711430cb09a9650e99fbe9e (DEFAULT_TABLE table=default.Table_8b8ed719985f42128e56bc26105e7a9c [id=401822f4521f46eeb815147f643a6fcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.613176 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e28d66ce4711430cb09a9650e99fbe9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.613839 741 tablet_service.cc:1505] Processing CreateTablet for tablet 1a77e3e744ff42ccb68897704b4c2f1d (DEFAULT_TABLE table=default.Table_045fee0e26fa49119877f83dbd4b9159 [id=85ab3a18cda143e5a7f72e49e6671318]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.614317 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a77e3e744ff42ccb68897704b4c2f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.614856 1163 tablet_replica.cc:333] T bd8fd6c977204c90b558827fc0b15f43 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:26.615098 1163 raft_consensus.cc:2243] T bd8fd6c977204c90b558827fc0b15f43 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.615320 1163 raft_consensus.cc:2272] T bd8fd6c977204c90b558827fc0b15f43 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.616125 1163 ts_tablet_manager.cc:1916] T bd8fd6c977204c90b558827fc0b15f43 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.621259 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e28d66ce4711430cb09a9650e99fbe9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.622423 1163 ts_tablet_manager.cc:1929] T bd8fd6c977204c90b558827fc0b15f43 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.622578 1163 log.cc:1199] T bd8fd6c977204c90b558827fc0b15f43 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/bd8fd6c977204c90b558827fc0b15f43
I20251211 20:52:26.622937 1163 ts_tablet_manager.cc:1950] T bd8fd6c977204c90b558827fc0b15f43 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:26.623172 1164 ts_tablet_manager.cc:1929] T c58241f7e1404a8c8db92c4f9a47d143 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.623296 1164 log.cc:1199] T c58241f7e1404a8c8db92c4f9a47d143 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/c58241f7e1404a8c8db92c4f9a47d143
I20251211 20:52:26.625197 1168 tablet_bootstrap.cc:492] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:26.625348 1168 ts_tablet_manager.cc:1403] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.116s user 0.011s sys 0.014s
I20251211 20:52:26.626108 1168 raft_consensus.cc:359] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.626298 1168 raft_consensus.cc:385] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.626391 1168 raft_consensus.cc:740] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.626890 1164 ts_tablet_manager.cc:1950] T c58241f7e1404a8c8db92c4f9a47d143 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:26.627463 1168 consensus_queue.cc:260] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.631466 1168 ts_tablet_manager.cc:1434] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20251211 20:52:26.632822 1164 tablet_replica.cc:333] T 33cce042f33b4be198d11fd39f5f8838 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:26.633054 1164 raft_consensus.cc:2243] T 33cce042f33b4be198d11fd39f5f8838 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.633226 1164 raft_consensus.cc:2272] T 33cce042f33b4be198d11fd39f5f8838 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.635887 1163 tablet_replica.cc:333] T f19a83b557e24f929b172d926996ae64 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:26.636039 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet bd8fd6c977204c90b558827fc0b15f43 (table default.Table_5b8aa62e6fdf46089c4c8ae541856fbe [id=e5caa22bce84422290707816a93bd3f5]) successfully deleted
I20251211 20:52:26.636147 1163 raft_consensus.cc:2243] T f19a83b557e24f929b172d926996ae64 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.636307 1163 raft_consensus.cc:2272] T f19a83b557e24f929b172d926996ae64 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.636843 1168 tablet_bootstrap.cc:492] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:26.637086 1163 ts_tablet_manager.cc:1916] T f19a83b557e24f929b172d926996ae64 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.638619 1168 tablet_bootstrap.cc:654] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.641171 1164 ts_tablet_manager.cc:1916] T 33cce042f33b4be198d11fd39f5f8838 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.650020 1163 ts_tablet_manager.cc:1929] T f19a83b557e24f929b172d926996ae64 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.650210 1163 log.cc:1199] T f19a83b557e24f929b172d926996ae64 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/f19a83b557e24f929b172d926996ae64
I20251211 20:52:26.650619 1163 ts_tablet_manager.cc:1950] T f19a83b557e24f929b172d926996ae64 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:26.652473 1168 tablet_bootstrap.cc:492] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:26.652655 1168 ts_tablet_manager.cc:1403] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.000s
I20251211 20:52:26.653424 1168 raft_consensus.cc:359] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.653621 1168 raft_consensus.cc:385] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.653574 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet f19a83b557e24f929b172d926996ae64 (table default.Table_4a3a3fe6b0294c939ebc90871cbcb4de [id=a06a0b2452ea4100b09772b53320b537]) successfully deleted
I20251211 20:52:26.653693 1168 raft_consensus.cc:740] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
W20251211 20:52:26.653906 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 6a2442a4ebc940a281f9ea1a394f6621 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a2442a4ebc940a281f9ea1a394f6621 already in progress: creating tablet
I20251211 20:52:26.653909 1168 consensus_queue.cc:260] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.658126 1164 ts_tablet_manager.cc:1929] T 33cce042f33b4be198d11fd39f5f8838 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.658291 1164 log.cc:1199] T 33cce042f33b4be198d11fd39f5f8838 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/33cce042f33b4be198d11fd39f5f8838
I20251211 20:52:26.658625 1164 ts_tablet_manager.cc:1950] T 33cce042f33b4be198d11fd39f5f8838 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:26.663371 1168 ts_tablet_manager.cc:1434] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20251211 20:52:26.665395 1168 tablet_bootstrap.cc:492] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:26.668707 1168 tablet_bootstrap.cc:654] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.668723 1404 consensus_queue.cc:1048] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:26.670578 1510 tablet.cc:1722] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:26.676332 1420 raft_consensus.cc:493] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:26.676501 1420 raft_consensus.cc:515] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.677135 1420 leader_election.cc:290] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:26.678201 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4d98967714184fe6a3e38dd4ca7979dc" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:26.678716 711 leader_election.cc:343] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:26.681440 1168 tablet_bootstrap.cc:492] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:26.681879 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4d98967714184fe6a3e38dd4ca7979dc" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:26.682109 1047 raft_consensus.cc:2468] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:26.682621 710 leader_election.cc:304] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:26.683979 1420 raft_consensus.cc:493] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:26.684103 1420 raft_consensus.cc:515] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.684751 1420 leader_election.cc:290] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:26.684904 1420 raft_consensus.cc:2804] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:26.684996 1420 raft_consensus.cc:493] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:26.685106 1420 raft_consensus.cc:3060] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:26.688589 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "33e1d8a576cd41b1a34778b1458df396" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:26.688823 913 raft_consensus.cc:2468] T 33e1d8a576cd41b1a34778b1458df396 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:26.689368 711 leader_election.cc:304] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters:
I20251211 20:52:26.690485 1420 raft_consensus.cc:515] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.691115 1420 leader_election.cc:290] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:26.691301 1420 raft_consensus.cc:2804] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:26.691385 1420 raft_consensus.cc:493] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:26.691486 1420 raft_consensus.cc:3060] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:26.691777 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4d98967714184fe6a3e38dd4ca7979dc" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:26.692332 711 leader_election.cc:343] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:26.694228 1420 raft_consensus.cc:515] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.694895 1420 leader_election.cc:290] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:26.695438 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "33e1d8a576cd41b1a34778b1458df396" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:26.695662 913 raft_consensus.cc:3060] T 33e1d8a576cd41b1a34778b1458df396 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:26.698212 913 raft_consensus.cc:2468] T 33e1d8a576cd41b1a34778b1458df396 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:26.699393 711 leader_election.cc:304] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters:
I20251211 20:52:26.699965 1420 raft_consensus.cc:2804] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:26.700135 1420 raft_consensus.cc:697] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:26.700400 1420 consensus_queue.cc:237] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.703707 1404 consensus_queue.cc:1048] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:26.711951 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "33e1d8a576cd41b1a34778b1458df396" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:26.712198 1049 raft_consensus.cc:2468] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:26.713264 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4d98967714184fe6a3e38dd4ca7979dc" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:26.713436 1049 raft_consensus.cc:3060] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:26.716398 1049 raft_consensus.cc:2468] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:26.717211 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 1a77e3e744ff42ccb68897704b4c2f1d (DEFAULT_TABLE table=default.Table_045fee0e26fa49119877f83dbd4b9159 [id=85ab3a18cda143e5a7f72e49e6671318]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.717612 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a77e3e744ff42ccb68897704b4c2f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.718204 741 tablet_service.cc:1505] Processing CreateTablet for tablet 6c64fd8dd49b4e5d91c1ffb0fc2559e0 (DEFAULT_TABLE table=default.Table_cc48404bbbf24a7caae7641a8d42c247 [id=048a032339b3462ba676bcb44f4e2082]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.718595 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c64fd8dd49b4e5d91c1ffb0fc2559e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.725438 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "33e1d8a576cd41b1a34778b1458df396" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:26.725776 1049 raft_consensus.cc:3060] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:26.732368 1049 raft_consensus.cc:2468] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:26.732898 1508 tablet.cc:1722] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:26.734782 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 732c76659c0540bc8b6909b430ffb146 (table default.Table_a2a9717ed78046cab7c00077ac646665 [id=65dab4128d204227947dd4651a5aeb6c]) successfully deleted
I20251211 20:52:26.736004 710 leader_election.cc:304] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:26.736711 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 6c64fd8dd49b4e5d91c1ffb0fc2559e0 (DEFAULT_TABLE table=default.Table_cc48404bbbf24a7caae7641a8d42c247 [id=048a032339b3462ba676bcb44f4e2082]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.737214 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c64fd8dd49b4e5d91c1ffb0fc2559e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.739002 878 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a2442a4ebc940a281f9ea1a394f6621 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:26.736821 1168 ts_tablet_manager.cc:1403] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.072s user 0.003s sys 0.000s
I20251211 20:52:26.741484 750 tablet_service.cc:1505] Processing CreateTablet for tablet 87863daf77304a399318cdf56da9f9bb (DEFAULT_TABLE table=default.Table_28b13655ab2b47e68f2bd01d618a2b58 [id=9b92dd859a6641b8ade2b49515d1e810]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.741951 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87863daf77304a399318cdf56da9f9bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.744485 1160 tablet_bootstrap.cc:492] T 4870da0224e048ed82f227d7ba676a94 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:26.744678 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 87863daf77304a399318cdf56da9f9bb (DEFAULT_TABLE table=default.Table_28b13655ab2b47e68f2bd01d618a2b58 [id=9b92dd859a6641b8ade2b49515d1e810]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.744951 1168 raft_consensus.cc:359] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.745076 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87863daf77304a399318cdf56da9f9bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.745148 1168 raft_consensus.cc:385] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.745234 1168 raft_consensus.cc:740] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.745487 1168 consensus_queue.cc:260] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.746136 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 9662efc97a53461d888a062463e80ee6 (table default.Table_9d6a8e006ed241819eaecb22446277f5 [id=655f70b9f7e0400aa42179b922af2fa2]) successfully deleted
I20251211 20:52:26.746477 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 034f47d8564f4094a9d76aee2fea3119 (table default.Table_b5a3abc502a64d6083d78344400b5eeb [id=c58ec33f45c348329be4acee3d5f4f6f]) successfully deleted
I20251211 20:52:26.749253 1515 tablet.cc:1722] T 01ada50100bc48e39d1bc67082da750e P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:26.751540 1160 ts_tablet_manager.cc:1403] T 4870da0224e048ed82f227d7ba676a94 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.310s user 0.013s sys 0.034s
I20251211 20:52:26.751354 738 tablet_service.cc:1505] Processing CreateTablet for tablet c7bd218b3ff340b2a589c9517b49a649 (DEFAULT_TABLE table=default.Table_233d5484bd4e4456aed925bc271edfa5 [id=ca0c3c6978874d93a04b57cc0c425c83]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.752955 738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7bd218b3ff340b2a589c9517b49a649. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.751899 1420 raft_consensus.cc:2804] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:26.754611 1420 raft_consensus.cc:697] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:26.755331 1420 consensus_queue.cc:237] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.767052 1166 tablet_bootstrap.cc:492] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:26.767238 1166 ts_tablet_manager.cc:1403] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.191s user 0.003s sys 0.060s
I20251211 20:52:26.768074 1166 raft_consensus.cc:359] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.768293 1166 raft_consensus.cc:385] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.768374 1166 raft_consensus.cc:740] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.768654 1166 consensus_queue.cc:260] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.773226 1515 tablet.cc:1722] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:26.773379 1166 ts_tablet_manager.cc:1434] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20251211 20:52:26.776340 1166 tablet_bootstrap.cc:492] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:26.778893 1166 tablet_bootstrap.cc:654] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.781054 1510 tablet.cc:1722] T 694518e6c1ae4032b06b99e9bde2e5dd P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:26.781910 1512 raft_consensus.cc:493] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:26.782130 1512 raft_consensus.cc:515] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.782979 1512 leader_election.cc:290] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:26.783517 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ef87e052644c41bbbc54b8dad68ce152" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:26.784200 711 leader_election.cc:343] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:26.785876 1160 raft_consensus.cc:359] T 4870da0224e048ed82f227d7ba676a94 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.789021 1168 ts_tablet_manager.cc:1434] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.049s user 0.002s sys 0.000s
I20251211 20:52:26.789390 1168 tablet_bootstrap.cc:492] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:26.790285 1451 raft_consensus.cc:493] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:26.790448 1451 raft_consensus.cc:515] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.791115 1451 leader_election.cc:290] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:26.791249 1451 raft_consensus.cc:493] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:26.791283 1168 tablet_bootstrap.cc:654] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.791373 1451 raft_consensus.cc:515] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:26.791702 582 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 44775.3us
I20251211 20:52:26.791967 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet c58241f7e1404a8c8db92c4f9a47d143 (table default.Table_3570af5d8bff402ab3140ae51bae174c [id=90602e87d8b6425295bb04a5fca84ba7]) successfully deleted
W20251211 20:52:26.792207 582 catalog_manager.cc:4952] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 6c64fd8dd49b4e5d91c1ffb0fc2559e0 because the tablet was not found. No further retry: Not found: Tablet not found: 6c64fd8dd49b4e5d91c1ffb0fc2559e0
I20251211 20:52:26.792860 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 33cce042f33b4be198d11fd39f5f8838 (table default.Table_fdd4d94139714e3c954d6daf3c82a649 [id=5a7b1e562b084b109569a8733fd470a3]) successfully deleted
W20251211 20:52:26.793540 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 6a2442a4ebc940a281f9ea1a394f6621 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a2442a4ebc940a281f9ea1a394f6621 already in progress: creating tablet
I20251211 20:52:26.797917 1017 tablet_service.cc:1505] Processing CreateTablet for tablet c7bd218b3ff340b2a589c9517b49a649 (DEFAULT_TABLE table=default.Table_233d5484bd4e4456aed925bc271edfa5 [id=ca0c3c6978874d93a04b57cc0c425c83]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.798353 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7bd218b3ff340b2a589c9517b49a649. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.799608 738 tablet_service.cc:1505] Processing CreateTablet for tablet 8229b704e7464b30b7d2902a938ce33b (DEFAULT_TABLE table=default.Table_ea25e2f3ec124ff28f04f72f6f7121e1 [id=8fe47ac33eb2461784b859e64fbd3e24]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.799752 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 8229b704e7464b30b7d2902a938ce33b (DEFAULT_TABLE table=default.Table_ea25e2f3ec124ff28f04f72f6f7121e1 [id=8fe47ac33eb2461784b859e64fbd3e24]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.800098 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8229b704e7464b30b7d2902a938ce33b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.801702 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 6a526a6d1bc84cefad2c5d897223292e (DEFAULT_TABLE table=default.Table_254d66657cb14d0f8bbf713f794806f6 [id=f96a54c77528480ebf36ecc5741f7e17]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.802083 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a526a6d1bc84cefad2c5d897223292e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.804879 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 27a08a64d92342d08ab4406e8006a97b
I20251211 20:52:26.805285 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet ed9ee283045a44978422cd080aa2cda5
I20251211 20:52:26.805367 594 catalog_manager.cc:6724] ReplaceTablet: tablet 27a08a64d92342d08ab4406e8006a97b deleted and replaced by tablet bfb52935296443c58190d4d06e2e6e9c
I20251211 20:52:26.805778 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 27a08a64d92342d08ab4406e8006a97b with delete_type TABLET_DATA_DELETED (replaced by tablet bfb52935296443c58190d4d06e2e6e9c) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:26.806315 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d
I20251211 20:52:26.806351 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet ed9ee283045a44978422cd080aa2cda5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:26.806579 1163 tablet_replica.cc:333] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:26.806773 1163 raft_consensus.cc:2243] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:26.806818 593 catalog_manager.cc:6724] ReplaceTablet: tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d deleted and replaced by tablet e4647105ab57454d883ebfcde7bde891
I20251211 20:52:26.806967 1163 raft_consensus.cc:2272] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:26.807015 593 master_service.cc:954] ReplaceTablet: received request to replace tablet f93006227cc6489e91f3593e11789b54 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:26.807523 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d with delete_type TABLET_DATA_DELETED (replaced by tablet e4647105ab57454d883ebfcde7bde891) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:26.807791 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 27a08a64d92342d08ab4406e8006a97b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27a08a64d92342d08ab4406e8006a97b already in progress: creating tablet
I20251211 20:52:26.809229 1451 leader_election.cc:290] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:26.810768 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 8b89ae724b7f41c98741a41d75592da7
I20251211 20:52:26.811628 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b89ae724b7f41c98741a41d75592da7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:26.814620 739 tablet_service.cc:1552] Processing DeleteTablet for tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d with delete_type TABLET_DATA_DELETED (replaced by tablet e4647105ab57454d883ebfcde7bde891) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:26.815189 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b89ae724b7f41c98741a41d75592da7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:26.815755 752 tablet_service.cc:1552] Processing DeleteTablet for tablet ed9ee283045a44978422cd080aa2cda5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:26.815984 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 27a08a64d92342d08ab4406e8006a97b with delete_type TABLET_DATA_DELETED (replaced by tablet bfb52935296443c58190d4d06e2e6e9c) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:26.816670 750 tablet_service.cc:1505] Processing CreateTablet for tablet 6a526a6d1bc84cefad2c5d897223292e (DEFAULT_TABLE table=default.Table_254d66657cb14d0f8bbf713f794806f6 [id=f96a54c77528480ebf36ecc5741f7e17]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:26.817234 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d already in progress: creating tablet
I20251211 20:52:26.817303 738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8229b704e7464b30b7d2902a938ce33b. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:26.817476 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 8b89ae724b7f41c98741a41d75592da7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b89ae724b7f41c98741a41d75592da7 already in progress: creating tablet
I20251211 20:52:26.817907 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a526a6d1bc84cefad2c5d897223292e. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:26.818141 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet ed9ee283045a44978422cd080aa2cda5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ed9ee283045a44978422cd080aa2cda5 already in progress: creating tablet
W20251211 20:52:26.818392 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 27a08a64d92342d08ab4406e8006a97b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27a08a64d92342d08ab4406e8006a97b already in progress: creating tablet
I20251211 20:52:26.822495 1163 ts_tablet_manager.cc:1916] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:26.792613 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ed9ee283045a44978422cd080aa2cda5" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:26.823915 983 leader_election.cc:343] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:26.824133 1160 raft_consensus.cc:385] T 4870da0224e048ed82f227d7ba676a94 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.824263 1160 raft_consensus.cc:740] T 4870da0224e048ed82f227d7ba676a94 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.825098 1160 consensus_queue.cc:260] T 4870da0224e048ed82f227d7ba676a94 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.825642 1160 ts_tablet_manager.cc:1434] T 4870da0224e048ed82f227d7ba676a94 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.074s user 0.001s sys 0.001s
I20251211 20:52:26.794891 878 tablet_service.cc:1505] Processing CreateTablet for tablet 1a77e3e744ff42ccb68897704b4c2f1d (DEFAULT_TABLE table=default.Table_045fee0e26fa49119877f83dbd4b9159 [id=85ab3a18cda143e5a7f72e49e6671318]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.825943 1160 tablet_bootstrap.cc:492] T f92d26bcde3b410aa56bc23a61b1ac35 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:26.826176 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a77e3e744ff42ccb68897704b4c2f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.796291 889 tablet_service.cc:1505] Processing CreateTablet for tablet 6c64fd8dd49b4e5d91c1ffb0fc2559e0 (DEFAULT_TABLE table=default.Table_cc48404bbbf24a7caae7641a8d42c247 [id=048a032339b3462ba676bcb44f4e2082]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.829589 1163 ts_tablet_manager.cc:1929] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:26.829731 1163 log.cc:1199] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/ed9ee283045a44978422cd080aa2cda5
I20251211 20:52:26.803259 880 tablet_service.cc:1505] Processing CreateTablet for tablet e28d66ce4711430cb09a9650e99fbe9e (DEFAULT_TABLE table=default.Table_8b8ed719985f42128e56bc26105e7a9c [id=401822f4521f46eeb815147f643a6fcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.811300 890 tablet_service.cc:1505] Processing CreateTablet for tablet 87863daf77304a399318cdf56da9f9bb (DEFAULT_TABLE table=default.Table_28b13655ab2b47e68f2bd01d618a2b58 [id=9b92dd859a6641b8ade2b49515d1e810]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.830108 1163 ts_tablet_manager.cc:1950] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:26.813375 894 tablet_service.cc:1505] Processing CreateTablet for tablet c7bd218b3ff340b2a589c9517b49a649 (DEFAULT_TABLE table=default.Table_233d5484bd4e4456aed925bc271edfa5 [id=ca0c3c6978874d93a04b57cc0c425c83]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.817269 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "78922f26cedb4bb989aa1c1cb40bfa3a" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:26.830915 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet ed9ee283045a44978422cd080aa2cda5 (table default.Table_786e1caf9f674d48a6aad3ef7edc5b01 [id=33c5cbdd3d004439ab01b4ae69db56d1]) successfully deleted
W20251211 20:52:26.834421 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d already in progress: creating tablet
W20251211 20:52:26.835317 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 8b89ae724b7f41c98741a41d75592da7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b89ae724b7f41c98741a41d75592da7 already in progress: creating tablet
I20251211 20:52:26.835364 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ed9ee283045a44978422cd080aa2cda5" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:26.836324 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "78922f26cedb4bb989aa1c1cb40bfa3a" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:26.837033 710 leader_election.cc:343] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: ef87e052644c41bbbc54b8dad68ce152
I20251211 20:52:26.837229 710 leader_election.cc:304] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:26.837877 982 leader_election.cc:343] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:26.838096 982 leader_election.cc:304] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:26.838172 1517 raft_consensus.cc:2749] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:26.838431 982 raft_consensus.cc:2686] T ed9ee283045a44978422cd080aa2cda5 P e715a811a82446ce8d5bb4aafd5b81c7: unable to run election callback: Service unavailable: Thread pool token was shut down
W20251211 20:52:26.839119 982 leader_election.cc:343] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:26.843011 1508 tablet.cc:1722] T 694518e6c1ae4032b06b99e9bde2e5dd P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:26.843775 593 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet a5eeb56709f046d9be92ac600ebc330e (table default.Table_f2ce490e0c5d445ab84989aae77c2f19 [id=f78e1da479af4d23a552af1c4fc55f81]). Expected version 1 got 0
I20251211 20:52:26.846302 589 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 01ada50100bc48e39d1bc67082da750e (table default.Table_18d19a317a9c4656b4c764df4a64dbea [id=a24d8c58b9d84cdea2b278ae9c6e0ba5]). Expected version 2 got 0
W20251211 20:52:26.850729 983 leader_election.cc:343] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:26.850903 983 leader_election.cc:304] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:26.851783 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87863daf77304a399318cdf56da9f9bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.853322 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 340730a17a834a77bbc20b95aba98e2c
I20251211 20:52:26.853864 596 catalog_manager.cc:6724] ReplaceTablet: tablet 340730a17a834a77bbc20b95aba98e2c deleted and replaced by tablet 3b2c6064c5eb4c0f8979c485f5d85a75
I20251211 20:52:26.854642 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet 340730a17a834a77bbc20b95aba98e2c with delete_type TABLET_DATA_DELETED (replaced by tablet 3b2c6064c5eb4c0f8979c485f5d85a75) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:26.855472 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 340730a17a834a77bbc20b95aba98e2c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 340730a17a834a77bbc20b95aba98e2c already in progress: creating tablet
I20251211 20:52:26.857990 1522 raft_consensus.cc:2749] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:26.858741 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 340730a17a834a77bbc20b95aba98e2c with delete_type TABLET_DATA_DELETED (replaced by tablet 3b2c6064c5eb4c0f8979c485f5d85a75) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:26.859611 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 340730a17a834a77bbc20b95aba98e2c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 340730a17a834a77bbc20b95aba98e2c already in progress: creating tablet
I20251211 20:52:26.860146 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7bd218b3ff340b2a589c9517b49a649. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.861694 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c64fd8dd49b4e5d91c1ffb0fc2559e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.865999 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e28d66ce4711430cb09a9650e99fbe9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.868434 1166 tablet_bootstrap.cc:492] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:26.868623 1166 ts_tablet_manager.cc:1403] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.092s user 0.002s sys 0.002s
I20251211 20:52:26.869457 1166 raft_consensus.cc:359] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.869659 1166 raft_consensus.cc:385] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.869731 1166 raft_consensus.cc:740] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.869974 1166 consensus_queue.cc:260] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.871515 1166 ts_tablet_manager.cc:1434] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251211 20:52:26.871863 1166 tablet_bootstrap.cc:492] T dc40b9826ea84728910b1fd61ce62a25 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:26.873062 1160 tablet_bootstrap.cc:654] T f92d26bcde3b410aa56bc23a61b1ac35 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.876592 1166 tablet_bootstrap.cc:654] T dc40b9826ea84728910b1fd61ce62a25 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.880110 891 tablet_service.cc:1505] Processing CreateTablet for tablet 6a526a6d1bc84cefad2c5d897223292e (DEFAULT_TABLE table=default.Table_254d66657cb14d0f8bbf713f794806f6 [id=f96a54c77528480ebf36ecc5741f7e17]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.880589 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a526a6d1bc84cefad2c5d897223292e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.885149 1166 tablet_bootstrap.cc:492] T dc40b9826ea84728910b1fd61ce62a25 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:26.885331 1166 ts_tablet_manager.cc:1403] T dc40b9826ea84728910b1fd61ce62a25 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.006s
I20251211 20:52:26.886165 1166 raft_consensus.cc:359] T dc40b9826ea84728910b1fd61ce62a25 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.886392 1166 raft_consensus.cc:385] T dc40b9826ea84728910b1fd61ce62a25 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.886480 1166 raft_consensus.cc:740] T dc40b9826ea84728910b1fd61ce62a25 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.886725 1166 consensus_queue.cc:260] T dc40b9826ea84728910b1fd61ce62a25 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.887255 1166 ts_tablet_manager.cc:1434] T dc40b9826ea84728910b1fd61ce62a25 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:26.888242 894 tablet_service.cc:1505] Processing CreateTablet for tablet 8229b704e7464b30b7d2902a938ce33b (DEFAULT_TABLE table=default.Table_ea25e2f3ec124ff28f04f72f6f7121e1 [id=8fe47ac33eb2461784b859e64fbd3e24]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:26.888743 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8229b704e7464b30b7d2902a938ce33b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:26.894169 1166 tablet_bootstrap.cc:492] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:26.896028 1166 tablet_bootstrap.cc:654] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.902458 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_7215dcc06d384c7d94b29ff2ea8ee8ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:26.903066 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7215dcc06d384c7d94b29ff2ea8ee8ee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:26.903848 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_287197890af94935922681dd3ac7d185" } new_table_name: "default.Table_77601e74ced34f649b1f2227839ee6dd" modify_external_catalogs: true
I20251211 20:52:26.906909 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_8044d887ac4b408283fbb97b19ac6af6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:26.907450 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8044d887ac4b408283fbb97b19ac6af6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:26.907788 588 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_1bf756c0a806499fbfd10745831a653f" } new_table_name: "default.Table_59fd4cd22d6b4065bc70dc11849fa628" modify_external_catalogs: true
W20251211 20:52:26.914047 592 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2630ad57845a4c31b20c8a104dd27335 consensus state has no leader
I20251211 20:52:26.915033 1160 tablet_bootstrap.cc:492] T f92d26bcde3b410aa56bc23a61b1ac35 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:26.915217 1160 ts_tablet_manager.cc:1403] T f92d26bcde3b410aa56bc23a61b1ac35 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.089s user 0.002s sys 0.002s
W20251211 20:52:26.915483 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3eac21a07ba34e1fa275e25ee9dcabb1 consensus state has no leader
I20251211 20:52:26.915896 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_488913be2ade428593d6c0934e54a8f0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:26.916029 1160 raft_consensus.cc:359] T f92d26bcde3b410aa56bc23a61b1ac35 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.916270 1160 raft_consensus.cc:385] T f92d26bcde3b410aa56bc23a61b1ac35 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.916334 1160 raft_consensus.cc:740] T f92d26bcde3b410aa56bc23a61b1ac35 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
W20251211 20:52:26.916445 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_488913be2ade428593d6c0934e54a8f0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:26.916640 1160 consensus_queue.cc:260] T f92d26bcde3b410aa56bc23a61b1ac35 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.919389 1160 ts_tablet_manager.cc:1434] T f92d26bcde3b410aa56bc23a61b1ac35 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20251211 20:52:26.919772 1160 tablet_bootstrap.cc:492] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:26.923770 1160 tablet_bootstrap.cc:654] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:26.925141 595 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a2058b67ddf1413db338bfa2f64e7a77 has no consensus state
I20251211 20:52:26.925627 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_1c43c2cd6c064780936260e292759cf1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:26.926285 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1c43c2cd6c064780936260e292759cf1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:26.934171 878 tablet_service.cc:1552] Processing DeleteTablet for tablet 27a08a64d92342d08ab4406e8006a97b with delete_type TABLET_DATA_DELETED (replaced by tablet bfb52935296443c58190d4d06e2e6e9c) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:26.936566 1168 tablet_bootstrap.cc:492] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:26.936758 1168 ts_tablet_manager.cc:1403] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.147s user 0.000s sys 0.003s
I20251211 20:52:26.937070 878 tablet_service.cc:1552] Processing DeleteTablet for tablet ed9ee283045a44978422cd080aa2cda5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:26.937564 1168 raft_consensus.cc:359] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.937789 1168 raft_consensus.cc:385] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.937871 1168 raft_consensus.cc:740] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.938120 1168 consensus_queue.cc:260] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.938639 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d with delete_type TABLET_DATA_DELETED (replaced by tablet e4647105ab57454d883ebfcde7bde891) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:26.938683 1168 ts_tablet_manager.cc:1434] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:26.939106 1168 tablet_bootstrap.cc:492] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:26.939307 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b89ae724b7f41c98741a41d75592da7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:26.940860 1168 tablet_bootstrap.cc:654] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.941483 1160 tablet_bootstrap.cc:492] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:26.941704 1160 ts_tablet_manager.cc:1403] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.002s
I20251211 20:52:26.948936 1508 tablet.cc:1722] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:26.950712 1510 tablet.cc:1722] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:26.955636 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 340730a17a834a77bbc20b95aba98e2c with delete_type TABLET_DATA_DELETED (replaced by tablet 3b2c6064c5eb4c0f8979c485f5d85a75) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:26.957242 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_6bfc3963d4e346858acdfd9b9016245d" } modify_external_catalogs: true
I20251211 20:52:26.957448 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_6bfc3963d4e346858acdfd9b9016245d" } modify_external_catalogs: true
I20251211 20:52:26.959873 1160 raft_consensus.cc:359] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.960067 1160 raft_consensus.cc:385] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.960160 1160 raft_consensus.cc:740] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.964380 1160 consensus_queue.cc:260] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
W20251211 20:52:26.966188 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 27a08a64d92342d08ab4406e8006a97b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27a08a64d92342d08ab4406e8006a97b already in progress: creating tablet
W20251211 20:52:26.966392 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet ed9ee283045a44978422cd080aa2cda5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ed9ee283045a44978422cd080aa2cda5 already in progress: creating tablet
I20251211 20:52:26.966712 1515 tablet.cc:1722] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:26.968719 1160 ts_tablet_manager.cc:1434] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.021s user 0.002s sys 0.001s
I20251211 20:52:26.970991 1160 tablet_bootstrap.cc:492] T ef87e052644c41bbbc54b8dad68ce152 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:26.974761 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b072b759d1254859b0b8d401e3078792 consensus state has no leader
I20251211 20:52:26.974937 1166 tablet_bootstrap.cc:492] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:26.975127 1166 ts_tablet_manager.cc:1403] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.081s user 0.028s sys 0.004s
I20251211 20:52:26.976023 1166 raft_consensus.cc:359] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.976238 1166 raft_consensus.cc:385] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.976308 1166 raft_consensus.cc:740] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.976595 1166 consensus_queue.cc:260] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:26.977120 1166 ts_tablet_manager.cc:1434] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:26.977491 1166 tablet_bootstrap.cc:492] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:26.977901 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_069bb219e15e43be995ff011e2289a84" } modify_external_catalogs: true
I20251211 20:52:26.978085 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_069bb219e15e43be995ff011e2289a84" } modify_external_catalogs: true
W20251211 20:52:26.966547 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d already in progress: creating tablet
W20251211 20:52:26.980472 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 8b89ae724b7f41c98741a41d75592da7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b89ae724b7f41c98741a41d75592da7 already in progress: creating tablet
W20251211 20:52:26.980953 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 340730a17a834a77bbc20b95aba98e2c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 340730a17a834a77bbc20b95aba98e2c already in progress: creating tablet
I20251211 20:52:26.983943 1166 tablet_bootstrap.cc:654] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.984666 1168 tablet_bootstrap.cc:492] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:26.984828 1168 ts_tablet_manager.cc:1403] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.046s user 0.003s sys 0.000s
I20251211 20:52:26.987741 1168 raft_consensus.cc:359] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.987929 1168 raft_consensus.cc:385] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:26.988019 1168 raft_consensus.cc:740] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:26.988139 1166 tablet_bootstrap.cc:492] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:26.988282 1166 ts_tablet_manager.cc:1403] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20251211 20:52:26.988245 1168 consensus_queue.cc:260] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:26.988839 1168 ts_tablet_manager.cc:1434] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20251211 20:52:26.987478 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_f574e04a625b4f00bd2e2e8693a7f496"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:26.989426 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f574e04a625b4f00bd2e2e8693a7f496 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:26.990260 1168 tablet_bootstrap.cc:492] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:26.991088 1160 tablet_bootstrap.cc:654] T ef87e052644c41bbbc54b8dad68ce152 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:26.996624 1168 tablet_bootstrap.cc:654] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:26.999537 588 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fb2d30147cd74d1893783be1e1694d1f consensus state has no leader
I20251211 20:52:27.000000 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_9349d035f81c4ae0bc0bc8625d0b116c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:27.000559 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9349d035f81c4ae0bc0bc8625d0b116c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.000692 1166 raft_consensus.cc:359] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:27.000949 1166 raft_consensus.cc:385] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:27.001039 1166 raft_consensus.cc:740] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:27.001576 1522 raft_consensus.cc:493] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:27.001713 1522 raft_consensus.cc:515] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:27.002576 1522 leader_election.cc:290] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:27.003525 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7474a08c51cf4346ae9a518963d56b7a" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:27.004453 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7474a08c51cf4346ae9a518963d56b7a" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:27.004568 1160 tablet_bootstrap.cc:492] T ef87e052644c41bbbc54b8dad68ce152 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:27.004747 1160 ts_tablet_manager.cc:1403] T ef87e052644c41bbbc54b8dad68ce152 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.034s user 0.001s sys 0.006s
W20251211 20:52:27.005123 982 leader_election.cc:343] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:27.005507 1160 raft_consensus.cc:359] T ef87e052644c41bbbc54b8dad68ce152 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
W20251211 20:52:27.005669 983 leader_election.cc:343] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:27.005801 983 leader_election.cc:304] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:27.005825 1160 raft_consensus.cc:385] T ef87e052644c41bbbc54b8dad68ce152 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:27.005909 1160 raft_consensus.cc:740] T ef87e052644c41bbbc54b8dad68ce152 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:27.006151 1160 consensus_queue.cc:260] T ef87e052644c41bbbc54b8dad68ce152 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.007615 1160 ts_tablet_manager.cc:1434] T ef87e052644c41bbbc54b8dad68ce152 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:27.007929 1160 tablet_bootstrap.cc:492] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:27.008946 1522 raft_consensus.cc:2749] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:27.001343 1166 consensus_queue.cc:260] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:27.010982 1160 tablet_bootstrap.cc:654] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:27.012658 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_1d00ad67ef20403b94de5d39425be56d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:27.013387 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1d00ad67ef20403b94de5d39425be56d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.019526 1166 ts_tablet_manager.cc:1434] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.031s user 0.003s sys 0.000s
I20251211 20:52:27.022516 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_0c26504b5b4a4095b3a9e795754eb7d0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:27.022708 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c7e035d673434c588da10ea5a480f541" } modify_external_catalogs: true
I20251211 20:52:27.023448 1160 tablet_bootstrap.cc:492] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:27.025408 1168 tablet_bootstrap.cc:492] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:27.025611 1168 ts_tablet_manager.cc:1403] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.035s user 0.003s sys 0.000s
I20251211 20:52:27.026409 1168 raft_consensus.cc:359] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.026623 1168 raft_consensus.cc:385] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:27.026695 1168 raft_consensus.cc:740] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:27.026959 1168 consensus_queue.cc:260] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.028163 1166 tablet_bootstrap.cc:492] T 4b736a1b347d44b398980654caa21332 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:27.029274 1160 ts_tablet_manager.cc:1403] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.004s
I20251211 20:52:27.030135 1160 raft_consensus.cc:359] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:27.030247 1168 ts_tablet_manager.cc:1434] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251211 20:52:27.030422 1160 raft_consensus.cc:385] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:27.030573 1168 tablet_bootstrap.cc:492] T 2433068ec179490dac3e91d046a42b70 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:27.030566 1160 raft_consensus.cc:740] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:27.030923 1160 consensus_queue.cc:260] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:27.032442 1168 tablet_bootstrap.cc:654] T 2433068ec179490dac3e91d046a42b70 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:27.032631 1160 ts_tablet_manager.cc:1434] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:27.032821 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c7e035d673434c588da10ea5a480f541" } modify_external_catalogs: true
I20251211 20:52:27.036548 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_e6e48a22fc3d43b69d3dd391b9d1c60b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:27.037147 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e6e48a22fc3d43b69d3dd391b9d1c60b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:27.039973 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c26504b5b4a4095b3a9e795754eb7d0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.042410 596 master_service.cc:954] ReplaceTablet: received request to replace tablet 78922f26cedb4bb989aa1c1cb40bfa3a from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:27.042761 1166 tablet_bootstrap.cc:654] T 4b736a1b347d44b398980654caa21332 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:27.043419 1420 raft_consensus.cc:493] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:27.043635 1420 raft_consensus.cc:515] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.044289 1420 leader_election.cc:290] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:27.045313 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4870da0224e048ed82f227d7ba676a94" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:27.045535 1047 raft_consensus.cc:2468] T 4870da0224e048ed82f227d7ba676a94 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:27.046255 710 leader_election.cc:304] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:27.046769 1420 raft_consensus.cc:2804] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:27.046860 1420 raft_consensus.cc:493] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:27.046959 1420 raft_consensus.cc:3060] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:27.050668 1420 raft_consensus.cc:515] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.045871 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4870da0224e048ed82f227d7ba676a94" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:27.051847 913 raft_consensus.cc:2468] T 4870da0224e048ed82f227d7ba676a94 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:27.051924 1160 tablet_bootstrap.cc:492] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:27.054520 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4870da0224e048ed82f227d7ba676a94" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:27.054718 1047 raft_consensus.cc:3060] T 4870da0224e048ed82f227d7ba676a94 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:27.057669 1047 raft_consensus.cc:2468] T 4870da0224e048ed82f227d7ba676a94 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:27.058231 710 leader_election.cc:304] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:27.058614 1517 raft_consensus.cc:2804] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:27.058835 1517 raft_consensus.cc:697] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:27.059198 1517 consensus_queue.cc:237] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.064899 1160 tablet_bootstrap.cc:654] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:27.065428 1166 tablet_bootstrap.cc:492] T 4b736a1b347d44b398980654caa21332 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:27.065608 1166 ts_tablet_manager.cc:1403] T 4b736a1b347d44b398980654caa21332 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.037s user 0.012s sys 0.007s
I20251211 20:52:27.066663 1420 leader_election.cc:290] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:27.066608 1166 raft_consensus.cc:359] T 4b736a1b347d44b398980654caa21332 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:27.066797 1166 raft_consensus.cc:385] T 4b736a1b347d44b398980654caa21332 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:27.066854 1166 raft_consensus.cc:740] T 4b736a1b347d44b398980654caa21332 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:27.067085 1166 consensus_queue.cc:260] T 4b736a1b347d44b398980654caa21332 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:27.069373 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4870da0224e048ed82f227d7ba676a94" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:27.069645 913 raft_consensus.cc:3060] T 4870da0224e048ed82f227d7ba676a94 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:27.074013 1166 ts_tablet_manager.cc:1434] T 4b736a1b347d44b398980654caa21332 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20251211 20:52:27.085399 738 tablet_service.cc:1505] Processing CreateTablet for tablet ccc3ddd4b7f74f11a3835997cb7ae8d0 (DEFAULT_TABLE table=default.Table_63e0e08eb63c40b9aece6be45b6d4944 [id=8e1b3bb8f14e49d19eecdb540ff2d85d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.085542 592 catalog_manager.cc:5654] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } }
I20251211 20:52:27.085882 738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccc3ddd4b7f74f11a3835997cb7ae8d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.086115 592 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet a5eeb56709f046d9be92ac600ebc330e (table default.Table_f2ce490e0c5d445ab84989aae77c2f19 [id=f78e1da479af4d23a552af1c4fc55f81]). Expected version 1 got 0
I20251211 20:52:27.086459 592 catalog_manager.cc:5654] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } }
I20251211 20:52:27.086788 592 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 33e1d8a576cd41b1a34778b1458df396 (table default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]). Expected version 2 got 0
I20251211 20:52:27.088956 593 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_18d19a317a9c4656b4c764df4a64dbea [id=a24d8c58b9d84cdea2b278ae9c6e0ba5] alter complete (version 2)
I20251211 20:52:27.090263 1168 tablet_bootstrap.cc:492] T 2433068ec179490dac3e91d046a42b70 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:27.090407 1168 ts_tablet_manager.cc:1403] T 2433068ec179490dac3e91d046a42b70 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.060s user 0.005s sys 0.004s
I20251211 20:52:27.092414 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet ed9ee283045a44978422cd080aa2cda5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:27.092679 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet 78922f26cedb4bb989aa1c1cb40bfa3a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:27.092654 1168 raft_consensus.cc:359] T 2433068ec179490dac3e91d046a42b70 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.092844 1168 raft_consensus.cc:385] T 2433068ec179490dac3e91d046a42b70 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:27.092924 1168 raft_consensus.cc:740] T 2433068ec179490dac3e91d046a42b70 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:27.093078 1017 tablet_service.cc:1505] Processing CreateTablet for tablet ccc3ddd4b7f74f11a3835997cb7ae8d0 (DEFAULT_TABLE table=default.Table_63e0e08eb63c40b9aece6be45b6d4944 [id=8e1b3bb8f14e49d19eecdb540ff2d85d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.093127 1168 consensus_queue.cc:260] T 2433068ec179490dac3e91d046a42b70 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.093451 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccc3ddd4b7f74f11a3835997cb7ae8d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.094463 1420 raft_consensus.cc:493] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:27.094606 1420 raft_consensus.cc:515] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.095234 1420 leader_election.cc:290] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:27.095801 1168 ts_tablet_manager.cc:1434] T 2433068ec179490dac3e91d046a42b70 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251211 20:52:27.095953 1163 tablet_replica.cc:333] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:27.096089 1168 tablet_bootstrap.cc:492] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:27.096151 1163 raft_consensus.cc:2243] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:27.096311 1163 raft_consensus.cc:2272] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:27.097085 1163 ts_tablet_manager.cc:1916] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:27.097643 1168 tablet_bootstrap.cc:654] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:27.098894 749 tablet_service.cc:1505] Processing CreateTablet for tablet c7a2610f20df4c1ca81eae2fc7dba6bb (DEFAULT_TABLE table=default.Table_03b10e1dfcd94a08a73fd396ce0b6f98 [id=9a15a8a075af4969ae2c8b3be1bfed78]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.099304 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7a2610f20df4c1ca81eae2fc7dba6bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.099906 750 tablet_service.cc:1505] Processing CreateTablet for tablet 2d849c62f7434099a16f62c2a2fc5e85 (DEFAULT_TABLE table=default.Table_740dd6a3832643c49f5f2bf34e924c41 [id=2af76b9d32094a4ca656060a4478e08a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.100260 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d849c62f7434099a16f62c2a2fc5e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.103134 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 92a71dff4ea8445d8484fc2747471148
I20251211 20:52:27.103915 741 tablet_service.cc:1505] Processing CreateTablet for tablet 25454e67437541d9ac2bfab29bad55f9 (DEFAULT_TABLE table=default.Table_b0e00b07650849958a1a4486eeb3c660 [id=73346680c9a043cea75e13865bdc7bed]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.104106 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 7d4bba0aa7ea403e8dfcb74cec2e1f9b
I20251211 20:52:27.104257 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25454e67437541d9ac2bfab29bad55f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.105202 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 92a71dff4ea8445d8484fc2747471148 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:27.105479 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 7d4bba0aa7ea403e8dfcb74cec2e1f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:27.106247 581 catalog_manager.cc:4952] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 92a71dff4ea8445d8484fc2747471148 because the tablet was not found. No further retry: Not found: Tablet not found: 92a71dff4ea8445d8484fc2747471148
W20251211 20:52:27.106508 581 catalog_manager.cc:4952] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 7d4bba0aa7ea403e8dfcb74cec2e1f9b because the tablet was not found. No further retry: Not found: Tablet not found: 7d4bba0aa7ea403e8dfcb74cec2e1f9b
I20251211 20:52:27.108155 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 92a71dff4ea8445d8484fc2747471148 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:27.108206 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet 7d4bba0aa7ea403e8dfcb74cec2e1f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:27.108376 913 raft_consensus.cc:2468] T 4870da0224e048ed82f227d7ba676a94 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:27.108600 1025 tablet_service.cc:1505] Processing CreateTablet for tablet c7a2610f20df4c1ca81eae2fc7dba6bb (DEFAULT_TABLE table=default.Table_03b10e1dfcd94a08a73fd396ce0b6f98 [id=9a15a8a075af4969ae2c8b3be1bfed78]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.108971 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7a2610f20df4c1ca81eae2fc7dba6bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.111941 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 25454e67437541d9ac2bfab29bad55f9 (DEFAULT_TABLE table=default.Table_b0e00b07650849958a1a4486eeb3c660 [id=73346680c9a043cea75e13865bdc7bed]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.112309 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25454e67437541d9ac2bfab29bad55f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.112948 1163 ts_tablet_manager.cc:1929] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:27.113188 1163 log.cc:1199] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/78922f26cedb4bb989aa1c1cb40bfa3a
I20251211 20:52:27.113528 1163 ts_tablet_manager.cc:1950] T 78922f26cedb4bb989aa1c1cb40bfa3a P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:27.113751 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f92d26bcde3b410aa56bc23a61b1ac35" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:27.114013 1047 raft_consensus.cc:2468] T f92d26bcde3b410aa56bc23a61b1ac35 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:27.114526 710 leader_election.cc:304] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:27.119712 1420 raft_consensus.cc:2804] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:27.119838 1420 raft_consensus.cc:493] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:27.119942 1420 raft_consensus.cc:3060] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:27.120986 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet fb030a62796248869d46b682656eaeb0
I20251211 20:52:27.123955 589 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet c4e048ca28824329b924845ccf5067cf (table default.Table_4a0345a7a9eb4dfc89d48517fbe15ad1 [id=6b55ad645bec4a7daefc996978420295]). Expected version 1 got 0
I20251211 20:52:27.123957 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f92d26bcde3b410aa56bc23a61b1ac35" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:27.124183 913 raft_consensus.cc:2468] T f92d26bcde3b410aa56bc23a61b1ac35 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:27.125995 894 tablet_service.cc:1505] Processing CreateTablet for tablet ccc3ddd4b7f74f11a3835997cb7ae8d0 (DEFAULT_TABLE table=default.Table_63e0e08eb63c40b9aece6be45b6d4944 [id=8e1b3bb8f14e49d19eecdb540ff2d85d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.126463 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccc3ddd4b7f74f11a3835997cb7ae8d0. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:27.129438 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet ed9ee283045a44978422cd080aa2cda5 because the tablet was not found. No further retry: Not found: Tablet not found: ed9ee283045a44978422cd080aa2cda5
I20251211 20:52:27.129673 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 78922f26cedb4bb989aa1c1cb40bfa3a (table default.Table_736d6c2eb4dd4eb9a72dca6b9ed2073f [id=289e09e5a445461a9d0949c41ddcbd18]) successfully deleted
W20251211 20:52:27.129863 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 92a71dff4ea8445d8484fc2747471148 because the tablet was not found. No further retry: Not found: Tablet not found: 92a71dff4ea8445d8484fc2747471148
W20251211 20:52:27.130046 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 7d4bba0aa7ea403e8dfcb74cec2e1f9b because the tablet was not found. No further retry: Not found: Tablet not found: 7d4bba0aa7ea403e8dfcb74cec2e1f9b
I20251211 20:52:27.130719 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet fb030a62796248869d46b682656eaeb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:27.131230 750 tablet_service.cc:1552] Processing DeleteTablet for tablet fb030a62796248869d46b682656eaeb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:27.133524 597 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet f92d26bcde3b410aa56bc23a61b1ac35 (table default.Table_7c42321f3e494dc8835cc2db68e653e1 [id=bee5829ed7d44761a9ae57d3134d38dc]). Expected version 1 got 0
I20251211 20:52:27.133801 597 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 4870da0224e048ed82f227d7ba676a94 (table default.Table_5cb009a3a8b24126af4e28df1795d049 [id=28f3ebe9b7a64b5e89404b0407b0e5ae]). Expected version 1 got 0
I20251211 20:52:27.136257 878 tablet_service.cc:1505] Processing CreateTablet for tablet c7a2610f20df4c1ca81eae2fc7dba6bb (DEFAULT_TABLE table=default.Table_03b10e1dfcd94a08a73fd396ce0b6f98 [id=9a15a8a075af4969ae2c8b3be1bfed78]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.136682 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7a2610f20df4c1ca81eae2fc7dba6bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.141054 592 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_86450313ee0040bb87db3b340566266b [id=d49d5216afba41d68497f42fc72b1bf0] alter complete (version 2)
I20251211 20:52:27.144209 880 tablet_service.cc:1505] Processing CreateTablet for tablet 25454e67437541d9ac2bfab29bad55f9 (DEFAULT_TABLE table=default.Table_b0e00b07650849958a1a4486eeb3c660 [id=73346680c9a043cea75e13865bdc7bed]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.144644 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25454e67437541d9ac2bfab29bad55f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.145601 1168 tablet_bootstrap.cc:492] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:27.145771 1168 ts_tablet_manager.cc:1403] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.050s user 0.002s sys 0.006s
I20251211 20:52:27.146552 1168 raft_consensus.cc:359] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.146751 1168 raft_consensus.cc:385] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:27.146826 1168 raft_consensus.cc:740] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:27.147061 1168 consensus_queue.cc:260] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.147524 1168 ts_tablet_manager.cc:1434] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:27.151067 750 tablet_service.cc:1552] Processing DeleteTablet for tablet ef87e052644c41bbbc54b8dad68ce152 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:27.151413 1165 tablet_replica.cc:333] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:27.151646 1165 raft_consensus.cc:2243] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:27.151855 1165 raft_consensus.cc:2272] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:27.155978 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet fb030a62796248869d46b682656eaeb0 because the tablet was not found. No further retry: Not found: Tablet not found: fb030a62796248869d46b682656eaeb0
I20251211 20:52:27.157784 589 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_f2ce490e0c5d445ab84989aae77c2f19 [id=f78e1da479af4d23a552af1c4fc55f81] alter complete (version 1)
I20251211 20:52:27.158490 1165 ts_tablet_manager.cc:1916] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:27.164057 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_045fee0e26fa49119877f83dbd4b9159" } new_table_name: "default.Table_d70f80161dcd4063815a0e079c5821de" modify_external_catalogs: true
I20251211 20:52:27.164158 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_491fd0036d9e4f9fa410f3517c2c8286"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:27.164769 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_491fd0036d9e4f9fa410f3517c2c8286 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.166589 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_60c74dd24cd34456b535dc8445e13383" } new_table_name: "default.Table_fc80da3fac43460dabeea7e43bf098df" modify_external_catalogs: true
I20251211 20:52:27.167124 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b06a3971c3be404e9787e05466931423" } new_table_name: "default.Table_01b77d8f2f0c4c52b733bec1855fba9c" modify_external_catalogs: true
W20251211 20:52:27.167862 581 catalog_manager.cc:4952] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet fb030a62796248869d46b682656eaeb0 because the tablet was not found. No further retry: Not found: Tablet not found: fb030a62796248869d46b682656eaeb0
I20251211 20:52:27.168254 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b3e8496af4d340218ce858c81796a29c" } modify_external_catalogs: true
I20251211 20:52:27.168408 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b3e8496af4d340218ce858c81796a29c" } modify_external_catalogs: true
I20251211 20:52:27.168555 1165 ts_tablet_manager.cc:1929] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:27.168689 1165 log.cc:1199] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/ef87e052644c41bbbc54b8dad68ce152
I20251211 20:52:27.169064 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b736a1b347d44b398980654caa21332 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:27.169399 1163 tablet_replica.cc:333] T 4b736a1b347d44b398980654caa21332 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:27.169358 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet dc40b9826ea84728910b1fd61ce62a25 with delete_type TABLET_DATA_DELETED (replaced by tablet bc32a778e5424850b23ccaf74bb9df07) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:27.169601 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet b8828385cec34d70b717ae7f3813b9c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:27.169598 1163 raft_consensus.cc:2243] T 4b736a1b347d44b398980654caa21332 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:27.169853 1163 raft_consensus.cc:2272] T 4b736a1b347d44b398980654caa21332 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:27.169906 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_28b13655ab2b47e68f2bd01d618a2b58" } modify_external_catalogs: true
I20251211 20:52:27.169960 1165 ts_tablet_manager.cc:1950] T ef87e052644c41bbbc54b8dad68ce152 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:27.170086 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_28b13655ab2b47e68f2bd01d618a2b58" } modify_external_catalogs: true
I20251211 20:52:27.171051 1163 ts_tablet_manager.cc:1916] T 4b736a1b347d44b398980654caa21332 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:27.172308 590 catalog_manager.cc:5654] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } }
I20251211 20:52:27.172686 590 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 4870da0224e048ed82f227d7ba676a94 (table default.Table_5cb009a3a8b24126af4e28df1795d049 [id=28f3ebe9b7a64b5e89404b0407b0e5ae]). Expected version 1 got 0
I20251211 20:52:27.174633 750 tablet_service.cc:1505] Processing CreateTablet for tablet 4201da4241aa48dfb670a713e4453ca5 (DEFAULT_TABLE table=default.Table_4000fbd0bb614d0c902f539be2923421 [id=a8f3e7213c3040a3bb34e8827a1029bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.175022 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4201da4241aa48dfb670a713e4453ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.174957 752 tablet_service.cc:1505] Processing CreateTablet for tablet 92a71dff4ea8445d8484fc2747471148 (DEFAULT_TABLE table=default.Table_c7e035d673434c588da10ea5a480f541 [id=6a07ca569d504b8292feabcab2e3f70c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.175288 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92a71dff4ea8445d8484fc2747471148. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.175859 1024 tablet_service.cc:1505] Processing CreateTablet for tablet 4201da4241aa48dfb670a713e4453ca5 (DEFAULT_TABLE table=default.Table_4000fbd0bb614d0c902f539be2923421 [id=a8f3e7213c3040a3bb34e8827a1029bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.176292 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4201da4241aa48dfb670a713e4453ca5. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:27.188110 591 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1a77e3e744ff42ccb68897704b4c2f1d consensus state has no leader
I20251211 20:52:27.190905 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 2d849c62f7434099a16f62c2a2fc5e85 (DEFAULT_TABLE table=default.Table_740dd6a3832643c49f5f2bf34e924c41 [id=2af76b9d32094a4ca656060a4478e08a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.192467 744 tablet_service.cc:1505] Processing CreateTablet for tablet 2dde7f28e61d41f48015ba6e4f3757f2 (DEFAULT_TABLE table=default.Table_8b66a5217c494c8ebfa10a2b986e5e78 [id=499af44163bd46e7a93b9edcdd695ae7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.192919 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dde7f28e61d41f48015ba6e4f3757f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.192379 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 92a71dff4ea8445d8484fc2747471148 (DEFAULT_TABLE table=default.Table_c7e035d673434c588da10ea5a480f541 [id=6a07ca569d504b8292feabcab2e3f70c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.193533 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92a71dff4ea8445d8484fc2747471148. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.196748 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_18d19a317a9c4656b4c764df4a64dbea" } new_table_name: "default.Table_85e0192a945545faad36b2be67d76e2a" modify_external_catalogs: true
I20251211 20:52:27.199005 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet f7f52d2547534ef99255c5d3f59410c8
I20251211 20:52:27.199561 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 87863daf77304a399318cdf56da9f9bb
W20251211 20:52:27.200030 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 648f43e95e034f2188a20e1b837f2818 consensus state has no leader
W20251211 20:52:27.201007 592 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 49a33084c0fc460eb9592224b6dfc279 consensus state has no leader
I20251211 20:52:27.202090 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 2dde7f28e61d41f48015ba6e4f3757f2 (DEFAULT_TABLE table=default.Table_8b66a5217c494c8ebfa10a2b986e5e78 [id=499af44163bd46e7a93b9edcdd695ae7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.202450 754 tablet_service.cc:1505] Processing CreateTablet for tablet 4f807596fa6a4e66b89566b5d5dd1080 (DEFAULT_TABLE table=default.Table_ae0290b97d234621bbd1ff141b5c2400 [id=5d17fc32609b43c6b4fe461d3277dd2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.202534 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dde7f28e61d41f48015ba6e4f3757f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.202847 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f807596fa6a4e66b89566b5d5dd1080. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.204524 1163 ts_tablet_manager.cc:1929] T 4b736a1b347d44b398980654caa21332 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:27.204665 1163 log.cc:1199] T 4b736a1b347d44b398980654caa21332 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/4b736a1b347d44b398980654caa21332
I20251211 20:52:27.205065 1163 ts_tablet_manager.cc:1950] T 4b736a1b347d44b398980654caa21332 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:27.208148 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_e5def7ad31744669b691d5517da9c02b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:27.208717 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e5def7ad31744669b691d5517da9c02b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.208736 739 tablet_service.cc:1505] Processing CreateTablet for tablet bb112b6431af42a6a7b328dfdefbc2a2 (DEFAULT_TABLE table=default.Table_0c2f5186505f4a14b5c9bba07ca14178 [id=b4d34bd45a214d3b898045da7537ee10]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.209082 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb112b6431af42a6a7b328dfdefbc2a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.210903 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d849c62f7434099a16f62c2a2fc5e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.211107 1163 tablet_replica.cc:333] T dc40b9826ea84728910b1fd61ce62a25 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:27.207810 1024 tablet_service.cc:1505] Processing CreateTablet for tablet 4f807596fa6a4e66b89566b5d5dd1080 (DEFAULT_TABLE table=default.Table_ae0290b97d234621bbd1ff141b5c2400 [id=5d17fc32609b43c6b4fe461d3277dd2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.211328 1163 raft_consensus.cc:2243] T dc40b9826ea84728910b1fd61ce62a25 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:27.211515 1163 raft_consensus.cc:2272] T dc40b9826ea84728910b1fd61ce62a25 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:27.211526 1517 raft_consensus.cc:493] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:27.211678 1517 raft_consensus.cc:515] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.212291 1517 leader_election.cc:290] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:27.211540 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f807596fa6a4e66b89566b5d5dd1080. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.212697 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_5cb009a3a8b24126af4e28df1795d049" } new_table_name: "default.Table_595d8c100e69422ab5e6d44cce9e733c" modify_external_catalogs: true
I20251211 20:52:27.212764 1016 tablet_service.cc:1505] Processing CreateTablet for tablet bb112b6431af42a6a7b328dfdefbc2a2 (DEFAULT_TABLE table=default.Table_0c2f5186505f4a14b5c9bba07ca14178 [id=b4d34bd45a214d3b898045da7537ee10]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.213105 1016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb112b6431af42a6a7b328dfdefbc2a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.213115 745 tablet_service.cc:1505] Processing CreateTablet for tablet 0b267c7feaac43f6aacb7229759c7ec3 (DEFAULT_TABLE table=default.Table_e5979cadea8746f4afd0ae1b2673b272 [id=32aa02b009a64faca225b65dadd3b479]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.213438 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b267c7feaac43f6aacb7229759c7ec3. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:27.217052 710 leader_election.cc:343] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 4ef506349e5b434db7a78809dc44014e
I20251211 20:52:27.217809 1028 tablet_service.cc:1505] Processing CreateTablet for tablet 0b267c7feaac43f6aacb7229759c7ec3 (DEFAULT_TABLE table=default.Table_e5979cadea8746f4afd0ae1b2673b272 [id=32aa02b009a64faca225b65dadd3b479]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.218209 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b267c7feaac43f6aacb7229759c7ec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.218436 742 tablet_service.cc:1505] Processing CreateTablet for tablet fb030a62796248869d46b682656eaeb0 (DEFAULT_TABLE table=default.Table_069bb219e15e43be995ff011e2289a84 [id=bc0b945e86cf43fdb99d56658ca1576c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.218780 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb030a62796248869d46b682656eaeb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.222891 890 tablet_service.cc:1552] Processing DeleteTablet for tablet 92a71dff4ea8445d8484fc2747471148 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:27.225983 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 7d4bba0aa7ea403e8dfcb74cec2e1f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:27.226759 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_10e55a82b5d1413b8b68da9df264da06"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:27.227288 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_10e55a82b5d1413b8b68da9df264da06 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.227823 1163 ts_tablet_manager.cc:1916] T dc40b9826ea84728910b1fd61ce62a25 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:27.223698 878 tablet_service.cc:1505] Processing CreateTablet for tablet 2d849c62f7434099a16f62c2a2fc5e85 (DEFAULT_TABLE table=default.Table_740dd6a3832643c49f5f2bf34e924c41 [id=2af76b9d32094a4ca656060a4478e08a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.228433 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_9f6d83209f134db4a00fbddf8f323113"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:27.228900 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f6d83209f134db4a00fbddf8f323113 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.230013 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_00bd643f7bb84217970178aefb485a69"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:27.230453 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_00bd643f7bb84217970178aefb485a69 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.230868 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet 0246f1df99684d0da43d97acbf1182a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:27.231235 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_528b4823aba44a72a7af20911a8d2002"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:27.231462 883 tablet_service.cc:1552] Processing DeleteTablet for tablet ef87e052644c41bbbc54b8dad68ce152 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:27.231696 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_528b4823aba44a72a7af20911a8d2002 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.232849 1420 raft_consensus.cc:515] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.233142 1023 tablet_service.cc:1505] Processing CreateTablet for tablet fb030a62796248869d46b682656eaeb0 (DEFAULT_TABLE table=default.Table_069bb219e15e43be995ff011e2289a84 [id=bc0b945e86cf43fdb99d56658ca1576c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.233539 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb030a62796248869d46b682656eaeb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.233554 1420 leader_election.cc:290] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:27.234149 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f92d26bcde3b410aa56bc23a61b1ac35" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:27.234378 1047 raft_consensus.cc:3060] T f92d26bcde3b410aa56bc23a61b1ac35 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:27.237360 1047 raft_consensus.cc:2468] T f92d26bcde3b410aa56bc23a61b1ac35 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:27.237725 1163 ts_tablet_manager.cc:1929] T dc40b9826ea84728910b1fd61ce62a25 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:27.237856 1163 log.cc:1199] T dc40b9826ea84728910b1fd61ce62a25 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/dc40b9826ea84728910b1fd61ce62a25
I20251211 20:52:27.238603 1163 ts_tablet_manager.cc:1950] T dc40b9826ea84728910b1fd61ce62a25 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:27.238584 710 leader_election.cc:304] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:27.239199 1420 raft_consensus.cc:2804] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:27.239384 1420 raft_consensus.cc:697] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:27.239449 592 master_service.cc:954] ReplaceTablet: received request to replace tablet 1c04187b8ed146c289a6b17d62b07199 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:27.239699 1420 consensus_queue.cc:237] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.241348 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 4b736a1b347d44b398980654caa21332 (table default.Table_f9014306b36e47b98c8e4c6070ea4a37 [id=4e64e54fb4fc42d48679371e4bc1efe5]) successfully deleted
I20251211 20:52:27.241626 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 72e23f3033e14c9db6de8539830bac3b (DEFAULT_TABLE table=default.Table_70adbfea12154508bd1988d085007a3e [id=c7c19676e1484bbcb73b3bf14f147ae5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.241871 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet dc40b9826ea84728910b1fd61ce62a25 (table default.Table_880a8408f1fb4d68a3f9d30b26acffa5 [id=db38368266194271a02ad265fd991439]) successfully deleted
I20251211 20:52:27.242053 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72e23f3033e14c9db6de8539830bac3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.242323 743 tablet_service.cc:1505] Processing CreateTablet for tablet b631e6b3048e428f901af1b5b0b1ab3c (DEFAULT_TABLE table=default.Table_dd84a1d3fe964c76b146f798f6781baa [id=e80ce2c7f3aa48539b6d2fad357b3b49]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.242746 743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b631e6b3048e428f901af1b5b0b1ab3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.079841 1166 tablet_bootstrap.cc:492] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:27.245740 1166 tablet_bootstrap.cc:654] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:27.248248 753 tablet_service.cc:1505] Processing CreateTablet for tablet 52ef1707f6f2425aafd454f303af0333 (DEFAULT_TABLE table=default.Table_cedad3ae249f4796b8c3756c4f4d6f74 [id=c5b8a1dc09f146b782f6d67ac53f0e8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.248975 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 3b2c6064c5eb4c0f8979c485f5d85a75 (DEFAULT_TABLE table=default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.249131 1020 tablet_service.cc:1505] Processing CreateTablet for tablet b529b38d754f46ba8cb60b19e9705256 (DEFAULT_TABLE table=default.Table_0a0fab7f167b4e34b48892ee705fd1e2 [id=6e5fe0a78e4b48f98dac028802f1ddf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.249184 1021 tablet_service.cc:1505] Processing CreateTablet for tablet a2058b67ddf1413db338bfa2f64e7a77 (DEFAULT_TABLE table=default.Table_906e264983df4cbcb0dd73a1edd16c7f [id=e1b92959cd2f46469c90526286f30b27]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.249378 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b2c6064c5eb4c0f8979c485f5d85a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.249639 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b529b38d754f46ba8cb60b19e9705256. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.250679 1016 tablet_service.cc:1505] Processing CreateTablet for tablet b631e6b3048e428f901af1b5b0b1ab3c (DEFAULT_TABLE table=default.Table_dd84a1d3fe964c76b146f798f6781baa [id=e80ce2c7f3aa48539b6d2fad357b3b49]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.251154 1016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b631e6b3048e428f901af1b5b0b1ab3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.253793 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2058b67ddf1413db338bfa2f64e7a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.254498 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52ef1707f6f2425aafd454f303af0333. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.259917 1029 tablet_service.cc:1505] Processing CreateTablet for tablet c74427cb9d6f4a679e43eae796ff8c3f (DEFAULT_TABLE table=default.Table_31d573e7af144e9997dfe480f8466902 [id=f689abc175c94366bfda8217545156aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.260438 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c74427cb9d6f4a679e43eae796ff8c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.260464 747 tablet_service.cc:1505] Processing CreateTablet for tablet a2058b67ddf1413db338bfa2f64e7a77 (DEFAULT_TABLE table=default.Table_906e264983df4cbcb0dd73a1edd16c7f [id=e1b92959cd2f46469c90526286f30b27]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.260869 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2058b67ddf1413db338bfa2f64e7a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.151875 1168 tablet_bootstrap.cc:492] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:27.268339 1015 tablet_service.cc:1505] Processing CreateTablet for tablet 14e2650f8ec1482ea9511b989fdc963c (DEFAULT_TABLE table=default.Table_60175b4e1c9744e0b6aefa1dece53aa0 [id=2594e7a272e14856879b5d458c1a392b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.268666 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 87863daf77304a399318cdf56da9f9bb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:27.268709 756 tablet_service.cc:1505] Processing CreateTablet for tablet b529b38d754f46ba8cb60b19e9705256 (DEFAULT_TABLE table=default.Table_0a0fab7f167b4e34b48892ee705fd1e2 [id=6e5fe0a78e4b48f98dac028802f1ddf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.269203 1014 tablet_service.cc:1505] Processing CreateTablet for tablet f737a7bfb0f44196abdc831bb22d5285 (DEFAULT_TABLE table=default.Table_f12876c06fc04c63a2f23a6fe5b89f2b [id=a8664a1ea42f411bba24a8ac719155ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.269766 891 tablet_service.cc:1552] Processing DeleteTablet for tablet fb030a62796248869d46b682656eaeb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:27.268443 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet f7f52d2547534ef99255c5d3f59410c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:27.272109 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_55b2abeacec041418b6359c21e87b3d8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:27.272746 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_55b2abeacec041418b6359c21e87b3d8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.273907 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 2197822e0f11484f9679c4858626ac2c (DEFAULT_TABLE table=default.Table_a0c412985adf4382a7ea28c77654fd6c [id=4a74b9a60cbb4198b55cee630b2f4bef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.274327 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_40d59cf8fbb14f7e9ff94da21b038c0b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:27.274959 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_40d59cf8fbb14f7e9ff94da21b038c0b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.276233 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_54d8f0fdaacf4be8b1043f2bae13c56e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:27.276703 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_54d8f0fdaacf4be8b1043f2bae13c56e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.285686 757 tablet_service.cc:1505] Processing CreateTablet for tablet 72e23f3033e14c9db6de8539830bac3b (DEFAULT_TABLE table=default.Table_70adbfea12154508bd1988d085007a3e [id=c7c19676e1484bbcb73b3bf14f147ae5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.287055 741 tablet_service.cc:1505] Processing CreateTablet for tablet cce44b80c97948e1af7ac68e77cccfd6 (DEFAULT_TABLE table=default.Table_349fa502a7c44fe79baed28f7da89ff9 [id=1bd6b7930f2447659da365b7c06b1640]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.287321 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b529b38d754f46ba8cb60b19e9705256. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.287999 738 tablet_service.cc:1505] Processing CreateTablet for tablet 2197822e0f11484f9679c4858626ac2c (DEFAULT_TABLE table=default.Table_a0c412985adf4382a7ea28c77654fd6c [id=4a74b9a60cbb4198b55cee630b2f4bef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.288388 751 tablet_service.cc:1505] Processing CreateTablet for tablet 3b2c6064c5eb4c0f8979c485f5d85a75 (DEFAULT_TABLE table=default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.292402 1505 raft_consensus.cc:493] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:27.296840 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 97a519a8ca7c4f5380a230a70ee84052 with delete_type TABLET_DATA_DELETED (replaced by tablet 27a08a64d92342d08ab4406e8006a97b) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:27.297906 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f92d26bcde3b410aa56bc23a61b1ac35" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:27.298206 1505 raft_consensus.cc:515] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:27.299867 890 tablet_service.cc:1505] Processing CreateTablet for tablet 2dde7f28e61d41f48015ba6e4f3757f2 (DEFAULT_TABLE table=default.Table_8b66a5217c494c8ebfa10a2b986e5e78 [id=499af44163bd46e7a93b9edcdd695ae7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.300246 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2197822e0f11484f9679c4858626ac2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.300392 883 tablet_service.cc:1505] Processing CreateTablet for tablet 92a71dff4ea8445d8484fc2747471148 (DEFAULT_TABLE table=default.Table_c7e035d673434c588da10ea5a480f541 [id=6a07ca569d504b8292feabcab2e3f70c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.302014 1014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f737a7bfb0f44196abdc831bb22d5285. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.303787 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14e2650f8ec1482ea9511b989fdc963c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.305282 876 tablet_service.cc:1505] Processing CreateTablet for tablet 52ef1707f6f2425aafd454f303af0333 (DEFAULT_TABLE table=default.Table_cedad3ae249f4796b8c3756c4f4d6f74 [id=c5b8a1dc09f146b782f6d67ac53f0e8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.305806 888 tablet_service.cc:1505] Processing CreateTablet for tablet 72e23f3033e14c9db6de8539830bac3b (DEFAULT_TABLE table=default.Table_70adbfea12154508bd1988d085007a3e [id=c7c19676e1484bbcb73b3bf14f147ae5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.306273 885 tablet_service.cc:1505] Processing CreateTablet for tablet 3b2c6064c5eb4c0f8979c485f5d85a75 (DEFAULT_TABLE table=default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.306703 877 tablet_service.cc:1505] Processing CreateTablet for tablet b529b38d754f46ba8cb60b19e9705256 (DEFAULT_TABLE table=default.Table_0a0fab7f167b4e34b48892ee705fd1e2 [id=6e5fe0a78e4b48f98dac028802f1ddf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.307116 875 tablet_service.cc:1505] Processing CreateTablet for tablet a2058b67ddf1413db338bfa2f64e7a77 (DEFAULT_TABLE table=default.Table_906e264983df4cbcb0dd73a1edd16c7f [id=e1b92959cd2f46469c90526286f30b27]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.308076 882 tablet_service.cc:1505] Processing CreateTablet for tablet 2197822e0f11484f9679c4858626ac2c (DEFAULT_TABLE table=default.Table_a0c412985adf4382a7ea28c77654fd6c [id=4a74b9a60cbb4198b55cee630b2f4bef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.308609 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 52ef1707f6f2425aafd454f303af0333 (DEFAULT_TABLE table=default.Table_cedad3ae249f4796b8c3756c4f4d6f74 [id=c5b8a1dc09f146b782f6d67ac53f0e8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.318554 1026 tablet_service.cc:1505] Processing CreateTablet for tablet cce44b80c97948e1af7ac68e77cccfd6 (DEFAULT_TABLE table=default.Table_349fa502a7c44fe79baed28f7da89ff9 [id=1bd6b7930f2447659da365b7c06b1640]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.319340 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 793f543d6b614ad7b3ab8e717db2ceaf (DEFAULT_TABLE table=default.Table_bf0900ef3fb8447c834a43e509caa105 [id=861b00629ceb42588a1d1be34153b792]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.331663 1021 tablet_service.cc:1505] Processing CreateTablet for tablet e6ba12f94f5f4d688f4222a4c2fb5436 (DEFAULT_TABLE table=default.Table_70a00e0b1675435d9553c29cf9b0d3ea [id=3d49177647c544c2af198e8811dbbb42]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.334278 889 tablet_service.cc:1505] Processing CreateTablet for tablet 4201da4241aa48dfb670a713e4453ca5 (DEFAULT_TABLE table=default.Table_4000fbd0bb614d0c902f539be2923421 [id=a8f3e7213c3040a3bb34e8827a1029bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.335382 914 raft_consensus.cc:3060] T f92d26bcde3b410aa56bc23a61b1ac35 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:27.338333 914 raft_consensus.cc:2468] T f92d26bcde3b410aa56bc23a61b1ac35 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:27.338725 891 tablet_service.cc:1552] Processing DeleteTablet for tablet e41f052151624df39a3c83744870c94e with delete_type TABLET_DATA_DELETED (replaced by tablet b33fd6c238e049b382d8ec9da910fe92) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:27.339159 891 tablet_service.cc:1505] Processing CreateTablet for tablet 14e2650f8ec1482ea9511b989fdc963c (DEFAULT_TABLE table=default.Table_60175b4e1c9744e0b6aefa1dece53aa0 [id=2594e7a272e14856879b5d458c1a392b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.339553 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4ef506349e5b434db7a78809dc44014e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:27.340509 1505 leader_election.cc:290] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:27.340749 1164 tablet_replica.cc:333] T ef87e052644c41bbbc54b8dad68ce152 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:27.340853 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d849c62f7434099a16f62c2a2fc5e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.340945 1164 raft_consensus.cc:2243] T ef87e052644c41bbbc54b8dad68ce152 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:27.341150 1164 raft_consensus.cc:2272] T ef87e052644c41bbbc54b8dad68ce152 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:27.341212 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4201da4241aa48dfb670a713e4453ca5. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:27.341351 583 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 340730a17a834a77bbc20b95aba98e2c (table default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 340730a17a834a77bbc20b95aba98e2c consensus state has no leader
I20251211 20:52:27.341583 884 tablet_service.cc:1505] Processing CreateTablet for tablet 4f807596fa6a4e66b89566b5d5dd1080 (DEFAULT_TABLE table=default.Table_ae0290b97d234621bbd1ff141b5c2400 [id=5d17fc32609b43c6b4fe461d3277dd2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.341940 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f807596fa6a4e66b89566b5d5dd1080. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.343760 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14e2650f8ec1482ea9511b989fdc963c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.347673 881 tablet_service.cc:1505] Processing CreateTablet for tablet bb112b6431af42a6a7b328dfdefbc2a2 (DEFAULT_TABLE table=default.Table_0c2f5186505f4a14b5c9bba07ca14178 [id=b4d34bd45a214d3b898045da7537ee10]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.348047 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb112b6431af42a6a7b328dfdefbc2a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.351469 887 tablet_service.cc:1505] Processing CreateTablet for tablet 0b267c7feaac43f6aacb7229759c7ec3 (DEFAULT_TABLE table=default.Table_e5979cadea8746f4afd0ae1b2673b272 [id=32aa02b009a64faca225b65dadd3b479]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.351851 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b267c7feaac43f6aacb7229759c7ec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.355666 893 tablet_service.cc:1505] Processing CreateTablet for tablet fb030a62796248869d46b682656eaeb0 (DEFAULT_TABLE table=default.Table_069bb219e15e43be995ff011e2289a84 [id=bc0b945e86cf43fdb99d56658ca1576c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.356040 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb030a62796248869d46b682656eaeb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.359098 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52ef1707f6f2425aafd454f303af0333. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.360476 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b2c6064c5eb4c0f8979c485f5d85a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.361115 1023 tablet_service.cc:1505] Processing CreateTablet for tablet e4b5a7553110424d901788c7a2ecdce6 (DEFAULT_TABLE table=default.Table_9ee6894ba6364a54be16bb427ac7197b [id=04d7acd50017488eaa11f783233b6d4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.361829 892 tablet_service.cc:1505] Processing CreateTablet for tablet cce44b80c97948e1af7ac68e77cccfd6 (DEFAULT_TABLE table=default.Table_349fa502a7c44fe79baed28f7da89ff9 [id=1bd6b7930f2447659da365b7c06b1640]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.362142 1018 tablet_service.cc:1505] Processing CreateTablet for tablet 8b793241ee3b42b1a38e9f2f05e1173b (DEFAULT_TABLE table=default.Table_8adf3dfe2fb0445385c730198775d1e9 [id=279f39d718ba45318f52f95e8aa13652]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.362793 886 tablet_service.cc:1505] Processing CreateTablet for tablet c74427cb9d6f4a679e43eae796ff8c3f (DEFAULT_TABLE table=default.Table_31d573e7af144e9997dfe480f8466902 [id=f689abc175c94366bfda8217545156aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.363077 1164 ts_tablet_manager.cc:1916] T ef87e052644c41bbbc54b8dad68ce152 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:27.363828 1020 tablet_service.cc:1505] Processing CreateTablet for tablet bfb52935296443c58190d4d06e2e6e9c (DEFAULT_TABLE table=default.Table_56733fc3806a4af7bf52401c572bb186 [id=383392d835da4c50848068d83261cecc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.364349 1028 tablet_service.cc:1505] Processing CreateTablet for tablet 676faec7f3d1488f99b220f8e968e9a0 (DEFAULT_TABLE table=default.Table_7aa604df1f25488f9f06c6b0e6daaf88 [id=a2bbfccfe3974c378521d5ef2333df86]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:27.364555 847 leader_election.cc:343] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 97a519a8ca7c4f5380a230a70ee84052
W20251211 20:52:27.364701 847 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 55181.4us
I20251211 20:52:27.368695 879 tablet_service.cc:1505] Processing CreateTablet for tablet b631e6b3048e428f901af1b5b0b1ab3c (DEFAULT_TABLE table=default.Table_dd84a1d3fe964c76b146f798f6781baa [id=e80ce2c7f3aa48539b6d2fad357b3b49]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.379169 1522 raft_consensus.cc:493] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:27.385000 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6ba12f94f5f4d688f4222a4c2fb5436. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.391657 1029 tablet_service.cc:1505] Processing CreateTablet for tablet e4647105ab57454d883ebfcde7bde891 (DEFAULT_TABLE table=default.Table_ae7f102bdee3470990e863235f08d636 [id=d555d1fc6aef4a21b4d3f2804264d030]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.392092 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4647105ab57454d883ebfcde7bde891. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.394347 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dde7f28e61d41f48015ba6e4f3757f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.395741 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b793241ee3b42b1a38e9f2f05e1173b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.401775 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4b5a7553110424d901788c7a2ecdce6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.407614 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 676faec7f3d1488f99b220f8e968e9a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.407830 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cce44b80c97948e1af7ac68e77cccfd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.411824 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 793f543d6b614ad7b3ab8e717db2ceaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.416095 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cce44b80c97948e1af7ac68e77cccfd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.419028 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c74427cb9d6f4a679e43eae796ff8c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.419906 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfb52935296443c58190d4d06e2e6e9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.427358 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b631e6b3048e428f901af1b5b0b1ab3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.427554 1522 raft_consensus.cc:515] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:27.428243 1522 leader_election.cc:290] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:27.428838 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 7d4bba0aa7ea403e8dfcb74cec2e1f9b (DEFAULT_TABLE table=default.Table_6bfc3963d4e346858acdfd9b9016245d [id=4ba40e6e45c7463f9730a2380b5ff856]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.429214 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d4bba0aa7ea403e8dfcb74cec2e1f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.433370 1017 tablet_service.cc:1505] Processing CreateTablet for tablet bb0fc179a337469eae056c5bb3815e6b (DEFAULT_TABLE table=default.Table_c27589191ac249f4a9460f5bbae2611b [id=461d63ce3b4f4d12b5f395bb8c1cdbb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.433996 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb0fc179a337469eae056c5bb3815e6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.433455 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b8828385cec34d70b717ae7f3813b9c3" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:27.436280 983 leader_election.cc:343] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:27.436569 884 tablet_service.cc:1552] Processing DeleteTablet for tablet f7f52d2547534ef99255c5d3f59410c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:27.438155 582 catalog_manager.cc:4952] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 92a71dff4ea8445d8484fc2747471148 because the tablet was not found. No further retry: Not found: Tablet not found: 92a71dff4ea8445d8484fc2747471148
W20251211 20:52:27.439705 582 catalog_manager.cc:4952] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 7d4bba0aa7ea403e8dfcb74cec2e1f9b because the tablet was not found. No further retry: Not found: Tablet not found: 7d4bba0aa7ea403e8dfcb74cec2e1f9b
I20251211 20:52:27.447676 1163 tablet_replica.cc:333] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:27.448906 1163 raft_consensus.cc:2243] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:27.449201 1163 raft_consensus.cc:2272] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:27.450279 876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52ef1707f6f2425aafd454f303af0333. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.451030 1163 ts_tablet_manager.cc:1916] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:27.457213 884 tablet_service.cc:1505] Processing CreateTablet for tablet f737a7bfb0f44196abdc831bb22d5285 (DEFAULT_TABLE table=default.Table_f12876c06fc04c63a2f23a6fe5b89f2b [id=a8664a1ea42f411bba24a8ac719155ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.460986 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 87863daf77304a399318cdf56da9f9bb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:27.469573 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72e23f3033e14c9db6de8539830bac3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.472610 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b2c6064c5eb4c0f8979c485f5d85a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.474027 1517 raft_consensus.cc:493] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:27.474337 1512 raft_consensus.cc:493] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251211 20:52:27.476013 711 leader_election.cc:343] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:27.476147 711 leader_election.cc:304] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:27.476284 1517 raft_consensus.cc:515] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.476848 1512 raft_consensus.cc:515] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.477420 1512 leader_election.cc:290] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:27.473223 748 tablet_service.cc:1505] Processing CreateTablet for tablet c74427cb9d6f4a679e43eae796ff8c3f (DEFAULT_TABLE table=default.Table_31d573e7af144e9997dfe480f8466902 [id=f689abc175c94366bfda8217545156aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.482558 1517 leader_election.cc:290] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:27.482729 889 tablet_service.cc:1505] Processing CreateTablet for tablet bb0fc179a337469eae056c5bb3815e6b (DEFAULT_TABLE table=default.Table_c27589191ac249f4a9460f5bbae2611b [id=461d63ce3b4f4d12b5f395bb8c1cdbb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.482303 1018 tablet_service.cc:1505] Processing CreateTablet for tablet 260bdeb73439437cbf234d20f7264ab3 (DEFAULT_TABLE table=default.Table_10e55a82b5d1413b8b68da9df264da06 [id=7c485abbfb12494c89c81b0b51965923]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.482170 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 4ce495233a5e4f3792faf44680df54ba (DEFAULT_TABLE table=default.Table_488913be2ade428593d6c0934e54a8f0 [id=f8adc6a8019d4b6b949eb34bde21031a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.484015 1021 tablet_service.cc:1505] Processing CreateTablet for tablet a9bb22dce13f4c9da76ca446715cd99a (DEFAULT_TABLE table=default.Table_528b4823aba44a72a7af20911a8d2002 [id=940343c556c54c4fb2110f4f47ca91f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.484491 1016 tablet_service.cc:1505] Processing CreateTablet for tablet 645b08b33676420c96e880de3d460829 (DEFAULT_TABLE table=default.Table_e5def7ad31744669b691d5517da9c02b [id=6d42f260246e420d8818b3841926302b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.485570 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 260bdeb73439437cbf234d20f7264ab3. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:27.486773 847 leader_election.cc:343] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 97a519a8ca7c4f5380a230a70ee84052
I20251211 20:52:27.486581 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ce495233a5e4f3792faf44680df54ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.489203 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 5710fc2f24764f818f7e5c86c6fd985e (DEFAULT_TABLE table=default.Table_491fd0036d9e4f9fa410f3517c2c8286 [id=0e93a84ec95341889e108170806b2525]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.490608 1024 tablet_service.cc:1505] Processing CreateTablet for tablet 4df68d1b6a91458e9f212ae81aa25438 (DEFAULT_TABLE table=default.Table_f574e04a625b4f00bd2e2e8693a7f496 [id=e266c3e0cd804232bcecc4db812ca13b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.491621 1014 tablet_service.cc:1505] Processing CreateTablet for tablet 2207af105bc3471ca930d35a91d38e70 (DEFAULT_TABLE table=default.Table_1c43c2cd6c064780936260e292759cf1 [id=767f50bee5464316a18f73e64bab817a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.492399 1015 tablet_service.cc:1505] Processing CreateTablet for tablet 68792ccef54944a09af9b5456e9ff1fe (DEFAULT_TABLE table=default.Table_00bd643f7bb84217970178aefb485a69 [id=1b90cdd71d064d27948a1a9a7f3f852b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.489697 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9bb22dce13f4c9da76ca446715cd99a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.493752 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ddf1bd1e18f4187bf050d6f7767b722 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:27.494496 1165 tablet_replica.cc:333] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:27.494978 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 2433068ec179490dac3e91d046a42b70 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:27.495383 742 tablet_service.cc:1552] Processing DeleteTablet for tablet f7f52d2547534ef99255c5d3f59410c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:27.495609 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 7ec261837cf04bf88a48ff8add1b0d66 (DEFAULT_TABLE table=default.Table_e6e48a22fc3d43b69d3dd391b9d1c60b [id=1921595e79a7402dad44c6aee6417e78]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.495945 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ec261837cf04bf88a48ff8add1b0d66. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.496191 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68792ccef54944a09af9b5456e9ff1fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.496733 1014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2207af105bc3471ca930d35a91d38e70. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.489625 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 28ddb8a59a214576a25d613d42c2fef5 (DEFAULT_TABLE table=default.Table_e98180a63672423688b0de80af514e8b [id=658bb4c0b37e492c94080811d899c16b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.489492 1028 tablet_service.cc:1505] Processing CreateTablet for tablet fed300d31c414f4a9b27c66e78097688 (DEFAULT_TABLE table=default.Table_9f6d83209f134db4a00fbddf8f323113 [id=e7d1c65b881746eeb12169e26fe0db70]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.499825 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4df68d1b6a91458e9f212ae81aa25438. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.503217 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28ddb8a59a214576a25d613d42c2fef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.504691 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b8828385cec34d70b717ae7f3813b9c3" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:27.505312 1022 tablet_service.cc:1505] Processing CreateTablet for tablet bb4004212441473483a209d23c0800ce (DEFAULT_TABLE table=default.Table_55b2abeacec041418b6359c21e87b3d8 [id=75609081b0c7488eb388faac157db3e5]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:27.505354 982 leader_election.cc:343] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:27.505913 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5710fc2f24764f818f7e5c86c6fd985e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.507262 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2197822e0f11484f9679c4858626ac2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.508971 982 leader_election.cc:304] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:27.511993 1026 tablet_service.cc:1505] Processing CreateTablet for tablet a3b1aadf17ea4bf0a896d23cd543fca7 (DEFAULT_TABLE table=default.Table_1d00ad67ef20403b94de5d39425be56d [id=a07662f8dc3f4178be5dfcc1cdc85837]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.515761 847 leader_election.cc:304] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:27.523784 894 tablet_service.cc:1505] Processing CreateTablet for tablet 793f543d6b614ad7b3ab8e717db2ceaf (DEFAULT_TABLE table=default.Table_bf0900ef3fb8447c834a43e509caa105 [id=861b00629ceb42588a1d1be34153b792]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.526669 1512 raft_consensus.cc:493] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:27.528420 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_6ea777b3e1c3474f926a3e1ab782c69b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:27.528604 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0c2f5186505f4a14b5c9bba07ca14178" } modify_external_catalogs: true
I20251211 20:52:27.528791 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0c2f5186505f4a14b5c9bba07ca14178" } modify_external_catalogs: true
W20251211 20:52:27.529050 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6ea777b3e1c3474f926a3e1ab782c69b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.529894 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4ddf1bd1e18f4187bf050d6f7767b722" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:27.530252 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_40d59cf8fbb14f7e9ff94da21b038c0b" } modify_external_catalogs: true
I20251211 20:52:27.530412 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_40d59cf8fbb14f7e9ff94da21b038c0b" } modify_external_catalogs: true
I20251211 20:52:27.491654 1166 tablet_bootstrap.cc:492] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:27.531035 1166 ts_tablet_manager.cc:1403] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.451s user 0.005s sys 0.013s
I20251211 20:52:27.531528 1517 raft_consensus.cc:2749] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:27.531638 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ebec874d437748c487f8bf9d0ccaa7fa" } modify_external_catalogs: true
I20251211 20:52:27.531773 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ebec874d437748c487f8bf9d0ccaa7fa" } modify_external_catalogs: true
I20251211 20:52:27.533737 743 tablet_service.cc:1552] Processing DeleteTablet for tablet 87863daf77304a399318cdf56da9f9bb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:27.533782 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 0129162969d74f5da3c08a1e6a27ab22 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:27.534101 593 master_service.cc:954] ReplaceTablet: received request to replace tablet 88066d94d5d34e0fab20da9fb6ef178e from {username='slave'} at 127.0.0.1:44772
W20251211 20:52:27.534655 982 raft_consensus.cc:2686] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251211 20:52:27.535836 738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2197822e0f11484f9679c4858626ac2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.531906 1512 raft_consensus.cc:515] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.537374 755 tablet_service.cc:1505] Processing CreateTablet for tablet 14e2650f8ec1482ea9511b989fdc963c (DEFAULT_TABLE table=default.Table_60175b4e1c9744e0b6aefa1dece53aa0 [id=2594e7a272e14856879b5d458c1a392b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.544348 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 22273287aa9245a2a66131fed2c8861c
I20251211 20:52:27.544865 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_53c0c6163e2447959bc8eb3303c1685f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:27.539408 1013 tablet_service.cc:1505] Processing CreateTablet for tablet 3de5b34b96404cdc9c191fed7f055638 (DEFAULT_TABLE table=default.Table_7215dcc06d384c7d94b29ff2ea8ee8ee [id=593f8b9b01134fb09a1c7232509635bb]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:27.545364 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_53c0c6163e2447959bc8eb3303c1685f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.546015 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 4d98967714184fe6a3e38dd4ca7979dc
I20251211 20:52:27.547813 1165 raft_consensus.cc:2243] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:27.548000 1165 raft_consensus.cc:2272] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:27.548569 1165 ts_tablet_manager.cc:1916] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:27.548691 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cce44b80c97948e1af7ac68e77cccfd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.547557 752 tablet_service.cc:1505] Processing CreateTablet for tablet 793f543d6b614ad7b3ab8e717db2ceaf (DEFAULT_TABLE table=default.Table_bf0900ef3fb8447c834a43e509caa105 [id=861b00629ceb42588a1d1be34153b792]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.553927 743 tablet_service.cc:1505] Processing CreateTablet for tablet bfb52935296443c58190d4d06e2e6e9c (DEFAULT_TABLE table=default.Table_56733fc3806a4af7bf52401c572bb186 [id=383392d835da4c50848068d83261cecc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.555655 753 tablet_service.cc:1505] Processing CreateTablet for tablet e6ba12f94f5f4d688f4222a4c2fb5436 (DEFAULT_TABLE table=default.Table_70a00e0b1675435d9553c29cf9b0d3ea [id=3d49177647c544c2af198e8811dbbb42]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.557965 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 88066d94d5d34e0fab20da9fb6ef178e
I20251211 20:52:27.558485 593 catalog_manager.cc:6724] ReplaceTablet: tablet 88066d94d5d34e0fab20da9fb6ef178e deleted and replaced by tablet 6a61cf8f5de14adaa4a1de5bddde9a2f
I20251211 20:52:27.559084 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet bb112b6431af42a6a7b328dfdefbc2a2
I20251211 20:52:27.560328 588 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_56733fc3806a4af7bf52401c572bb186" } new_table_name: "default.Table_47249158f89f4c16af084f0e1147ef9f" modify_external_catalogs: true
I20251211 20:52:27.562224 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72e23f3033e14c9db6de8539830bac3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.562259 1160 tablet_bootstrap.cc:492] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:27.562194 880 tablet_service.cc:1505] Processing CreateTablet for tablet bfb52935296443c58190d4d06e2e6e9c (DEFAULT_TABLE table=default.Table_56733fc3806a4af7bf52401c572bb186 [id=383392d835da4c50848068d83261cecc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.566006 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d19975cc513e414b80f478a6c5eb09de" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:27.571038 1505 raft_consensus.cc:2749] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:27.571440 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb0fc179a337469eae056c5bb3815e6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.555378 746 tablet_service.cc:1505] Processing CreateTablet for tablet f737a7bfb0f44196abdc831bb22d5285 (DEFAULT_TABLE table=default.Table_f12876c06fc04c63a2f23a6fe5b89f2b [id=a8664a1ea42f411bba24a8ac719155ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.574733 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14e2650f8ec1482ea9511b989fdc963c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.576881 891 tablet_service.cc:1505] Processing CreateTablet for tablet e6ba12f94f5f4d688f4222a4c2fb5436 (DEFAULT_TABLE table=default.Table_70a00e0b1675435d9553c29cf9b0d3ea [id=3d49177647c544c2af198e8811dbbb42]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.577961 881 tablet_service.cc:1505] Processing CreateTablet for tablet e4b5a7553110424d901788c7a2ecdce6 (DEFAULT_TABLE table=default.Table_9ee6894ba6364a54be16bb427ac7197b [id=04d7acd50017488eaa11f783233b6d4b]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:27.578488 588 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bfb52935296443c58190d4d06e2e6e9c consensus state has no leader
I20251211 20:52:27.578629 1512 leader_election.cc:290] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:27.579424 754 tablet_service.cc:1505] Processing CreateTablet for tablet bb0fc179a337469eae056c5bb3815e6b (DEFAULT_TABLE table=default.Table_c27589191ac249f4a9460f5bbae2611b [id=461d63ce3b4f4d12b5f395bb8c1cdbb8]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:27.578065 711 leader_election.cc:343] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:27.579983 750 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ef506349e5b434db7a78809dc44014e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:27.581579 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92a71dff4ea8445d8484fc2747471148. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:27.581715 847 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.1:42177, user_credentials={real_user=slave}} blocked reactor thread for 80433.8us
I20251211 20:52:27.583642 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b529b38d754f46ba8cb60b19e9705256. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.583889 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f737a7bfb0f44196abdc831bb22d5285. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.584381 1160 ts_tablet_manager.cc:1403] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.511s user 0.020s sys 0.069s
I20251211 20:52:27.584712 743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfb52935296443c58190d4d06e2e6e9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.585673 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 793f543d6b614ad7b3ab8e717db2ceaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.586122 1168 tablet_bootstrap.cc:654] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:27.586650 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c74427cb9d6f4a679e43eae796ff8c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.587423 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f737a7bfb0f44196abdc831bb22d5285. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.588075 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9349d035f81c4ae0bc0bc8625d0b116c" } modify_external_catalogs: true
I20251211 20:52:27.588261 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9349d035f81c4ae0bc0bc8625d0b116c" } modify_external_catalogs: true
I20251211 20:52:27.589039 875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2058b67ddf1413db338bfa2f64e7a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.589923 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6ba12f94f5f4d688f4222a4c2fb5436. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.590209 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_70a00e0b1675435d9553c29cf9b0d3ea" } modify_external_catalogs: true
I20251211 20:52:27.590375 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_70a00e0b1675435d9553c29cf9b0d3ea" } modify_external_catalogs: true
W20251211 20:52:27.590993 711 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 48816.6us
W20251211 20:52:27.591868 711 leader_election.cc:343] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:27.581199 740 tablet_service.cc:1505] Processing CreateTablet for tablet e4647105ab57454d883ebfcde7bde891 (DEFAULT_TABLE table=default.Table_ae7f102bdee3470990e863235f08d636 [id=d555d1fc6aef4a21b4d3f2804264d030]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.592214 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_6669f84cc5f04b8fb74c1d226054f2b8" } new_table_name: "default.Table_734c2082e7ad4fd4a10082f89e767abe" modify_external_catalogs: true
I20251211 20:52:27.592851 750 tablet_service.cc:1505] Processing CreateTablet for tablet 8b793241ee3b42b1a38e9f2f05e1173b (DEFAULT_TABLE table=default.Table_8adf3dfe2fb0445385c730198775d1e9 [id=279f39d718ba45318f52f95e8aa13652]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.593565 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_15a4631cb9ae4a8d892505a404f973f1" } new_table_name: "default.Table_370d3069ceee4b8e8856bd74fe3a1815" modify_external_catalogs: true
I20251211 20:52:27.593681 739 tablet_service.cc:1505] Processing CreateTablet for tablet 7d4bba0aa7ea403e8dfcb74cec2e1f9b (DEFAULT_TABLE table=default.Table_6bfc3963d4e346858acdfd9b9016245d [id=4ba40e6e45c7463f9730a2380b5ff856]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.594746 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_906e264983df4cbcb0dd73a1edd16c7f" } new_table_name: "default.Table_91453272b5a341baab304eedbc5b2b34" modify_external_catalogs: true
I20251211 20:52:27.595882 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_f4fa28c6ade240c69a5b4a98b44eac5a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:27.596378 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f4fa28c6ade240c69a5b4a98b44eac5a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.596679 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_77601e74ced34f649b1f2227839ee6dd" } new_table_name: "default.Table_dbd8238e6f3c477085d6f3d231cc68fb" modify_external_catalogs: true
I20251211 20:52:27.597659 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_e991174a45694a1fa40bd58fe9dc40e4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:27.598131 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e991174a45694a1fa40bd58fe9dc40e4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.599226 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_678702734654439b9b10658a0b6b3421"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:27.599764 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_678702734654439b9b10658a0b6b3421 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.599821 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_a4f93c6107724875b3de06f538971317"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:27.599970 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb0fc179a337469eae056c5bb3815e6b. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:27.600221 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a4f93c6107724875b3de06f538971317 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.605525 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d4bba0aa7ea403e8dfcb74cec2e1f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.605883 893 tablet_service.cc:1505] Processing CreateTablet for tablet 7d4bba0aa7ea403e8dfcb74cec2e1f9b (DEFAULT_TABLE table=default.Table_6bfc3963d4e346858acdfd9b9016245d [id=4ba40e6e45c7463f9730a2380b5ff856]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.606364 745 tablet_service.cc:1505] Processing CreateTablet for tablet e4b5a7553110424d901788c7a2ecdce6 (DEFAULT_TABLE table=default.Table_9ee6894ba6364a54be16bb427ac7197b [id=04d7acd50017488eaa11f783233b6d4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.607976 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet e6ba12f94f5f4d688f4222a4c2fb5436
I20251211 20:52:27.608093 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4647105ab57454d883ebfcde7bde891. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.608973 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 793f543d6b614ad7b3ab8e717db2ceaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.611469 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0129162969d74f5da3c08a1e6a27ab22" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:27.611398 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_ed11d3864a3b4e04a8c20b0c3a42d684"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:27.612304 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ed11d3864a3b4e04a8c20b0c3a42d684 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.613284 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b793241ee3b42b1a38e9f2f05e1173b. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:27.619475 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0f046ea2637e4a5583b7969e19a605d5 consensus state has no leader
I20251211 20:52:27.624264 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_06ad70486e504f4ca804a4470a0b65a1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:27.624971 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_06ad70486e504f4ca804a4470a0b65a1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.626616 892 tablet_service.cc:1505] Processing CreateTablet for tablet e4647105ab57454d883ebfcde7bde891 (DEFAULT_TABLE table=default.Table_ae7f102bdee3470990e863235f08d636 [id=d555d1fc6aef4a21b4d3f2804264d030]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.630275 744 tablet_service.cc:1505] Processing CreateTablet for tablet 676faec7f3d1488f99b220f8e968e9a0 (DEFAULT_TABLE table=default.Table_7aa604df1f25488f9f06c6b0e6daaf88 [id=a2bbfccfe3974c378521d5ef2333df86]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:27.631034 711 leader_election.cc:343] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:27.631351 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4b5a7553110424d901788c7a2ecdce6. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:27.632148 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5fd2a6cf80a04f20aef8ff9c84bc88db consensus state has no leader
W20251211 20:52:27.634449 711 consensus_peers.cc:597] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:27.634575 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 676faec7f3d1488f99b220f8e968e9a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.635922 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_7a09ac6e8f3e441f83b5738731d01544"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:27.636556 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a09ac6e8f3e441f83b5738731d01544 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.638417 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6ba12f94f5f4d688f4222a4c2fb5436. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.639258 887 tablet_service.cc:1505] Processing CreateTablet for tablet 676faec7f3d1488f99b220f8e968e9a0 (DEFAULT_TABLE table=default.Table_7aa604df1f25488f9f06c6b0e6daaf88 [id=a2bbfccfe3974c378521d5ef2333df86]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.640069 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb4004212441473483a209d23c0800ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.644891 1012 tablet_service.cc:1505] Processing CreateTablet for tablet 8b1ce6d2a12a4c7ea044b349412b4bf5 (DEFAULT_TABLE table=default.Table_8044d887ac4b408283fbb97b19ac6af6 [id=f964a6ca434049278c880c3089197039]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.646802 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fed300d31c414f4a9b27c66e78097688. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.645330 1166 raft_consensus.cc:359] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:27.647029 1166 raft_consensus.cc:385] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:27.648241 1166 raft_consensus.cc:740] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:27.648375 1020 tablet_service.cc:1505] Processing CreateTablet for tablet f2d6c9781b4d4c37bc08703347963d5d (DEFAULT_TABLE table=default.Table_54d8f0fdaacf4be8b1043f2bae13c56e [id=b50ab5b032ba4c98938324460512468f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.651374 1016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 645b08b33676420c96e880de3d460829. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.651608 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4b5a7553110424d901788c7a2ecdce6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.646760 1011 tablet_service.cc:1505] Processing CreateTablet for tablet c7147571483e41ad970e573a08c368ca (DEFAULT_TABLE table=default.Table_0c26504b5b4a4095b3a9e795754eb7d0 [id=560abe606b114f1fb022031dd6d9192f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.653053 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3b1aadf17ea4bf0a896d23cd543fca7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.654623 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 676faec7f3d1488f99b220f8e968e9a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.658473 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 4d98967714184fe6a3e38dd4ca7979dc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:27.659550 1166 consensus_queue.cc:260] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:27.657538 885 tablet_service.cc:1505] Processing CreateTablet for tablet 8b793241ee3b42b1a38e9f2f05e1173b (DEFAULT_TABLE table=default.Table_8adf3dfe2fb0445385c730198775d1e9 [id=279f39d718ba45318f52f95e8aa13652]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.662721 1166 ts_tablet_manager.cc:1434] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.129s user 0.002s sys 0.000s
I20251211 20:52:27.662262 1010 tablet_service.cc:1505] Processing CreateTablet for tablet f3855ffb51a74101b26fd28429e65bc2 (DEFAULT_TABLE table=default.Table_9349d035f81c4ae0bc0bc8625d0b116c [id=a923a0eb52d84adfbf2fbabfeef9ac15]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.666332 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d4bba0aa7ea403e8dfcb74cec2e1f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.666831 1160 raft_consensus.cc:359] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.667162 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3de5b34b96404cdc9c191fed7f055638. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.667794 747 tablet_service.cc:1505] Processing CreateTablet for tablet f2d6c9781b4d4c37bc08703347963d5d (DEFAULT_TABLE table=default.Table_54d8f0fdaacf4be8b1043f2bae13c56e [id=b50ab5b032ba4c98938324460512468f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.668182 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2d6c9781b4d4c37bc08703347963d5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.668573 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_290ed4f01870438ca11f93a98c4ce8e8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:27.669195 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_290ed4f01870438ca11f93a98c4ce8e8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.675982 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 88066d94d5d34e0fab20da9fb6ef178e with delete_type TABLET_DATA_DELETED (replaced by tablet 6a61cf8f5de14adaa4a1de5bddde9a2f) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:27.677489 878 tablet_service.cc:1505] Processing CreateTablet for tablet 22273287aa9245a2a66131fed2c8861c (DEFAULT_TABLE table=default.Table_40d59cf8fbb14f7e9ff94da21b038c0b [id=0f2c6ae946bd42d4a64fd82ab2a83867]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.678259 1012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b1ce6d2a12a4c7ea044b349412b4bf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.679399 1011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7147571483e41ad970e573a08c368ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.679518 1166 tablet_bootstrap.cc:492] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:27.680032 876 tablet_service.cc:1505] Processing CreateTablet for tablet a9bb22dce13f4c9da76ca446715cd99a (DEFAULT_TABLE table=default.Table_528b4823aba44a72a7af20911a8d2002 [id=940343c556c54c4fb2110f4f47ca91f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.682471 1160 raft_consensus.cc:385] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20251211 20:52:27.684123 591 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b072b759d1254859b0b8d401e3078792 consensus state has no leader
I20251211 20:52:27.682250 890 tablet_service.cc:1505] Processing CreateTablet for tablet f2d6c9781b4d4c37bc08703347963d5d (DEFAULT_TABLE table=default.Table_54d8f0fdaacf4be8b1043f2bae13c56e [id=b50ab5b032ba4c98938324460512468f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.688512 757 tablet_service.cc:1505] Processing CreateTablet for tablet 22273287aa9245a2a66131fed2c8861c (DEFAULT_TABLE table=default.Table_40d59cf8fbb14f7e9ff94da21b038c0b [id=0f2c6ae946bd42d4a64fd82ab2a83867]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.692365 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b793241ee3b42b1a38e9f2f05e1173b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.694159 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22273287aa9245a2a66131fed2c8861c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.692230 1160 raft_consensus.cc:740] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:27.696072 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet ef87e052644c41bbbc54b8dad68ce152 (table default.Table_9b96425fc6f34e15ab398770cb57bc98 [id=f8384ab62c6749b7b9c458e8e90d3c02]) successfully deleted
I20251211 20:52:27.696252 1160 consensus_queue.cc:260] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.696846 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4647105ab57454d883ebfcde7bde891. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.698264 1165 ts_tablet_manager.cc:1929] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:27.699241 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2d6c9781b4d4c37bc08703347963d5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.703836 1165 log.cc:1199] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/4ddf1bd1e18f4187bf050d6f7767b722
I20251211 20:52:27.710983 883 tablet_service.cc:1505] Processing CreateTablet for tablet 260bdeb73439437cbf234d20f7264ab3 (DEFAULT_TABLE table=default.Table_10e55a82b5d1413b8b68da9df264da06 [id=7c485abbfb12494c89c81b0b51965923]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.718886 1505 raft_consensus.cc:493] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:27.719085 1505 raft_consensus.cc:515] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.719843 1505 leader_election.cc:290] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:27.722640 889 tablet_service.cc:1505] Processing CreateTablet for tablet 645b08b33676420c96e880de3d460829 (DEFAULT_TABLE table=default.Table_e5def7ad31744669b691d5517da9c02b [id=6d42f260246e420d8818b3841926302b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.724987 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 22273287aa9245a2a66131fed2c8861c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:27.726127 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet bb112b6431af42a6a7b328dfdefbc2a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:27.727183 1165 ts_tablet_manager.cc:1950] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
W20251211 20:52:27.728492 847 leader_election.cc:343] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: e41f052151624df39a3c83744870c94e
I20251211 20:52:27.729321 1165 tablet_replica.cc:333] T 2433068ec179490dac3e91d046a42b70 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:27.729516 1165 raft_consensus.cc:2243] T 2433068ec179490dac3e91d046a42b70 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:27.730414 884 tablet_service.cc:1505] Processing CreateTablet for tablet a3b1aadf17ea4bf0a896d23cd543fca7 (DEFAULT_TABLE table=default.Table_1d00ad67ef20403b94de5d39425be56d [id=a07662f8dc3f4178be5dfcc1cdc85837]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.732949 888 tablet_service.cc:1505] Processing CreateTablet for tablet 4ce495233a5e4f3792faf44680df54ba (DEFAULT_TABLE table=default.Table_488913be2ade428593d6c0934e54a8f0 [id=f8adc6a8019d4b6b949eb34bde21031a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.733173 879 tablet_service.cc:1505] Processing CreateTablet for tablet fed300d31c414f4a9b27c66e78097688 (DEFAULT_TABLE table=default.Table_9f6d83209f134db4a00fbddf8f323113 [id=e7d1c65b881746eeb12169e26fe0db70]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.734062 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 4ddf1bd1e18f4187bf050d6f7767b722 (table default.Table_307027e1160747b98c2fb2ea31eb8745 [id=3ade2e0cb4924ec38113bf929b5f554b]) successfully deleted
I20251211 20:52:27.734956 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a206b37e43247b5babe3113eab2c9fb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:27.735519 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet e6ba12f94f5f4d688f4222a4c2fb5436 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:27.735993 595 catalog_manager.cc:5654] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } }
I20251211 20:52:27.736415 595 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet f92d26bcde3b410aa56bc23a61b1ac35 (table default.Table_7c42321f3e494dc8835cc2db68e653e1 [id=bee5829ed7d44761a9ae57d3134d38dc]). Expected version 1 got 0
W20251211 20:52:27.738739 847 leader_election.cc:343] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: e41f052151624df39a3c83744870c94e
I20251211 20:52:27.738866 847 leader_election.cc:304] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:27.739286 1505 raft_consensus.cc:2749] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:27.739745 1165 raft_consensus.cc:2272] T 2433068ec179490dac3e91d046a42b70 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:27.740375 1165 ts_tablet_manager.cc:1916] T 2433068ec179490dac3e91d046a42b70 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:27.742293 1160 ts_tablet_manager.cc:1434] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.158s user 0.001s sys 0.001s
I20251211 20:52:27.745213 1160 tablet_bootstrap.cc:492] T 4d98967714184fe6a3e38dd4ca7979dc P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:27.747515 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a2058b67ddf1413db338bfa2f64e7a77 consensus state has no leader
I20251211 20:52:27.748765 886 tablet_service.cc:1505] Processing CreateTablet for tablet 28ddb8a59a214576a25d613d42c2fef5 (DEFAULT_TABLE table=default.Table_e98180a63672423688b0de80af514e8b [id=658bb4c0b37e492c94080811d899c16b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.751048 1164 ts_tablet_manager.cc:1929] T ef87e052644c41bbbc54b8dad68ce152 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:27.751247 1164 log.cc:1199] T ef87e052644c41bbbc54b8dad68ce152 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/ef87e052644c41bbbc54b8dad68ce152
I20251211 20:52:27.756669 594 master_service.cc:954] ReplaceTablet: received request to replace tablet e41f052151624df39a3c83744870c94e from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:27.761735 894 tablet_service.cc:1505] Processing CreateTablet for tablet 5710fc2f24764f818f7e5c86c6fd985e (DEFAULT_TABLE table=default.Table_491fd0036d9e4f9fa410f3517c2c8286 [id=0e93a84ec95341889e108170806b2525]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.763222 882 tablet_service.cc:1505] Processing CreateTablet for tablet 4df68d1b6a91458e9f212ae81aa25438 (DEFAULT_TABLE table=default.Table_f574e04a625b4f00bd2e2e8693a7f496 [id=e266c3e0cd804232bcecc4db812ca13b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.766258 877 tablet_service.cc:1505] Processing CreateTablet for tablet bb4004212441473483a209d23c0800ce (DEFAULT_TABLE table=default.Table_55b2abeacec041418b6359c21e87b3d8 [id=75609081b0c7488eb388faac157db3e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.767428 887 tablet_service.cc:1505] Processing CreateTablet for tablet 7ec261837cf04bf88a48ff8add1b0d66 (DEFAULT_TABLE table=default.Table_e6e48a22fc3d43b69d3dd391b9d1c60b [id=1921595e79a7402dad44c6aee6417e78]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.767516 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet f3855ffb51a74101b26fd28429e65bc2
I20251211 20:52:27.730727 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 22273287aa9245a2a66131fed2c8861c (DEFAULT_TABLE table=default.Table_40d59cf8fbb14f7e9ff94da21b038c0b [id=0f2c6ae946bd42d4a64fd82ab2a83867]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.774214 881 tablet_service.cc:1505] Processing CreateTablet for tablet 3de5b34b96404cdc9c191fed7f055638 (DEFAULT_TABLE table=default.Table_7215dcc06d384c7d94b29ff2ea8ee8ee [id=593f8b9b01134fb09a1c7232509635bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.775688 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfb52935296443c58190d4d06e2e6e9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.775991 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22273287aa9245a2a66131fed2c8861c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.779721 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 260bdeb73439437cbf234d20f7264ab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.785866 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet f3855ffb51a74101b26fd28429e65bc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:27.787357 876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9bb22dce13f4c9da76ca446715cd99a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.801030 742 tablet_service.cc:1505] Processing CreateTablet for tablet 260bdeb73439437cbf234d20f7264ab3 (DEFAULT_TABLE table=default.Table_10e55a82b5d1413b8b68da9df264da06 [id=7c485abbfb12494c89c81b0b51965923]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.806806 743 tablet_service.cc:1505] Processing CreateTablet for tablet 645b08b33676420c96e880de3d460829 (DEFAULT_TABLE table=default.Table_e5def7ad31744669b691d5517da9c02b [id=6d42f260246e420d8818b3841926302b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.813066 746 tablet_service.cc:1505] Processing CreateTablet for tablet a9bb22dce13f4c9da76ca446715cd99a (DEFAULT_TABLE table=default.Table_528b4823aba44a72a7af20911a8d2002 [id=940343c556c54c4fb2110f4f47ca91f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.817562 1168 tablet_bootstrap.cc:492] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:27.819541 756 tablet_service.cc:1505] Processing CreateTablet for tablet 4ce495233a5e4f3792faf44680df54ba (DEFAULT_TABLE table=default.Table_488913be2ade428593d6c0934e54a8f0 [id=f8adc6a8019d4b6b949eb34bde21031a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.819908 1163 ts_tablet_manager.cc:1929] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:27.820071 1163 log.cc:1199] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/b8828385cec34d70b717ae7f3813b9c3
I20251211 20:52:27.820426 1163 ts_tablet_manager.cc:1950] T b8828385cec34d70b717ae7f3813b9c3 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:27.821048 1168 ts_tablet_manager.cc:1403] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.666s user 0.003s sys 0.003s
I20251211 20:52:27.821357 1163 tablet_replica.cc:333] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:27.821740 1014 tablet_service.cc:1505] Processing CreateTablet for tablet 2e7e158809b9471fbd6e6ee9fb9ac569 (DEFAULT_TABLE table=default.Table_6ea777b3e1c3474f926a3e1ab782c69b [id=fc9d41c089a443fab5b61579f4c788f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.822162 1166 tablet_bootstrap.cc:654] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:27.822419 1522 raft_consensus.cc:493] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:27.822593 1522 raft_consensus.cc:515] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:27.822891 748 tablet_service.cc:1505] Processing CreateTablet for tablet a3b1aadf17ea4bf0a896d23cd543fca7 (DEFAULT_TABLE table=default.Table_1d00ad67ef20403b94de5d39425be56d [id=a07662f8dc3f4178be5dfcc1cdc85837]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.823253 1522 leader_election.cc:290] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:27.823413 1163 raft_consensus.cc:2243] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:27.823824 1163 raft_consensus.cc:2272] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:27.824400 1163 ts_tablet_manager.cc:1916] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:27.824630 1550 raft_consensus.cc:493] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:27.824920 1550 raft_consensus.cc:515] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:27.826296 1550 leader_election.cc:290] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:27.826994 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0246f1df99684d0da43d97acbf1182a0" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:27.826689 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2d6c9781b4d4c37bc08703347963d5d. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:27.827833 982 leader_election.cc:343] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:27.828363 1014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e7e158809b9471fbd6e6ee9fb9ac569. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.829421 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6a206b37e43247b5babe3113eab2c9fb" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:27.829897 982 leader_election.cc:343] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:27.834384 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet b8828385cec34d70b717ae7f3813b9c3 (table default.Table_8352c9da1e324dd4847fd22078f22e53 [id=9dc54153dda4495684c2b07732795bfc]) successfully deleted
I20251211 20:52:27.835147 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3855ffb51a74101b26fd28429e65bc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.832763 738 tablet_service.cc:1505] Processing CreateTablet for tablet 5710fc2f24764f818f7e5c86c6fd985e (DEFAULT_TABLE table=default.Table_491fd0036d9e4f9fa410f3517c2c8286 [id=0e93a84ec95341889e108170806b2525]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.836426 740 tablet_service.cc:1505] Processing CreateTablet for tablet 4df68d1b6a91458e9f212ae81aa25438 (DEFAULT_TABLE table=default.Table_f574e04a625b4f00bd2e2e8693a7f496 [id=e266c3e0cd804232bcecc4db812ca13b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.840073 752 tablet_service.cc:1505] Processing CreateTablet for tablet 2207af105bc3471ca930d35a91d38e70 (DEFAULT_TABLE table=default.Table_1c43c2cd6c064780936260e292759cf1 [id=767f50bee5464316a18f73e64bab817a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.838227 751 tablet_service.cc:1505] Processing CreateTablet for tablet 28ddb8a59a214576a25d613d42c2fef5 (DEFAULT_TABLE table=default.Table_e98180a63672423688b0de80af514e8b [id=658bb4c0b37e492c94080811d899c16b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.844427 1168 raft_consensus.cc:359] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.844635 1168 raft_consensus.cc:385] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:27.845824 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22273287aa9245a2a66131fed2c8861c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.846390 1168 raft_consensus.cc:740] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:27.846513 1550 raft_consensus.cc:493] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:27.846669 1168 consensus_queue.cc:260] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.846683 1550 raft_consensus.cc:515] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:27.849035 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c4e048ca28824329b924845ccf5067cf" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:27.849608 982 leader_election.cc:343] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:27.850742 753 tablet_service.cc:1505] Processing CreateTablet for tablet 7ec261837cf04bf88a48ff8add1b0d66 (DEFAULT_TABLE table=default.Table_e6e48a22fc3d43b69d3dd391b9d1c60b [id=1921595e79a7402dad44c6aee6417e78]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.852849 1550 leader_election.cc:290] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:27.862287 755 tablet_service.cc:1505] Processing CreateTablet for tablet fed300d31c414f4a9b27c66e78097688 (DEFAULT_TABLE table=default.Table_9f6d83209f134db4a00fbddf8f323113 [id=e7d1c65b881746eeb12169e26fe0db70]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.862646 1168 ts_tablet_manager.cc:1434] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.026s user 0.001s sys 0.001s
I20251211 20:52:27.862880 1163 ts_tablet_manager.cc:1929] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:27.863013 1163 log.cc:1199] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/0246f1df99684d0da43d97acbf1182a0
I20251211 20:52:27.863301 1163 ts_tablet_manager.cc:1950] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:27.864696 1014 tablet_service.cc:1505] Processing CreateTablet for tablet 5262c2241e524395a73c0eb368917169 (DEFAULT_TABLE table=default.Table_f4fa28c6ade240c69a5b4a98b44eac5a [id=e5ea4528b8bc429186dfc7000428db96]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.864998 1168 tablet_bootstrap.cc:492] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:27.865087 1014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5262c2241e524395a73c0eb368917169. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.868842 1012 tablet_service.cc:1505] Processing CreateTablet for tablet 2f5916a4d9b24d7b8db3fcf028bbe3c9 (DEFAULT_TABLE table=default.Table_53c0c6163e2447959bc8eb3303c1685f [id=6e0536d7748d456a9de9df3ba7e914be]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.869243 1012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f5916a4d9b24d7b8db3fcf028bbe3c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.870254 744 tablet_service.cc:1505] Processing CreateTablet for tablet 8b1ce6d2a12a4c7ea044b349412b4bf5 (DEFAULT_TABLE table=default.Table_8044d887ac4b408283fbb97b19ac6af6 [id=f964a6ca434049278c880c3089197039]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.870920 754 tablet_service.cc:1505] Processing CreateTablet for tablet 68792ccef54944a09af9b5456e9ff1fe (DEFAULT_TABLE table=default.Table_00bd643f7bb84217970178aefb485a69 [id=1b90cdd71d064d27948a1a9a7f3f852b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.871295 739 tablet_service.cc:1505] Processing CreateTablet for tablet 3de5b34b96404cdc9c191fed7f055638 (DEFAULT_TABLE table=default.Table_7215dcc06d384c7d94b29ff2ea8ee8ee [id=593f8b9b01134fb09a1c7232509635bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.871662 1420 raft_consensus.cc:493] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:27.871793 1420 raft_consensus.cc:515] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.872632 1420 leader_election.cc:290] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:27.872993 741 tablet_service.cc:1505] Processing CreateTablet for tablet bb4004212441473483a209d23c0800ce (DEFAULT_TABLE table=default.Table_55b2abeacec041418b6359c21e87b3d8 [id=75609081b0c7488eb388faac157db3e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.874708 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 6a61cf8f5de14adaa4a1de5bddde9a2f (DEFAULT_TABLE table=default.Table_b06f9587f78645bb9712c3b080f3c0d4 [id=ea12bdaa894f446da22635f95d97ac2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.875741 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a61cf8f5de14adaa4a1de5bddde9a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.886049 757 tablet_service.cc:1505] Processing CreateTablet for tablet c7147571483e41ad970e573a08c368ca (DEFAULT_TABLE table=default.Table_0c26504b5b4a4095b3a9e795754eb7d0 [id=560abe606b114f1fb022031dd6d9192f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.889957 743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 645b08b33676420c96e880de3d460829. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.895792 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fed300d31c414f4a9b27c66e78097688. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.898201 1011 tablet_service.cc:1505] Processing CreateTablet for tablet 2e33606930774fe4aafc6e3819618af8 (DEFAULT_TABLE table=default.Table_290ed4f01870438ca11f93a98c4ce8e8 [id=9e23a3e5095f4a358fe7859424c27d59]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.898324 1420 raft_consensus.cc:493] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:27.898583 1011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e33606930774fe4aafc6e3819618af8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.898485 1420 raft_consensus.cc:515] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:27.899327 1420 leader_election.cc:290] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:27.899725 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ce495233a5e4f3792faf44680df54ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.903769 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9bb22dce13f4c9da76ca446715cd99a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.903983 1553 tablet.cc:1722] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:27.907760 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3b1aadf17ea4bf0a896d23cd543fca7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.908725 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 861d2d70a24946699dccd65cab05195e (DEFAULT_TABLE table=default.Table_06ad70486e504f4ca804a4470a0b65a1 [id=28be836b71ed4107af7adc3fcb948b5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.909125 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 861d2d70a24946699dccd65cab05195e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.912832 1013 tablet_service.cc:1505] Processing CreateTablet for tablet 3acfef6f082641a199b80eefcfc772ac (DEFAULT_TABLE table=default.Table_ed11d3864a3b4e04a8c20b0c3a42d684 [id=1c8eceb8a080464192ef7ab198728cd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.912973 738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5710fc2f24764f818f7e5c86c6fd985e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.913256 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3acfef6f082641a199b80eefcfc772ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.915653 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 0246f1df99684d0da43d97acbf1182a0 (table default.Table_03fdaef4fcc144f485f238db0e76fab4 [id=4264180c89854054b14e98d5a49984fa]) successfully deleted
I20251211 20:52:27.915716 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68792ccef54944a09af9b5456e9ff1fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.916797 1163 tablet_replica.cc:333] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:27.917873 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 50f9f29c064c4f938a0f20f302ab5e8a (DEFAULT_TABLE table=default.Table_a4f93c6107724875b3de06f538971317 [id=da8cf2b99b42409091484a9513623d6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.918255 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50f9f29c064c4f938a0f20f302ab5e8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.920204 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ec261837cf04bf88a48ff8add1b0d66. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.920694 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2207af105bc3471ca930d35a91d38e70. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:27.921119 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 87863daf77304a399318cdf56da9f9bb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 87863daf77304a399318cdf56da9f9bb already in progress: creating tablet
W20251211 20:52:27.921348 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet f7f52d2547534ef99255c5d3f59410c8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f7f52d2547534ef99255c5d3f59410c8 already in progress: creating tablet
I20251211 20:52:27.922472 1163 raft_consensus.cc:2243] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:27.923223 1163 raft_consensus.cc:2272] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:27.924134 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4df68d1b6a91458e9f212ae81aa25438. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.924239 1163 ts_tablet_manager.cc:1916] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:27.925915 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb4004212441473483a209d23c0800ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.927881 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3de5b34b96404cdc9c191fed7f055638. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.930472 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28ddb8a59a214576a25d613d42c2fef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.933491 1012 tablet_service.cc:1505] Processing CreateTablet for tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb (DEFAULT_TABLE table=default.Table_678702734654439b9b10658a0b6b3421 [id=b5b24783442f46f7a87c9758c1ed1397]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.933784 745 tablet_service.cc:1505] Processing CreateTablet for tablet f3855ffb51a74101b26fd28429e65bc2 (DEFAULT_TABLE table=default.Table_9349d035f81c4ae0bc0bc8625d0b116c [id=a923a0eb52d84adfbf2fbabfeef9ac15]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.934399 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b1ce6d2a12a4c7ea044b349412b4bf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.935899 590 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_85e0192a945545faad36b2be67d76e2a [id=a24d8c58b9d84cdea2b278ae9c6e0ba5] alter complete (version 3)
I20251211 20:52:27.936509 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 22273287aa9245a2a66131fed2c8861c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:27.936709 710 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s user 0.020s sys 0.046s
I20251211 20:52:27.936762 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 4d98967714184fe6a3e38dd4ca7979dc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:27.936839 710 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s user 0.020s sys 0.046s
I20251211 20:52:27.936952 1012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.936987 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 88066d94d5d34e0fab20da9fb6ef178e with delete_type TABLET_DATA_DELETED (replaced by tablet 6a61cf8f5de14adaa4a1de5bddde9a2f) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:27.937472 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d19975cc513e414b80f478a6c5eb09de" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:27.937707 1049 raft_consensus.cc:2468] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:27.937857 747 tablet_service.cc:1552] Processing DeleteTablet for tablet bb112b6431af42a6a7b328dfdefbc2a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:27.938112 747 tablet_service.cc:1552] Processing DeleteTablet for tablet e6ba12f94f5f4d688f4222a4c2fb5436 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:27.938517 747 tablet_service.cc:1505] Processing CreateTablet for tablet 2e7e158809b9471fbd6e6ee9fb9ac569 (DEFAULT_TABLE table=default.Table_6ea777b3e1c3474f926a3e1ab782c69b [id=fc9d41c089a443fab5b61579f4c788f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.939167 710 leader_election.cc:304] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:27.941900 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 41c30bded6504bbc9f0af8f34dc81e8d (DEFAULT_TABLE table=default.Table_e991174a45694a1fa40bd58fe9dc40e4 [id=89e5024dde2e47b69383704e49911d89]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.943003 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b06f9587f78645bb9712c3b080f3c0d4" } modify_external_catalogs: true
I20251211 20:52:27.943185 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b06f9587f78645bb9712c3b080f3c0d4" } modify_external_catalogs: true
I20251211 20:52:27.944471 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_d8c241666f7840b8bf0cb7672482aa55" } modify_external_catalogs: true
I20251211 20:52:27.944660 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_d8c241666f7840b8bf0cb7672482aa55" } modify_external_catalogs: true
I20251211 20:52:27.949522 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41c30bded6504bbc9f0af8f34dc81e8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.950867 1168 tablet_bootstrap.cc:654] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:27.942203 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2260850f32b541aab1128818f144b5a7" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:27.952744 1046 raft_consensus.cc:2468] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:27.957969 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 2260850f32b541aab1128818f144b5a7
I20251211 20:52:27.958690 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 6a61cf8f5de14adaa4a1de5bddde9a2f
I20251211 20:52:27.959218 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 2260850f32b541aab1128818f144b5a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:27.959394 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 940dfd8781de444a9caa78dd363c7451 (DEFAULT_TABLE table=default.Table_7a09ac6e8f3e441f83b5738731d01544 [id=926a3069e1e946beb5cb0d5ddcfc3660]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.959828 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 940dfd8781de444a9caa78dd363c7451. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.960738 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a61cf8f5de14adaa4a1de5bddde9a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:27.963376 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ce495233a5e4f3792faf44680df54ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.964061 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28ddb8a59a214576a25d613d42c2fef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:27.967093 1163 ts_tablet_manager.cc:1929] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:27.973574 875 tablet_service.cc:1505] Processing CreateTablet for tablet 68792ccef54944a09af9b5456e9ff1fe (DEFAULT_TABLE table=default.Table_00bd643f7bb84217970178aefb485a69 [id=1b90cdd71d064d27948a1a9a7f3f852b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.974359 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0246f1df99684d0da43d97acbf1182a0" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:27.977123 1164 ts_tablet_manager.cc:1950] T ef87e052644c41bbbc54b8dad68ce152 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:27.976625 891 tablet_service.cc:1505] Processing CreateTablet for tablet 2207af105bc3471ca930d35a91d38e70 (DEFAULT_TABLE table=default.Table_1c43c2cd6c064780936260e292759cf1 [id=767f50bee5464316a18f73e64bab817a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.979369 1160 tablet_bootstrap.cc:654] T 4d98967714184fe6a3e38dd4ca7979dc P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:27.980233 1164 tablet_replica.cc:333] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:27.979903 892 tablet_service.cc:1505] Processing CreateTablet for tablet 8b1ce6d2a12a4c7ea044b349412b4bf5 (DEFAULT_TABLE table=default.Table_8044d887ac4b408283fbb97b19ac6af6 [id=f964a6ca434049278c880c3089197039]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:27.981003 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2260850f32b541aab1128818f144b5a7" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:27.982476 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6a206b37e43247b5babe3113eab2c9fb" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:27.983073 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c5b164dc0ce1461fae98d950f7c7a726" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:27.972095 1166 tablet_bootstrap.cc:492] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:27.983409 1166 ts_tablet_manager.cc:1403] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.307s user 0.011s sys 0.005s
I20251211 20:52:27.983880 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3b1aadf17ea4bf0a896d23cd543fca7. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:27.984108 983 leader_election.cc:343] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:27.984232 1166 raft_consensus.cc:359] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:27.984249 983 leader_election.cc:304] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:27.984432 1166 raft_consensus.cc:385] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:27.984503 1166 raft_consensus.cc:740] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
W20251211 20:52:27.984572 983 raft_consensus.cc:2686] T 0246f1df99684d0da43d97acbf1182a0 P e715a811a82446ce8d5bb4aafd5b81c7: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251211 20:52:27.984767 1166 consensus_queue.cc:260] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:27.985689 1166 ts_tablet_manager.cc:1434] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20251211 20:52:27.985651 983 leader_election.cc:343] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:27.985975 983 leader_election.cc:304] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:27.986321 1522 raft_consensus.cc:2749] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:27.986584 1166 tablet_bootstrap.cc:492] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:27.991629 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9ee6894ba6364a54be16bb427ac7197b" } modify_external_catalogs: true
I20251211 20:52:27.991844 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9ee6894ba6364a54be16bb427ac7197b" } modify_external_catalogs: true
I20251211 20:52:27.993185 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_7aa604df1f25488f9f06c6b0e6daaf88" } new_table_name: "default.Table_e523eadf39e648adba47da28461e85e3" modify_external_catalogs: true
I20251211 20:52:27.961588 1542 raft_consensus.cc:2804] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:27.994468 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_893c9504152b412eb7177a511fcedc04"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:27.983541 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7147571483e41ad970e573a08c368ca. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:27.994983 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_893c9504152b412eb7177a511fcedc04 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:27.954612 749 tablet_service.cc:1552] Processing DeleteTablet for tablet f3855ffb51a74101b26fd28429e65bc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:27.996168 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b0e00b07650849958a1a4486eeb3c660" } modify_external_catalogs: true
I20251211 20:52:27.996342 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b0e00b07650849958a1a4486eeb3c660" } modify_external_catalogs: true
I20251211 20:52:27.997591 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_7c42321f3e494dc8835cc2db68e653e1" } new_table_name: "default.Table_c2f4d6a8642a44b1b5566b7631b598e6" modify_external_catalogs: true
I20251211 20:52:27.998795 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_9c7198623d9b41158b2ac00adc92f327"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:27.999400 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c7198623d9b41158b2ac00adc92f327 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.001267 1166 tablet_bootstrap.cc:654] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:28.002645 1012 tablet_service.cc:1552] Processing DeleteTablet for tablet c38c1a4c0e574eb0957de127d41c26ad with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:28.002610 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_5e65e4dae9454b3e993a46b2255d955d" } new_table_name: "default.Table_021d4fb1667a475d87424f23aa0d5e28" modify_external_catalogs: true
I20251211 20:52:28.002959 1163 log.cc:1199] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/4d98967714184fe6a3e38dd4ca7979dc
I20251211 20:52:28.003625 1163 ts_tablet_manager.cc:1950] T 4d98967714184fe6a3e38dd4ca7979dc P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:28.003815 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_96a88507426c4a45b670fa5002d8df08"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:28.004297 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_96a88507426c4a45b670fa5002d8df08 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.005529 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 4d98967714184fe6a3e38dd4ca7979dc (table default.Table_ebec874d437748c487f8bf9d0ccaa7fa [id=aea22a96d5b840399f2140486953c1ed]) successfully deleted
W20251211 20:52:28.012338 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 22273287aa9245a2a66131fed2c8861c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 22273287aa9245a2a66131fed2c8861c already in progress: creating tablet
W20251211 20:52:28.013414 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet bb112b6431af42a6a7b328dfdefbc2a2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb112b6431af42a6a7b328dfdefbc2a2 already in progress: creating tablet
W20251211 20:52:28.013688 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 676faec7f3d1488f99b220f8e968e9a0 consensus state has no leader
I20251211 20:52:28.014161 1163 tablet_replica.cc:333] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:28.014815 1163 raft_consensus.cc:2243] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:28.018469 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet e4b5a7553110424d901788c7a2ecdce6
I20251211 20:52:28.023697 1163 raft_consensus.cc:2272] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:28.024191 1420 consensus_queue.cc:1048] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20251211 20:52:28.021476 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet e4b5a7553110424d901788c7a2ecdce6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:27.968255 710 leader_election.cc:343] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 4ddf1bd1e18f4187bf050d6f7767b722
I20251211 20:52:28.025470 750 tablet_service.cc:1505] Processing CreateTablet for tablet 5262c2241e524395a73c0eb368917169 (DEFAULT_TABLE table=default.Table_f4fa28c6ade240c69a5b4a98b44eac5a [id=e5ea4528b8bc429186dfc7000428db96]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.026038 1163 ts_tablet_manager.cc:1916] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:28.026932 711 leader_election.cc:343] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:28.027215 1542 raft_consensus.cc:493] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:28.028386 1166 tablet_bootstrap.cc:492] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:28.028591 1166 ts_tablet_manager.cc:1403] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.042s user 0.000s sys 0.005s
I20251211 20:52:28.029091 710 leader_election.cc:304] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:28.029712 749 tablet_service.cc:1505] Processing CreateTablet for tablet 861d2d70a24946699dccd65cab05195e (DEFAULT_TABLE table=default.Table_06ad70486e504f4ca804a4470a0b65a1 [id=28be836b71ed4107af7adc3fcb948b5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.029600 1166 raft_consensus.cc:359] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:28.033291 1166 raft_consensus.cc:385] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:28.033452 1166 raft_consensus.cc:740] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:28.033635 1164 raft_consensus.cc:2243] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:28.033835 1164 raft_consensus.cc:2272] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:28.033771 1166 consensus_queue.cc:260] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:28.034303 1166 ts_tablet_manager.cc:1434] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20251211 20:52:28.033591 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c4e048ca28824329b924845ccf5067cf" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:28.035037 983 leader_election.cc:343] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:28.035188 983 leader_election.cc:304] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:28.035782 1550 raft_consensus.cc:2749] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:28.036295 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 645b08b33676420c96e880de3d460829. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.036342 746 tablet_service.cc:1505] Processing CreateTablet for tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb (DEFAULT_TABLE table=default.Table_678702734654439b9b10658a0b6b3421 [id=b5b24783442f46f7a87c9758c1ed1397]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.038336 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet ef87e052644c41bbbc54b8dad68ce152 (table default.Table_9b96425fc6f34e15ab398770cb57bc98 [id=f8384ab62c6749b7b9c458e8e90d3c02]) successfully deleted
I20251211 20:52:28.040328 1166 tablet_bootstrap.cc:492] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:28.042732 1166 tablet_bootstrap.cc:654] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:28.043138 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2207af105bc3471ca930d35a91d38e70. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.046808 1163 ts_tablet_manager.cc:1929] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20251211 20:52:28.047876 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet fb030a62796248869d46b682656eaeb0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fb030a62796248869d46b682656eaeb0 already in progress: creating tablet
I20251211 20:52:28.048069 1163 log.cc:1199] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/6a206b37e43247b5babe3113eab2c9fb
I20251211 20:52:28.048416 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b1ce6d2a12a4c7ea044b349412b4bf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.048471 1163 ts_tablet_manager.cc:1950] T 6a206b37e43247b5babe3113eab2c9fb P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:28.048882 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_9ab1d237fd114e33b86d1df0630ee7b9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:28.050024 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9ab1d237fd114e33b86d1df0630ee7b9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.050766 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 6a206b37e43247b5babe3113eab2c9fb (table default.Table_028df988934346debc0f8c62b4740309 [id=e1949419f3924474bf195ac17a794497]) successfully deleted
I20251211 20:52:28.050803 595 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e5979cadea8746f4afd0ae1b2673b272" } new_table_name: "default.Table_70c84990b84847979bb2408ce8aaea00" modify_external_catalogs: true
I20251211 20:52:28.051937 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_aa53337337ff420e8030af1bb87dd3f7" } new_table_name: "default.Table_f6a10556ff104db5bd5ecc9fc380e228" modify_external_catalogs: true
W20251211 20:52:28.052366 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet e6ba12f94f5f4d688f4222a4c2fb5436 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e6ba12f94f5f4d688f4222a4c2fb5436 already in progress: creating tablet
I20251211 20:52:28.053251 875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68792ccef54944a09af9b5456e9ff1fe. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:28.053803 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet f3855ffb51a74101b26fd28429e65bc2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f3855ffb51a74101b26fd28429e65bc2 already in progress: creating tablet
I20251211 20:52:28.053994 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b7f9abd9a3b04fd4a6ef3b477330ea9e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:28.054390 1163 tablet_replica.cc:333] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:28.054665 1163 raft_consensus.cc:2243] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
W20251211 20:52:28.054991 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b7f9abd9a3b04fd4a6ef3b477330ea9e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.055433 1163 raft_consensus.cc:2272] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:28.059114 589 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 2630ad57845a4c31b20c8a104dd27335 (table default.Table_063289c4a3514468a2c1fa058dde23f3 [id=61b2de16fee74487823292913de1b685]). Expected version 2 got 0
I20251211 20:52:28.059926 1163 ts_tablet_manager.cc:1916] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:28.060365 741 tablet_service.cc:1552] Processing DeleteTablet for tablet c5b164dc0ce1461fae98d950f7c7a726 with delete_type TABLET_DATA_DELETED (replaced by tablet 49a33084c0fc460eb9592224b6dfc279) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:28.060904 741 tablet_service.cc:1505] Processing CreateTablet for tablet 50f9f29c064c4f938a0f20f302ab5e8a (DEFAULT_TABLE table=default.Table_a4f93c6107724875b3de06f538971317 [id=da8cf2b99b42409091484a9513623d6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.061270 755 tablet_service.cc:1505] Processing CreateTablet for tablet 2f5916a4d9b24d7b8db3fcf028bbe3c9 (DEFAULT_TABLE table=default.Table_53c0c6163e2447959bc8eb3303c1685f [id=6e0536d7748d456a9de9df3ba7e914be]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.061625 748 tablet_service.cc:1505] Processing CreateTablet for tablet 2e33606930774fe4aafc6e3819618af8 (DEFAULT_TABLE table=default.Table_290ed4f01870438ca11f93a98c4ce8e8 [id=9e23a3e5095f4a358fe7859424c27d59]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.061733 1164 ts_tablet_manager.cc:1916] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:28.061928 1542 raft_consensus.cc:3060] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:28.062260 711 leader_election.cc:343] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:28.062312 710 raft_consensus.cc:2686] T 4ddf1bd1e18f4187bf050d6f7767b722 P 6f8d784fd1cd492bbe3809403ec34919: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251211 20:52:28.062371 1166 tablet_bootstrap.cc:492] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:28.062507 1166 ts_tablet_manager.cc:1403] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.002s
I20251211 20:52:28.062623 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4df68d1b6a91458e9f212ae81aa25438. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.062938 589 master_service.cc:954] ReplaceTablet: received request to replace tablet c38c1a4c0e574eb0957de127d41c26ad from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:28.063267 1166 raft_consensus.cc:359] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:28.063547 1166 raft_consensus.cc:385] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:28.063853 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5710fc2f24764f818f7e5c86c6fd985e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.064199 1166 raft_consensus.cc:740] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:28.064837 1166 consensus_queue.cc:260] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:28.065390 1166 ts_tablet_manager.cc:1434] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20251211 20:52:28.065896 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ec261837cf04bf88a48ff8add1b0d66. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.070576 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 25454e67437541d9ac2bfab29bad55f9
I20251211 20:52:28.073766 1166 tablet_bootstrap.cc:492] T 9adaee28edd74df68b44b25d505fdfcb P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:28.075641 1166 tablet_bootstrap.cc:654] T 9adaee28edd74df68b44b25d505fdfcb P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:28.059795 754 tablet_service.cc:1505] Processing CreateTablet for tablet 6a61cf8f5de14adaa4a1de5bddde9a2f (DEFAULT_TABLE table=default.Table_b06f9587f78645bb9712c3b080f3c0d4 [id=ea12bdaa894f446da22635f95d97ac2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.079003 743 tablet_service.cc:1505] Processing CreateTablet for tablet 3acfef6f082641a199b80eefcfc772ac (DEFAULT_TABLE table=default.Table_ed11d3864a3b4e04a8c20b0c3a42d684 [id=1c8eceb8a080464192ef7ab198728cd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.083138 589 master_service.cc:954] ReplaceTablet: received request to replace tablet 2433068ec179490dac3e91d046a42b70 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:28.084765 1550 raft_consensus.cc:493] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:28.084969 1550 raft_consensus.cc:515] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:28.085599 1550 leader_election.cc:290] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:28.087209 1163 ts_tablet_manager.cc:1929] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:28.087409 1163 log.cc:1199] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/2260850f32b541aab1128818f144b5a7
I20251211 20:52:28.087882 1163 ts_tablet_manager.cc:1950] T 2260850f32b541aab1128818f144b5a7 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:28.090387 740 tablet_service.cc:1505] Processing CreateTablet for tablet 940dfd8781de444a9caa78dd363c7451 (DEFAULT_TABLE table=default.Table_7a09ac6e8f3e441f83b5738731d01544 [id=926a3069e1e946beb5cb0d5ddcfc3660]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.091497 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 2260850f32b541aab1128818f144b5a7 (table default.Table_d8c241666f7840b8bf0cb7672482aa55 [id=81bc0640c53f4710895d487f7c6be071]) successfully deleted
W20251211 20:52:28.092417 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 126944us
W20251211 20:52:28.092792 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 6a61cf8f5de14adaa4a1de5bddde9a2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a61cf8f5de14adaa4a1de5bddde9a2f already in progress: creating tablet
I20251211 20:52:28.093442 1163 tablet_replica.cc:333] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:28.093878 1163 raft_consensus.cc:2243] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:28.094244 756 tablet_service.cc:1505] Processing CreateTablet for tablet 41c30bded6504bbc9f0af8f34dc81e8d (DEFAULT_TABLE table=default.Table_e991174a45694a1fa40bd58fe9dc40e4 [id=89e5024dde2e47b69383704e49911d89]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.094344 1163 raft_consensus.cc:2272] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:28.095423 1163 ts_tablet_manager.cc:1916] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:28.096185 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3855ffb51a74101b26fd28429e65bc2. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:28.096359 710 leader_election.cc:343] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 0129162969d74f5da3c08a1e6a27ab22
I20251211 20:52:28.096517 710 leader_election.cc:304] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:28.096881 1420 raft_consensus.cc:2749] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:28.095405 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c38c1a4c0e574eb0957de127d41c26ad" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:28.098003 710 consensus_peers.cc:597] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:28.097769 982 leader_election.cc:343] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:28.100025 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3de5b34b96404cdc9c191fed7f055638. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.100266 1512 consensus_queue.cc:1048] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
W20251211 20:52:28.100867 710 leader_election.cc:343] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: c5b164dc0ce1461fae98d950f7c7a726
I20251211 20:52:28.101900 710 leader_election.cc:304] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:28.102495 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.102845 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a61cf8f5de14adaa4a1de5bddde9a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.103078 710 leader_election.cc:304] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:28.104029 890 tablet_service.cc:1552] Processing DeleteTablet for tablet 22273287aa9245a2a66131fed2c8861c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:28.104061 885 tablet_service.cc:1505] Processing CreateTablet for tablet f3855ffb51a74101b26fd28429e65bc2 (DEFAULT_TABLE table=default.Table_9349d035f81c4ae0bc0bc8625d0b116c [id=a923a0eb52d84adfbf2fbabfeef9ac15]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.106527 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c38c1a4c0e574eb0957de127d41c26ad" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:28.107218 983 leader_election.cc:343] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:28.107368 983 leader_election.cc:304] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:28.107538 983 raft_consensus.cc:2686] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251211 20:52:28.108608 890 tablet_service.cc:1505] Processing CreateTablet for tablet 2e7e158809b9471fbd6e6ee9fb9ac569 (DEFAULT_TABLE table=default.Table_6ea777b3e1c3474f926a3e1ab782c69b [id=fc9d41c089a443fab5b61579f4c788f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.108706 1420 raft_consensus.cc:2749] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:28.111002 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 4d98967714184fe6a3e38dd4ca7979dc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:28.114172 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb4004212441473483a209d23c0800ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.115059 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 88066d94d5d34e0fab20da9fb6ef178e with delete_type TABLET_DATA_DELETED (replaced by tablet 6a61cf8f5de14adaa4a1de5bddde9a2f) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:28.115295 883 tablet_service.cc:1552] Processing DeleteTablet for tablet bb112b6431af42a6a7b328dfdefbc2a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:28.115509 883 tablet_service.cc:1552] Processing DeleteTablet for tablet e6ba12f94f5f4d688f4222a4c2fb5436 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:28.115783 743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3acfef6f082641a199b80eefcfc772ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.115963 883 tablet_service.cc:1552] Processing DeleteTablet for tablet f3855ffb51a74101b26fd28429e65bc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:28.116009 1512 raft_consensus.cc:2804] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:28.116127 1512 raft_consensus.cc:493] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:28.116236 1512 raft_consensus.cc:3060] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:28.116451 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e7e158809b9471fbd6e6ee9fb9ac569. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.116957 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f5916a4d9b24d7b8db3fcf028bbe3c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.117298 1547 raft_consensus.cc:493] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:28.117573 1168 tablet_bootstrap.cc:492] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:28.118515 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 260bdeb73439437cbf234d20f7264ab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.121035 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 861d2d70a24946699dccd65cab05195e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.122189 876 tablet_service.cc:1505] Processing CreateTablet for tablet c7147571483e41ad970e573a08c368ca (DEFAULT_TABLE table=default.Table_0c26504b5b4a4095b3a9e795754eb7d0 [id=560abe606b114f1fb022031dd6d9192f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.126974 416 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-task5TWsQc/build/asan/bin/kudu pid 492
I20251211 20:52:28.128350 556 generic_service.cc:196] Checking for leaks (request via RPC)
I20251211 20:52:28.128815 1168 ts_tablet_manager.cc:1403] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.253s user 0.006s sys 0.007s
I20251211 20:52:28.131184 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fed300d31c414f4a9b27c66e78097688. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.137877 883 tablet_service.cc:1505] Processing CreateTablet for tablet 5262c2241e524395a73c0eb368917169 (DEFAULT_TABLE table=default.Table_f4fa28c6ade240c69a5b4a98b44eac5a [id=e5ea4528b8bc429186dfc7000428db96]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.138439 1555 tablet.cc:1722] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:28.144549 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4000fbd0bb614d0c902f539be2923421" } modify_external_catalogs: true
I20251211 20:52:28.144735 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4000fbd0bb614d0c902f539be2923421" } modify_external_catalogs: true
I20251211 20:52:28.146675 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3855ffb51a74101b26fd28429e65bc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.146935 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e7e158809b9471fbd6e6ee9fb9ac569. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.150458 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_92ff976cb6724b6199d251d20d56b101"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:28.151014 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_92ff976cb6724b6199d251d20d56b101 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.154094 893 tablet_service.cc:1505] Processing CreateTablet for tablet 2f5916a4d9b24d7b8db3fcf028bbe3c9 (DEFAULT_TABLE table=default.Table_53c0c6163e2447959bc8eb3303c1685f [id=6e0536d7748d456a9de9df3ba7e914be]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.158341 1550 raft_consensus.cc:493] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:28.158532 1550 raft_consensus.cc:515] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:28.159302 1550 leader_election.cc:290] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:28.160041 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c910246b3d5d4e7e8a71d9f18e94d24c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:28.160686 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5262c2241e524395a73c0eb368917169. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:28.160913 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 773cf53745a64f62817113b33e3182e7 consensus state has no leader
W20251211 20:52:28.161496 595 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0b267c7feaac43f6aacb7229759c7ec3 consensus state has no leader
I20251211 20:52:28.144284 1547 raft_consensus.cc:515] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:28.167994 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet 25454e67437541d9ac2bfab29bad55f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:28.168159 1163 ts_tablet_manager.cc:1929] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:28.168645 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet c910246b3d5d4e7e8a71d9f18e94d24c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:28.171895 1163 log.cc:1199] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/c38c1a4c0e574eb0957de127d41c26ad
I20251211 20:52:28.172331 1163 ts_tablet_manager.cc:1950] T c38c1a4c0e574eb0957de127d41c26ad P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:28.173454 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet c38c1a4c0e574eb0957de127d41c26ad (table default.Table_88975c1ad9604f50b7f69fbbdacabce3 [id=52f34b88d367454286943a27138b20f2]) successfully deleted
I20251211 20:52:28.173630 1163 tablet_replica.cc:333] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:28.173643 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f5916a4d9b24d7b8db3fcf028bbe3c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.173831 1163 raft_consensus.cc:2243] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
W20251211 20:52:28.173985 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet e4b5a7553110424d901788c7a2ecdce6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e4b5a7553110424d901788c7a2ecdce6 already in progress: creating tablet
I20251211 20:52:28.174000 1163 raft_consensus.cc:2272] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:28.174208 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 25454e67437541d9ac2bfab29bad55f9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 25454e67437541d9ac2bfab29bad55f9 already in progress: creating tablet
W20251211 20:52:28.174202 983 leader_election.cc:343] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:28.174865 1163 ts_tablet_manager.cc:1916] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:28.181288 1163 ts_tablet_manager.cc:1929] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:28.181694 1163 log.cc:1199] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/c910246b3d5d4e7e8a71d9f18e94d24c
I20251211 20:52:28.182140 1163 ts_tablet_manager.cc:1950] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:28.182859 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet c910246b3d5d4e7e8a71d9f18e94d24c (table default.Table_c2300bb8bfe44c689969a2b233d69353 [id=996745d9420c4589a52eca6bad10d6a9]) successfully deleted
I20251211 20:52:28.183676 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_60e84f16bbb3462682f8bdaffd7d1542"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:28.184406 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_60e84f16bbb3462682f8bdaffd7d1542 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.185267 1547 leader_election.cc:290] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:28.186416 591 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 33e1d8a576cd41b1a34778b1458df396 (table default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]). Expected version 2 got 0
I20251211 20:52:28.187753 1166 tablet_bootstrap.cc:492] T 9adaee28edd74df68b44b25d505fdfcb P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:28.187963 1166 ts_tablet_manager.cc:1403] T 9adaee28edd74df68b44b25d505fdfcb P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.114s user 0.015s sys 0.033s
I20251211 20:52:28.188931 1166 raft_consensus.cc:359] T 9adaee28edd74df68b44b25d505fdfcb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:28.189213 1166 raft_consensus.cc:385] T 9adaee28edd74df68b44b25d505fdfcb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:28.189347 1166 raft_consensus.cc:740] T 9adaee28edd74df68b44b25d505fdfcb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:28.189620 1166 consensus_queue.cc:260] T 9adaee28edd74df68b44b25d505fdfcb P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:28.190233 1166 ts_tablet_manager.cc:1434] T 9adaee28edd74df68b44b25d505fdfcb P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:28.190573 1166 tablet_bootstrap.cc:492] T 8b89ae724b7f41c98741a41d75592da7 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:28.192886 1166 tablet_bootstrap.cc:654] T 8b89ae724b7f41c98741a41d75592da7 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:28.196022 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d19975cc513e414b80f478a6c5eb09de" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:28.196282 1045 raft_consensus.cc:2468] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 0.
I20251211 20:52:28.198761 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e33606930774fe4aafc6e3819618af8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.199023 1166 tablet_bootstrap.cc:492] T 8b89ae724b7f41c98741a41d75592da7 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:28.199259 1166 ts_tablet_manager.cc:1403] T 8b89ae724b7f41c98741a41d75592da7 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20251211 20:52:28.200131 1166 raft_consensus.cc:359] T 8b89ae724b7f41c98741a41d75592da7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:28.200383 1166 raft_consensus.cc:385] T 8b89ae724b7f41c98741a41d75592da7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:28.200505 1166 raft_consensus.cc:740] T 8b89ae724b7f41c98741a41d75592da7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:28.200791 1166 consensus_queue.cc:260] T 8b89ae724b7f41c98741a41d75592da7 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:28.201319 1166 ts_tablet_manager.cc:1434] T 8b89ae724b7f41c98741a41d75592da7 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:28.201702 1166 tablet_bootstrap.cc:492] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:28.203406 1166 tablet_bootstrap.cc:654] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:28.206612 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 2260850f32b541aab1128818f144b5a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:28.207015 847 leader_election.cc:304] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41; no voters:
W20251211 20:52:28.210847 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 33addf3d158941098f240cfcdec45d28 consensus state has no leader
I20251211 20:52:28.208789 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_274e759f68cb487da862431c34237c5c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:28.217275 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c910246b3d5d4e7e8a71d9f18e94d24c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:28.218052 982 leader_election.cc:343] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:28.218226 982 leader_election.cc:304] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:28.218472 982 raft_consensus.cc:2686] T c910246b3d5d4e7e8a71d9f18e94d24c P e715a811a82446ce8d5bb4aafd5b81c7: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251211 20:52:28.226528 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e98180a63672423688b0de80af514e8b" } modify_external_catalogs: true
I20251211 20:52:28.226780 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e98180a63672423688b0de80af514e8b" } modify_external_catalogs: true
W20251211 20:52:28.227150 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_274e759f68cb487da862431c34237c5c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.227639 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0c26504b5b4a4095b3a9e795754eb7d0" } modify_external_catalogs: true
I20251211 20:52:28.228858 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5262c2241e524395a73c0eb368917169. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.230598 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0c26504b5b4a4095b3a9e795754eb7d0" } modify_external_catalogs: true
I20251211 20:52:28.231482 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_987db791387d4fa79e285fae8bddce52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:28.232689 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_987db791387d4fa79e285fae8bddce52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.234279 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b426c9f38ff44d76af3480168e4003cf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:28.237043 1168 raft_consensus.cc:359] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:28.239391 876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7147571483e41ad970e573a08c368ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.239871 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a61cf8f5de14adaa4a1de5bddde9a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:28.240223 878 tablet_service.cc:1505] Processing CreateTablet for tablet 6a61cf8f5de14adaa4a1de5bddde9a2f (DEFAULT_TABLE table=default.Table_b06f9587f78645bb9712c3b080f3c0d4 [id=ea12bdaa894f446da22635f95d97ac2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.241885 1542 raft_consensus.cc:515] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:28.244627 1165 ts_tablet_manager.cc:1929] T 2433068ec179490dac3e91d046a42b70 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:28.245420 752 tablet_service.cc:1552] Processing DeleteTablet for tablet e4b5a7553110424d901788c7a2ecdce6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:28.245883 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d19975cc513e414b80f478a6c5eb09de" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:28.247474 891 tablet_service.cc:1505] Processing CreateTablet for tablet 2e33606930774fe4aafc6e3819618af8 (DEFAULT_TABLE table=default.Table_290ed4f01870438ca11f93a98c4ce8e8 [id=9e23a3e5095f4a358fe7859424c27d59]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.248310 1165 log.cc:1199] T 2433068ec179490dac3e91d046a42b70 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/2433068ec179490dac3e91d046a42b70
I20251211 20:52:28.248705 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50f9f29c064c4f938a0f20f302ab5e8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.251086 884 tablet_service.cc:1505] Processing CreateTablet for tablet 861d2d70a24946699dccd65cab05195e (DEFAULT_TABLE table=default.Table_06ad70486e504f4ca804a4470a0b65a1 [id=28be836b71ed4107af7adc3fcb948b5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.252725 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41c30bded6504bbc9f0af8f34dc81e8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.253902 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 940dfd8781de444a9caa78dd363c7451. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.254828 775 raft_consensus.cc:2393] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb0f1933d9a74fccaa4618047fc05c41 in current term 1: Already voted for candidate 6f8d784fd1cd492bbe3809403ec34919 in this term.
I20251211 20:52:28.254935 1164 ts_tablet_manager.cc:1929] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:28.257202 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 25454e67437541d9ac2bfab29bad55f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:28.257529 1542 leader_election.cc:290] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:28.260774 1168 raft_consensus.cc:385] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:28.260888 1168 raft_consensus.cc:740] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:28.265244 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d19975cc513e414b80f478a6c5eb09de" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:28.268862 1505 raft_consensus.cc:2804] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:28.269985 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a61cf8f5de14adaa4a1de5bddde9a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.271639 1168 consensus_queue.cc:260] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:28.272607 1165 ts_tablet_manager.cc:1950] T 2433068ec179490dac3e91d046a42b70 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:28.274097 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 2433068ec179490dac3e91d046a42b70 (table default.Table_26b3b7e5db73457da38332d4e3d943f6 [id=18edd3deb100406cb5f421ee06216e6b]) successfully deleted
W20251211 20:52:28.274930 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet f7f52d2547534ef99255c5d3f59410c8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f7f52d2547534ef99255c5d3f59410c8 already in progress: creating tablet
W20251211 20:52:28.275650 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 87863daf77304a399318cdf56da9f9bb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 87863daf77304a399318cdf56da9f9bb already in progress: creating tablet
I20251211 20:52:28.275990 1165 tablet_replica.cc:333] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:28.276247 1165 raft_consensus.cc:2243] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:28.276515 1165 raft_consensus.cc:2272] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:28.277215 1165 ts_tablet_manager.cc:1916] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:28.280326 886 tablet_service.cc:1505] Processing CreateTablet for tablet 3acfef6f082641a199b80eefcfc772ac (DEFAULT_TABLE table=default.Table_ed11d3864a3b4e04a8c20b0c3a42d684 [id=1c8eceb8a080464192ef7ab198728cd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.281459 1164 log.cc:1199] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/97a519a8ca7c4f5380a230a70ee84052
I20251211 20:52:28.283015 912 raft_consensus.cc:3060] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:28.235172 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b426c9f38ff44d76af3480168e4003cf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.284688 888 tablet_service.cc:1505] Processing CreateTablet for tablet 50f9f29c064c4f938a0f20f302ab5e8a (DEFAULT_TABLE table=default.Table_a4f93c6107724875b3de06f538971317 [id=da8cf2b99b42409091484a9513623d6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.296989 887 tablet_service.cc:1505] Processing CreateTablet for tablet 940dfd8781de444a9caa78dd363c7451 (DEFAULT_TABLE table=default.Table_7a09ac6e8f3e441f83b5738731d01544 [id=926a3069e1e946beb5cb0d5ddcfc3660]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.295039 883 tablet_service.cc:1505] Processing CreateTablet for tablet 41c30bded6504bbc9f0af8f34dc81e8d (DEFAULT_TABLE table=default.Table_e991174a45694a1fa40bd58fe9dc40e4 [id=89e5024dde2e47b69383704e49911d89]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.298399 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 861d2d70a24946699dccd65cab05195e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.293387 882 tablet_service.cc:1505] Processing CreateTablet for tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb (DEFAULT_TABLE table=default.Table_678702734654439b9b10658a0b6b3421 [id=b5b24783442f46f7a87c9758c1ed1397]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.310731 1164 ts_tablet_manager.cc:1950] T 97a519a8ca7c4f5380a230a70ee84052 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:28.311347 590 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 33e1d8a576cd41b1a34778b1458df396 (table default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]). Expected version 2 got 0
I20251211 20:52:28.319533 1020 tablet_service.cc:1505] Processing CreateTablet for tablet b30d2054316b49fa8096345527a45d87 (DEFAULT_TABLE table=default.Table_893c9504152b412eb7177a511fcedc04 [id=4b02349b61c244cba852abf532b63999]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.319994 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b30d2054316b49fa8096345527a45d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.323737 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet c910246b3d5d4e7e8a71d9f18e94d24c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:28.326632 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3acfef6f082641a199b80eefcfc772ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.326627 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0a0fab7f167b4e34b48892ee705fd1e2" } new_table_name: "default.Table_654982f288d34c08bcf3ef3426638abd" modify_external_catalogs: true
I20251211 20:52:28.326761 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 287) took 1079 ms. Trace:
I20251211 20:52:28.328042 1164 rpcz_store.cc:276] 1211 20:52:27.232125 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:27.232273 (+ 148us) service_pool.cc:225] Handling call
1211 20:52:27.978184 (+745911us) ts_tablet_manager.cc:1151] Deleting tablet 97a519a8ca7c4f5380a230a70ee84052
1211 20:52:27.978255 (+ 71us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:28.141273 (+163018us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:28.254824 (+113551us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:28.311138 (+ 56314us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":681182}
W20251211 20:52:28.328491 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet c910246b3d5d4e7e8a71d9f18e94d24c because the tablet was not found. No further retry: Not found: Tablet not found: c910246b3d5d4e7e8a71d9f18e94d24c
I20251211 20:52:28.328724 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 97a519a8ca7c4f5380a230a70ee84052 (table default.Table_47249158f89f4c16af084f0e1147ef9f [id=383392d835da4c50848068d83261cecc]) successfully deleted
I20251211 20:52:28.329622 913 raft_consensus.cc:1275] T f92d26bcde3b410aa56bc23a61b1ac35 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Refusing update from remote peer 6f8d784fd1cd492bbe3809403ec34919: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251211 20:52:28.330466 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 940dfd8781de444a9caa78dd363c7451. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.333009 1555 tablet.cc:1717] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41: Skipping requested alter to schema version 3, tablet already version 3
I20251211 20:52:28.333309 1553 tablet.cc:1717] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7: Skipping requested alter to schema version 3, tablet already version 3
I20251211 20:52:28.334187 1012 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b89ae724b7f41c98741a41d75592da7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:28.333058 1164 tablet_replica.cc:333] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:28.334520 1163 tablet_replica.cc:333] T 8b89ae724b7f41c98741a41d75592da7 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:28.334671 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50f9f29c064c4f938a0f20f302ab5e8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.334720 1163 raft_consensus.cc:2243] T 8b89ae724b7f41c98741a41d75592da7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:28.334887 1163 raft_consensus.cc:2272] T 8b89ae724b7f41c98741a41d75592da7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:28.335358 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet 9adaee28edd74df68b44b25d505fdfcb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:28.336138 1163 ts_tablet_manager.cc:1916] T 8b89ae724b7f41c98741a41d75592da7 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:28.339803 1164 raft_consensus.cc:2243] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:28.345369 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_488913be2ade428593d6c0934e54a8f0" } modify_external_catalogs: true
I20251211 20:52:28.345599 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_488913be2ade428593d6c0934e54a8f0" } modify_external_catalogs: true
I20251211 20:52:28.346375 880 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43888 (request call id 312) took 1018 ms. Trace:
I20251211 20:52:28.346503 880 rpcz_store.cc:276] 1211 20:52:27.326181 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:27.475730 (+149549us) service_pool.cc:225] Handling call
1211 20:52:27.597728 (+121998us) ts_tablet_manager.cc:669] Acquired tablet manager lock
1211 20:52:27.597837 (+ 109us) ts_tablet_manager.cc:682] Creating new metadata...
1211 20:52:28.196419 (+598582us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:28.344096 (+147677us) tablet_replica.cc:206] Creating consensus instance
1211 20:52:28.344227 (+ 131us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20251211 20:52:28.348253 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a61cf8f5de14adaa4a1de5bddde9a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:28.348579 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 2260850f32b541aab1128818f144b5a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:28.348747 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 25454e67437541d9ac2bfab29bad55f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:28.349445 584 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 33addf3d158941098f240cfcdec45d28 (table default.Table_f6a10556ff104db5bd5ecc9fc380e228 [id=f75813add54e46189ff1bbed82d83ba7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 33addf3d158941098f240cfcdec45d28 consensus state has no leader
I20251211 20:52:28.349933 1164 raft_consensus.cc:2272] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:28.351001 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41c30bded6504bbc9f0af8f34dc81e8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.351768 880 tablet_service.cc:1505] Processing CreateTablet for tablet b30d2054316b49fa8096345527a45d87 (DEFAULT_TABLE table=default.Table_893c9504152b412eb7177a511fcedc04 [id=4b02349b61c244cba852abf532b63999]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.353229 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_4813c06395dd41c78eceebb97804cdad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:28.353480 894 tablet_service.cc:1552] Processing DeleteTablet for tablet e4b5a7553110424d901788c7a2ecdce6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:28.353855 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4813c06395dd41c78eceebb97804cdad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.355114 1164 ts_tablet_manager.cc:1916] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:28.356009 1163 ts_tablet_manager.cc:1929] T 8b89ae724b7f41c98741a41d75592da7 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:28.358688 1163 log.cc:1199] T 8b89ae724b7f41c98741a41d75592da7 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/8b89ae724b7f41c98741a41d75592da7
I20251211 20:52:28.358692 738 tablet_service.cc:1505] Processing CreateTablet for tablet b30d2054316b49fa8096345527a45d87 (DEFAULT_TABLE table=default.Table_893c9504152b412eb7177a511fcedc04 [id=4b02349b61c244cba852abf532b63999]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.359133 738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b30d2054316b49fa8096345527a45d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.359334 1163 ts_tablet_manager.cc:1950] T 8b89ae724b7f41c98741a41d75592da7 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:28.360304 1542 consensus_queue.cc:1048] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20251211 20:52:28.360411 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 8b89ae724b7f41c98741a41d75592da7 (table default.Table_c607094f0e4d4af48d2e54d4d728ae13 [id=9dae0e60f978448a840aa497da35966a]) successfully deleted
I20251211 20:52:28.362901 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.363232 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b30d2054316b49fa8096345527a45d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.363479 1163 tablet_replica.cc:333] T 9adaee28edd74df68b44b25d505fdfcb P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:28.363695 1163 raft_consensus.cc:2243] T 9adaee28edd74df68b44b25d505fdfcb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:28.363888 1163 raft_consensus.cc:2272] T 9adaee28edd74df68b44b25d505fdfcb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:28.365195 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fc8d485cc234fd490c487a8bc881bf4 with delete_type TABLET_DATA_DELETED (replaced by tablet 28ef13be3b84410d9c786c43502a3478) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:28.368219 1163 ts_tablet_manager.cc:1916] T 9adaee28edd74df68b44b25d505fdfcb P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:28.382305 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e6e48a22fc3d43b69d3dd391b9d1c60b" } modify_external_catalogs: true
I20251211 20:52:28.382553 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e6e48a22fc3d43b69d3dd391b9d1c60b" } modify_external_catalogs: true
I20251211 20:52:28.383639 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e33606930774fe4aafc6e3819618af8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.384727 595 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 33e1d8a576cd41b1a34778b1458df396 (table default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]). Expected version 2 got 0
I20251211 20:52:28.385208 1550 raft_consensus.cc:493] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:28.385358 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 33e1d8a576cd41b1a34778b1458df396 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:28.385389 1550 raft_consensus.cc:515] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:28.386099 1550 leader_election.cc:290] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:28.386685 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a0c412985adf4382a7ea28c77654fd6c" } new_table_name: "default.Table_f6f977edac0f45bea7e94de5bfa51ee4" modify_external_catalogs: true
I20251211 20:52:28.389196 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2630ad57845a4c31b20c8a104dd27335" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:28.389851 982 leader_election.cc:343] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:28.390249 1160 tablet_bootstrap.cc:492] T 4d98967714184fe6a3e38dd4ca7979dc P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:28.390352 1163 ts_tablet_manager.cc:1929] T 9adaee28edd74df68b44b25d505fdfcb P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:28.390384 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2630ad57845a4c31b20c8a104dd27335" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:28.390473 1163 log.cc:1199] T 9adaee28edd74df68b44b25d505fdfcb P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/9adaee28edd74df68b44b25d505fdfcb
I20251211 20:52:28.390652 1160 ts_tablet_manager.cc:1403] T 4d98967714184fe6a3e38dd4ca7979dc P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.646s user 0.013s sys 0.009s
I20251211 20:52:28.390862 1163 ts_tablet_manager.cc:1950] T 9adaee28edd74df68b44b25d505fdfcb P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
W20251211 20:52:28.390938 983 leader_election.cc:343] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:28.391090 983 leader_election.cc:304] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:28.391523 1550 raft_consensus.cc:2749] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:28.393447 912 raft_consensus.cc:2468] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:28.396234 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_45aae1bb922143a2b0fb90dd4b385f0c" } new_table_name: "default.Table_01240e3a77b44d43a567846655dbc681" modify_external_catalogs: true
I20251211 20:52:28.396499 1505 raft_consensus.cc:493] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:28.399046 1505 raft_consensus.cc:3060] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Advancing to term 2
I20251211 20:52:28.402709 711 leader_election.cc:304] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters:
I20251211 20:52:28.404201 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 9adaee28edd74df68b44b25d505fdfcb (table default.Table_4c974eff9ab442d499f0405abc8a4acd [id=35caba098ab545f7859c004af8568ef7]) successfully deleted
I20251211 20:52:28.407682 1512 raft_consensus.cc:515] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:28.408463 1512 leader_election.cc:290] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:28.414975 1574 raft_consensus.cc:2804] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:28.415246 1574 raft_consensus.cc:697] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:28.415700 1574 consensus_queue.cc:237] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:28.424031 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2260850f32b541aab1128818f144b5a7" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:28.425571 711 leader_election.cc:343] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:28.429170 1505 raft_consensus.cc:515] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:28.433832 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d19975cc513e414b80f478a6c5eb09de" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:28.434154 1046 raft_consensus.cc:3060] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 2
I20251211 20:52:28.434880 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d19975cc513e414b80f478a6c5eb09de" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:28.435458 1505 leader_election.cc:290] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 election: Requested vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:28.438398 1046 raft_consensus.cc:2468] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 2.
I20251211 20:52:28.447690 847 leader_election.cc:304] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919
I20251211 20:52:28.450495 1505 raft_consensus.cc:2804] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [term 2 FOLLOWER]: Leader election won for term 2
I20251211 20:52:28.451179 1505 raft_consensus.cc:697] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [term 2 LEADER]: Becoming Leader. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Running, Role: LEADER
I20251211 20:52:28.451507 1505 consensus_queue.cc:237] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:28.452174 1160 raft_consensus.cc:359] T 4d98967714184fe6a3e38dd4ca7979dc P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:28.453966 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d19975cc513e414b80f478a6c5eb09de" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:28.454232 1045 raft_consensus.cc:2368] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6f8d784fd1cd492bbe3809403ec34919 for earlier term 1. Current term is 2.
I20251211 20:52:28.455396 1045 raft_consensus.cc:1275] T f92d26bcde3b410aa56bc23a61b1ac35 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Refusing update from remote peer 6f8d784fd1cd492bbe3809403ec34919: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251211 20:52:28.459475 1160 raft_consensus.cc:385] T 4d98967714184fe6a3e38dd4ca7979dc P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:28.460843 1160 raft_consensus.cc:740] T 4d98967714184fe6a3e38dd4ca7979dc P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:28.461301 1160 consensus_queue.cc:260] T 4d98967714184fe6a3e38dd4ca7979dc P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:28.467092 1164 ts_tablet_manager.cc:1929] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:28.467305 1164 log.cc:1199] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/e41f052151624df39a3c83744870c94e
I20251211 20:52:28.470924 1160 ts_tablet_manager.cc:1434] T 4d98967714184fe6a3e38dd4ca7979dc P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.080s user 0.003s sys 0.000s
I20251211 20:52:28.482642 1160 tablet_bootstrap.cc:492] T 8dff13e22899452c8d4f2731b8dac7d5 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:28.489816 1164 ts_tablet_manager.cc:1950] T e41f052151624df39a3c83744870c94e P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:28.490303 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 288) took 1195 ms. Trace:
I20251211 20:52:28.490523 1164 rpcz_store.cc:276] 1211 20:52:27.294963 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:27.295080 (+ 117us) service_pool.cc:225] Handling call
1211 20:52:28.328486 (+1033406us) ts_tablet_manager.cc:1151] Deleting tablet e41f052151624df39a3c83744870c94e
1211 20:52:28.328498 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:28.463835 (+135337us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:28.466947 (+ 3112us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:28.490264 (+ 23317us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":989629}
I20251211 20:52:28.491086 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet e41f052151624df39a3c83744870c94e (table default.Table_bcccedcd97444712a0f74e76db8cd74a [id=3de8f4cc61014e96aa64e7d4dd768096]) successfully deleted
I20251211 20:52:28.495709 1160 tablet_bootstrap.cc:654] T 8dff13e22899452c8d4f2731b8dac7d5 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:28.496647 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet c7147571483e41ad970e573a08c368ca
I20251211 20:52:28.496852 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 28ddb8a59a214576a25d613d42c2fef5
I20251211 20:52:28.497480 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_300858c884e34abe9becdb6113469673" } new_table_name: "default.Table_6d94053661f142b79292e19752df02df" modify_external_catalogs: true
I20251211 20:52:28.497707 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 4ce495233a5e4f3792faf44680df54ba
I20251211 20:52:28.498400 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_7f5df390c11943918af61066a7c67c66"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:28.498852 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b529b38d754f46ba8cb60b19e9705256 consensus state has no leader
W20251211 20:52:28.498929 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7f5df390c11943918af61066a7c67c66 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.499207 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4a0345a7a9eb4dfc89d48517fbe15ad1" } new_table_name: "default.Table_2915f90c5dbb4d868402e18273217109" modify_external_catalogs: true
I20251211 20:52:28.500413 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 4201da4241aa48dfb670a713e4453ca5
I20251211 20:52:28.497501 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_424b336a2ac84471bac647eab716d19c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:28.501904 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_424b336a2ac84471bac647eab716d19c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.501310 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 4201da4241aa48dfb670a713e4453ca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:28.501473 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ce495233a5e4f3792faf44680df54ba with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:28.507326 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 28ddb8a59a214576a25d613d42c2fef5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:28.508122 891 tablet_service.cc:1552] Processing DeleteTablet for tablet c7147571483e41ad970e573a08c368ca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:28.509263 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 306) took 1206 ms. Trace:
I20251211 20:52:28.510821 1164 rpcz_store.cc:276] 1211 20:52:27.303030 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:27.411058 (+108028us) service_pool.cc:225] Handling call
1211 20:52:28.491167 (+1080109us) ts_tablet_manager.cc:1151] Deleting tablet f7f52d2547534ef99255c5d3f59410c8
1211 20:52:28.491178 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:28.509217 (+ 18039us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1054459}
I20251211 20:52:28.511508 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 307) took 1208 ms. Trace:
I20251211 20:52:28.512681 1164 rpcz_store.cc:276] 1211 20:52:27.303104 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:27.434544 (+131440us) service_pool.cc:225] Handling call
1211 20:52:28.511344 (+1076800us) ts_tablet_manager.cc:1151] Deleting tablet 87863daf77304a399318cdf56da9f9bb
1211 20:52:28.511356 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:28.511476 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1050192}
I20251211 20:52:28.514393 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 7ec261837cf04bf88a48ff8add1b0d66
W20251211 20:52:28.515424 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2197822e0f11484f9679c4858626ac2c consensus state has no leader
I20251211 20:52:28.515770 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ec261837cf04bf88a48ff8add1b0d66 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:28.515754 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet f7f52d2547534ef99255c5d3f59410c8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f7f52d2547534ef99255c5d3f59410c8 already in progress: creating tablet
W20251211 20:52:28.516047 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 87863daf77304a399318cdf56da9f9bb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 87863daf77304a399318cdf56da9f9bb already in progress: creating tablet
I20251211 20:52:28.516268 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 33e1d8a576cd41b1a34778b1458df396
W20251211 20:52:28.516283 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 22273287aa9245a2a66131fed2c8861c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 22273287aa9245a2a66131fed2c8861c already in progress: creating tablet
I20251211 20:52:28.516937 588 master_service.cc:954] ReplaceTablet: received request to replace tablet c85624ad640041c29bb87416e1992a9e from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:28.517474 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 33e1d8a576cd41b1a34778b1458df396 with delete_type TABLET_DATA_DELETED (replaced by tablet 6b7dd8232d974a43b99ec9b28075dc8d) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:28.517428 1555 tablet.cc:1722] T f92d26bcde3b410aa56bc23a61b1ac35 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:28.517503 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_a17987402c844d2cbef2310a1375db03"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:28.518045 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a17987402c844d2cbef2310a1375db03 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.523221 1164 tablet_replica.cc:333] T 4d98967714184fe6a3e38dd4ca7979dc P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:28.524001 1164 raft_consensus.cc:2243] T 4d98967714184fe6a3e38dd4ca7979dc P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:28.524914 1164 raft_consensus.cc:2272] T 4d98967714184fe6a3e38dd4ca7979dc P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:28.526990 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c4e048ca28824329b924845ccf5067cf consensus state has no leader
I20251211 20:52:28.527364 1164 ts_tablet_manager.cc:1916] T 4d98967714184fe6a3e38dd4ca7979dc P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:28.527441 591 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3c578d81e8dd41aaa0f707c4e3fed225 consensus state has no leader
I20251211 20:52:28.528472 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_2c7bb884f8b5488e9a80e5d20e75cc95"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:28.529052 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2c7bb884f8b5488e9a80e5d20e75cc95 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.531186 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_687a33fe231c48689b4e08ef5d2d087b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:28.531329 1160 tablet_bootstrap.cc:492] T 8dff13e22899452c8d4f2731b8dac7d5 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:28.531519 1160 ts_tablet_manager.cc:1403] T 8dff13e22899452c8d4f2731b8dac7d5 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.054s user 0.001s sys 0.004s
I20251211 20:52:28.531709 591 master_service.cc:954] ReplaceTablet: received request to replace tablet 3cd7a465249246c28f07b8fb2e42675f from {username='slave'} at 127.0.0.1:44772
W20251211 20:52:28.532012 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_687a33fe231c48689b4e08ef5d2d087b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.532341 1160 raft_consensus.cc:359] T 8dff13e22899452c8d4f2731b8dac7d5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:28.532627 1160 raft_consensus.cc:385] T 8dff13e22899452c8d4f2731b8dac7d5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:28.532752 1160 raft_consensus.cc:740] T 8dff13e22899452c8d4f2731b8dac7d5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:28.516732 595 catalog_manager.cc:6724] ReplaceTablet: tablet 33e1d8a576cd41b1a34778b1458df396 deleted and replaced by tablet 6b7dd8232d974a43b99ec9b28075dc8d
I20251211 20:52:28.533110 1160 consensus_queue.cc:260] T 8dff13e22899452c8d4f2731b8dac7d5 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:28.533440 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c8e17645a26947ab8c01f9b093fa422c consensus state has no leader
I20251211 20:52:28.533828 1160 ts_tablet_manager.cc:1434] T 8dff13e22899452c8d4f2731b8dac7d5 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251211 20:52:28.535315 1160 tablet_bootstrap.cc:492] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:28.538671 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet c7147571483e41ad970e573a08c368ca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:28.538926 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet 28ddb8a59a214576a25d613d42c2fef5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:28.539757 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ce495233a5e4f3792faf44680df54ba with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:28.540685 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet c7147571483e41ad970e573a08c368ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c7147571483e41ad970e573a08c368ca already in progress: creating tablet
W20251211 20:52:28.541043 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 28ddb8a59a214576a25d613d42c2fef5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 28ddb8a59a214576a25d613d42c2fef5 already in progress: creating tablet
I20251211 20:52:28.541289 1012 tablet_service.cc:1552] Processing DeleteTablet for tablet 33e1d8a576cd41b1a34778b1458df396 with delete_type TABLET_DATA_DELETED (replaced by tablet 6b7dd8232d974a43b99ec9b28075dc8d) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:28.541374 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 4ce495233a5e4f3792faf44680df54ba because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ce495233a5e4f3792faf44680df54ba already in progress: creating tablet
I20251211 20:52:28.541622 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ec261837cf04bf88a48ff8add1b0d66 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:28.541847 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet 4201da4241aa48dfb670a713e4453ca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:28.542078 1163 tablet_replica.cc:333] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:28.542277 1163 raft_consensus.cc:2243] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:28.542435 1163 pending_rounds.cc:70] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7: Trying to abort 1 pending ops.
I20251211 20:52:28.542425 589 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet f92d26bcde3b410aa56bc23a61b1ac35 (table default.Table_c2f4d6a8642a44b1b5566b7631b598e6 [id=bee5829ed7d44761a9ae57d3134d38dc]). Expected version 2 got 0
I20251211 20:52:28.542621 1163 pending_rounds.cc:77] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432076087742464 op_type: NO_OP noop_request { }
I20251211 20:52:28.542760 1163 raft_consensus.cc:2889] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:28.542881 1163 raft_consensus.cc:2272] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:28.544194 1163 ts_tablet_manager.cc:1916] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:28.551988 1166 tablet_bootstrap.cc:492] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:28.552242 1166 ts_tablet_manager.cc:1403] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.351s user 0.091s sys 0.105s
I20251211 20:52:28.552510 1545 mvcc.cc:204] Tried to move back new op lower bound from 7231432082453807104 to 7231432078296379392. Current Snapshot: MvccSnapshot[applied={T|T < 7231432082453807104}]
I20251211 20:52:28.554091 596 master_service.cc:954] ReplaceTablet: received request to replace tablet 773cf53745a64f62817113b33e3182e7 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:28.554476 1166 raft_consensus.cc:359] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:28.554759 1166 raft_consensus.cc:385] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:28.554890 1166 raft_consensus.cc:740] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:28.555183 1166 consensus_queue.cc:260] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:28.556583 710 leader_election.cc:400] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Vote denied by peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) with higher term. Message: Invalid argument: T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6f8d784fd1cd492bbe3809403ec34919 for earlier term 1. Current term is 2.
I20251211 20:52:28.556720 1555 tablet.cc:1722] T 33e1d8a576cd41b1a34778b1458df396 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:28.562657 1165 ts_tablet_manager.cc:1929] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:28.565789 1168 ts_tablet_manager.cc:1434] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.427s user 0.001s sys 0.013s
W20251211 20:52:28.566417 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 87949.4us
W20251211 20:52:28.572103 710 leader_election.cc:343] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 2260850f32b541aab1128818f144b5a7
I20251211 20:52:28.571830 1165 log.cc:1199] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/0129162969d74f5da3c08a1e6a27ab22
I20251211 20:52:28.572292 710 leader_election.cc:304] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:28.574184 1165 ts_tablet_manager.cc:1950] T 0129162969d74f5da3c08a1e6a27ab22 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:28.575618 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 311) took 1088 ms. Trace:
I20251211 20:52:28.585112 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_c4fc3456c46345c3955a0c7f79556c05"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:28.585683 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c4fc3456c46345c3955a0c7f79556c05 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.586865 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_29c42f496ae04f19b277dabacd834892"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:28.587332 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_29c42f496ae04f19b277dabacd834892 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.589037 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_07f1881614084afd936866d115aecbe7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:28.589468 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_07f1881614084afd936866d115aecbe7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.593504 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet c85624ad640041c29bb87416e1992a9e
I20251211 20:52:28.593956 588 catalog_manager.cc:6724] ReplaceTablet: tablet c85624ad640041c29bb87416e1992a9e deleted and replaced by tablet fa79b67b9eee45e2b41c4a90967606cc
I20251211 20:52:28.594277 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 3cd7a465249246c28f07b8fb2e42675f
I20251211 20:52:28.594671 591 catalog_manager.cc:6724] ReplaceTablet: tablet 3cd7a465249246c28f07b8fb2e42675f deleted and replaced by tablet 0763a475527546bb83cee8fb823893ac
I20251211 20:52:28.595772 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 773cf53745a64f62817113b33e3182e7
I20251211 20:52:28.596134 596 catalog_manager.cc:6724] ReplaceTablet: tablet 773cf53745a64f62817113b33e3182e7 deleted and replaced by tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce
I20251211 20:52:28.597992 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_c0ed4563b1d44eafa92065d3ec8adb93"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:28.598527 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c0ed4563b1d44eafa92065d3ec8adb93 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.574748 1420 raft_consensus.cc:2749] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20251211 20:52:28.578526 1168 tablet_bootstrap.cc:492] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:28.572988 1575 consensus_queue.cc:1048] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20251211 20:52:28.579850 1565 consensus_queue.cc:1048] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251211 20:52:28.580819 757 tablet_service.cc:1552] Processing DeleteTablet for tablet c7147571483e41ad970e573a08c368ca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:28.581475 755 tablet_service.cc:1552] Processing DeleteTablet for tablet 28ddb8a59a214576a25d613d42c2fef5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:28.583638 749 tablet_service.cc:1505] Processing CreateTablet for tablet c91ff29847d54e55b19321652a857e9f (DEFAULT_TABLE table=default.Table_9c7198623d9b41158b2ac00adc92f327 [id=ac770d79782241ad808c6533bd4e4350]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.584012 739 tablet_service.cc:1505] Processing CreateTablet for tablet 215f8dc74a71482daa2ca80e15010f51 (DEFAULT_TABLE table=default.Table_b7f9abd9a3b04fd4a6ef3b477330ea9e [id=9b089b452ac940fe929dc03bddca83e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.584213 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 33e1d8a576cd41b1a34778b1458df396 with delete_type TABLET_DATA_DELETED (replaced by tablet 6b7dd8232d974a43b99ec9b28075dc8d) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:28.597425 740 tablet_service.cc:1505] Processing CreateTablet for tablet 3e6f62c2deff4c11b137fc0cba5ee234 (DEFAULT_TABLE table=default.Table_96a88507426c4a45b670fa5002d8df08 [id=fdfaa0b24d2f4736aee695ed228212cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.597661 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ce495233a5e4f3792faf44680df54ba with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:28.598398 743 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ec261837cf04bf88a48ff8add1b0d66 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:28.600032 738 tablet_service.cc:1552] Processing DeleteTablet for tablet 4201da4241aa48dfb670a713e4453ca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:28.601647 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 3cd7a465249246c28f07b8fb2e42675f with delete_type TABLET_DATA_DELETED (replaced by tablet 0763a475527546bb83cee8fb823893ac) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:28.601812 750 tablet_service.cc:1552] Processing DeleteTablet for tablet c85624ad640041c29bb87416e1992a9e with delete_type TABLET_DATA_DELETED (replaced by tablet fa79b67b9eee45e2b41c4a90967606cc) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:28.601938 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 773cf53745a64f62817113b33e3182e7 with delete_type TABLET_DATA_DELETED (replaced by tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:28.602680 710 consensus_peers.cc:597] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:28.603858 1582 tablet.cc:1722] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:28.607081 1168 tablet_bootstrap.cc:654] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:28.608467 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c91ff29847d54e55b19321652a857e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.616001 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e6f62c2deff4c11b137fc0cba5ee234. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.619740 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 215f8dc74a71482daa2ca80e15010f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.621510 1583 tablet.cc:1722] T f92d26bcde3b410aa56bc23a61b1ac35 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:28.622681 1581 tablet.cc:1722] T 01ada50100bc48e39d1bc67082da750e P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:28.608368 1584 consensus_queue.cc:1048] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20251211 20:52:28.557754 1163 ts_tablet_manager.cc:1929] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:28.627774 1163 log.cc:1199] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/33e1d8a576cd41b1a34778b1458df396
I20251211 20:52:28.571070 1020 tablet_service.cc:1505] Processing CreateTablet for tablet c91ff29847d54e55b19321652a857e9f (DEFAULT_TABLE table=default.Table_9c7198623d9b41158b2ac00adc92f327 [id=ac770d79782241ad808c6533bd4e4350]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.628273 1166 ts_tablet_manager.cc:1434] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.076s user 0.001s sys 0.001s
I20251211 20:52:28.628515 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c91ff29847d54e55b19321652a857e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.628677 1166 tablet_bootstrap.cc:492] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:28.571461 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 3e6f62c2deff4c11b137fc0cba5ee234 (DEFAULT_TABLE table=default.Table_96a88507426c4a45b670fa5002d8df08 [id=fdfaa0b24d2f4736aee695ed228212cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.629078 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e6f62c2deff4c11b137fc0cba5ee234. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.579780 1012 tablet_service.cc:1505] Processing CreateTablet for tablet 215f8dc74a71482daa2ca80e15010f51 (DEFAULT_TABLE table=default.Table_b7f9abd9a3b04fd4a6ef3b477330ea9e [id=9b089b452ac940fe929dc03bddca83e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.631911 1012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 215f8dc74a71482daa2ca80e15010f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.601203 1019 tablet_service.cc:1552] Processing DeleteTablet for tablet c85624ad640041c29bb87416e1992a9e with delete_type TABLET_DATA_DELETED (replaced by tablet fa79b67b9eee45e2b41c4a90967606cc) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:28.601382 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 3cd7a465249246c28f07b8fb2e42675f with delete_type TABLET_DATA_DELETED (replaced by tablet 0763a475527546bb83cee8fb823893ac) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:28.601547 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 773cf53745a64f62817113b33e3182e7 with delete_type TABLET_DATA_DELETED (replaced by tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:28.627175 1553 tablet.cc:1722] T f92d26bcde3b410aa56bc23a61b1ac35 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:28.604311 1165 rpcz_store.cc:276] 1211 20:52:27.487509 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:27.488873 (+ 1364us) service_pool.cc:225] Handling call
1211 20:52:28.275919 (+787046us) ts_tablet_manager.cc:1151] Deleting tablet 0129162969d74f5da3c08a1e6a27ab22
1211 20:52:28.275928 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:28.407128 (+131200us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:28.493437 (+ 86309us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:28.575531 (+ 82094us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":742010}
I20251211 20:52:28.638185 1160 tablet_bootstrap.cc:654] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:28.639997 1165 tablet_replica.cc:333] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:28.640235 1165 raft_consensus.cc:2243] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:28.640563 1163 ts_tablet_manager.cc:1950] T 33e1d8a576cd41b1a34778b1458df396 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:28.640730 1165 raft_consensus.cc:2272] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:28.641220 1165 ts_tablet_manager.cc:1916] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:28.641733 891 tablet_service.cc:1505] Processing CreateTablet for tablet 3e6f62c2deff4c11b137fc0cba5ee234 (DEFAULT_TABLE table=default.Table_96a88507426c4a45b670fa5002d8df08 [id=fdfaa0b24d2f4736aee695ed228212cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.642130 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e6f62c2deff4c11b137fc0cba5ee234. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.640308 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 0129162969d74f5da3c08a1e6a27ab22 (table default.Table_daeaf2f3dd24423e9060e4f09a5f4c99 [id=755ac98365be4def8589dcfd1f3e2cbf]) successfully deleted
I20251211 20:52:28.644197 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 33e1d8a576cd41b1a34778b1458df396 (table default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]) successfully deleted
I20251211 20:52:28.644465 888 tablet_service.cc:1505] Processing CreateTablet for tablet 215f8dc74a71482daa2ca80e15010f51 (DEFAULT_TABLE table=default.Table_b7f9abd9a3b04fd4a6ef3b477330ea9e [id=9b089b452ac940fe929dc03bddca83e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.644840 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 215f8dc74a71482daa2ca80e15010f51. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:28.644842 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 7ec261837cf04bf88a48ff8add1b0d66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7ec261837cf04bf88a48ff8add1b0d66 already in progress: creating tablet
I20251211 20:52:28.645444 1168 tablet_bootstrap.cc:492] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:28.645606 1168 ts_tablet_manager.cc:1403] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.068s user 0.002s sys 0.003s
W20251211 20:52:28.645352 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 4201da4241aa48dfb670a713e4453ca5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4201da4241aa48dfb670a713e4453ca5 already in progress: creating tablet
W20251211 20:52:28.647271 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 3cd7a465249246c28f07b8fb2e42675f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3cd7a465249246c28f07b8fb2e42675f already in progress: creating tablet
I20251211 20:52:28.647465 880 tablet_service.cc:1552] Processing DeleteTablet for tablet c85624ad640041c29bb87416e1992a9e with delete_type TABLET_DATA_DELETED (replaced by tablet fa79b67b9eee45e2b41c4a90967606cc) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:28.647840 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 773cf53745a64f62817113b33e3182e7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 773cf53745a64f62817113b33e3182e7 already in progress: creating tablet
W20251211 20:52:28.648291 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet c85624ad640041c29bb87416e1992a9e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c85624ad640041c29bb87416e1992a9e already in progress: creating tablet
I20251211 20:52:28.651619 886 tablet_service.cc:1505] Processing CreateTablet for tablet c91ff29847d54e55b19321652a857e9f (DEFAULT_TABLE table=default.Table_9c7198623d9b41158b2ac00adc92f327 [id=ac770d79782241ad808c6533bd4e4350]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.652261 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c91ff29847d54e55b19321652a857e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.654084 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 3cd7a465249246c28f07b8fb2e42675f with delete_type TABLET_DATA_DELETED (replaced by tablet 0763a475527546bb83cee8fb823893ac) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:28.654124 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 773cf53745a64f62817113b33e3182e7 with delete_type TABLET_DATA_DELETED (replaced by tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:28.656163 1164 ts_tablet_manager.cc:1929] T 4d98967714184fe6a3e38dd4ca7979dc P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:28.656570 1164 log.cc:1199] T 4d98967714184fe6a3e38dd4ca7979dc P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/4d98967714184fe6a3e38dd4ca7979dc
I20251211 20:52:28.656934 1164 ts_tablet_manager.cc:1950] T 4d98967714184fe6a3e38dd4ca7979dc P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:28.655110 1168 raft_consensus.cc:359] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:28.658660 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 4d98967714184fe6a3e38dd4ca7979dc (table default.Table_ebec874d437748c487f8bf9d0ccaa7fa [id=aea22a96d5b840399f2140486953c1ed]) successfully deleted
I20251211 20:52:28.658708 1168 raft_consensus.cc:385] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20251211 20:52:28.659085 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet bb112b6431af42a6a7b328dfdefbc2a2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb112b6431af42a6a7b328dfdefbc2a2 already in progress: creating tablet
W20251211 20:52:28.659389 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet e6ba12f94f5f4d688f4222a4c2fb5436 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e6ba12f94f5f4d688f4222a4c2fb5436 already in progress: creating tablet
I20251211 20:52:28.659423 1168 raft_consensus.cc:740] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
W20251211 20:52:28.660073 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet f3855ffb51a74101b26fd28429e65bc2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f3855ffb51a74101b26fd28429e65bc2 already in progress: creating tablet
I20251211 20:52:28.660277 1168 consensus_queue.cc:260] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:28.660359 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 6a61cf8f5de14adaa4a1de5bddde9a2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a61cf8f5de14adaa4a1de5bddde9a2f already in progress: creating tablet
W20251211 20:52:28.660557 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 2260850f32b541aab1128818f144b5a7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2260850f32b541aab1128818f144b5a7 already in progress: creating tablet
W20251211 20:52:28.660801 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 25454e67437541d9ac2bfab29bad55f9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 25454e67437541d9ac2bfab29bad55f9 already in progress: creating tablet
W20251211 20:52:28.661046 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet e4b5a7553110424d901788c7a2ecdce6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e4b5a7553110424d901788c7a2ecdce6 already in progress: creating tablet
I20251211 20:52:28.661592 1168 ts_tablet_manager.cc:1434] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20251211 20:52:28.661967 1168 tablet_bootstrap.cc:492] T f00f7c1f6a4e44ff89341b4a55cc93fb P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:28.664026 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 4201da4241aa48dfb670a713e4453ca5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4201da4241aa48dfb670a713e4453ca5 already in progress: creating tablet
I20251211 20:52:28.667243 740 tablet_service.cc:1505] Processing CreateTablet for tablet 04086eaa32a74edd84235d3b28f9f965 (DEFAULT_TABLE table=default.Table_2c7bb884f8b5488e9a80e5d20e75cc95 [id=4942c719263148deb2c5cb57a605672a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.667559 887 tablet_service.cc:1505] Processing CreateTablet for tablet eb9da081cb9d43fd9a35c4bc51ea133e (DEFAULT_TABLE table=default.Table_a17987402c844d2cbef2310a1375db03 [id=e0cd4de0a942462d9e3b50079f6b757d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.667901 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04086eaa32a74edd84235d3b28f9f965. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.667980 882 tablet_service.cc:1505] Processing CreateTablet for tablet 3562b5effa124378beac7ccf0d27ee7e (DEFAULT_TABLE table=default.Table_7f5df390c11943918af61066a7c67c66 [id=157a0b5022e445e68ac805168ebc586f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.668679 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb9da081cb9d43fd9a35c4bc51ea133e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.668982 880 tablet_service.cc:1505] Processing CreateTablet for tablet 04086eaa32a74edd84235d3b28f9f965 (DEFAULT_TABLE table=default.Table_2c7bb884f8b5488e9a80e5d20e75cc95 [id=4942c719263148deb2c5cb57a605672a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.669674 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3562b5effa124378beac7ccf0d27ee7e. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:28.670284 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 4ce495233a5e4f3792faf44680df54ba because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ce495233a5e4f3792faf44680df54ba already in progress: creating tablet
I20251211 20:52:28.670753 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04086eaa32a74edd84235d3b28f9f965. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.674505 739 tablet_service.cc:1505] Processing CreateTablet for tablet eb9da081cb9d43fd9a35c4bc51ea133e (DEFAULT_TABLE table=default.Table_a17987402c844d2cbef2310a1375db03 [id=e0cd4de0a942462d9e3b50079f6b757d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.674888 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb9da081cb9d43fd9a35c4bc51ea133e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.678196 878 tablet_service.cc:1505] Processing CreateTablet for tablet d37a7d0d662e45079b0cfc02765d2e01 (DEFAULT_TABLE table=default.Table_b426c9f38ff44d76af3480168e4003cf [id=8a12299efe714617a970fc05c3bb6232]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.678472 749 tablet_service.cc:1505] Processing CreateTablet for tablet 3562b5effa124378beac7ccf0d27ee7e (DEFAULT_TABLE table=default.Table_7f5df390c11943918af61066a7c67c66 [id=157a0b5022e445e68ac805168ebc586f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.678783 885 tablet_service.cc:1505] Processing CreateTablet for tablet efe4d6bb07374294b8ab50a9fe605f87 (DEFAULT_TABLE table=default.Table_987db791387d4fa79e285fae8bddce52 [id=9538422c0e9e48a5ae065754c84ba581]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.678787 747 tablet_service.cc:1505] Processing CreateTablet for tablet 4a39870c7eea4a379b284ea2e5ddf760 (DEFAULT_TABLE table=default.Table_4813c06395dd41c78eceebb97804cdad [id=b595480f79174541ba491a21a2082692]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.679064 883 tablet_service.cc:1505] Processing CreateTablet for tablet 4a39870c7eea4a379b284ea2e5ddf760 (DEFAULT_TABLE table=default.Table_4813c06395dd41c78eceebb97804cdad [id=b595480f79174541ba491a21a2082692]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.678884 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3562b5effa124378beac7ccf0d27ee7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.682364 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a39870c7eea4a379b284ea2e5ddf760. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.685580 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d37a7d0d662e45079b0cfc02765d2e01. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.687898 879 tablet_service.cc:1505] Processing CreateTablet for tablet ee7b93b0b87541d9af6eb9e68b4cea41 (DEFAULT_TABLE table=default.Table_9ab1d237fd114e33b86d1df0630ee7b9 [id=a4f984b4263a4bc38f6248628f643ae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.688304 756 tablet_service.cc:1505] Processing CreateTablet for tablet d37a7d0d662e45079b0cfc02765d2e01 (DEFAULT_TABLE table=default.Table_b426c9f38ff44d76af3480168e4003cf [id=8a12299efe714617a970fc05c3bb6232]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.688637 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d37a7d0d662e45079b0cfc02765d2e01. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.688834 890 tablet_service.cc:1505] Processing CreateTablet for tablet 4d2b7e2a0abe4592bc806d5ef3f894d8 (DEFAULT_TABLE table=default.Table_687a33fe231c48689b4e08ef5d2d087b [id=126810ac1b5c435ba83fc41c093cc2ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.689169 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a39870c7eea4a379b284ea2e5ddf760. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.689579 1164 tablet_replica.cc:333] T 33e1d8a576cd41b1a34778b1458df396 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:28.690274 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d2b7e2a0abe4592bc806d5ef3f894d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.691064 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee7b93b0b87541d9af6eb9e68b4cea41. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.691730 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efe4d6bb07374294b8ab50a9fe605f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.693266 884 tablet_service.cc:1505] Processing CreateTablet for tablet 75dd170c5828409fa69bd68fdf28a72b (DEFAULT_TABLE table=default.Table_92ff976cb6724b6199d251d20d56b101 [id=3c8c4257d5834939af83e366994f096d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.693863 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75dd170c5828409fa69bd68fdf28a72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.697736 881 tablet_service.cc:1505] Processing CreateTablet for tablet 6b7dd8232d974a43b99ec9b28075dc8d (DEFAULT_TABLE table=default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.698621 1164 raft_consensus.cc:2243] T 33e1d8a576cd41b1a34778b1458df396 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:28.698776 1582 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:37886 (request call id 317) took 1161 ms. Trace:
I20251211 20:52:28.700310 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b7dd8232d974a43b99ec9b28075dc8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.700891 876 tablet_service.cc:1505] Processing CreateTablet for tablet 5453f68789fd4e89b69e98860050865b (DEFAULT_TABLE table=default.Table_424b336a2ac84471bac647eab716d19c [id=50d88b663a4d4891b113aee9c5906699]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.701072 1581 tablet.cc:1717] T 01ada50100bc48e39d1bc67082da750e P 6f8d784fd1cd492bbe3809403ec34919: Skipping requested alter to schema version 3, tablet already version 3
I20251211 20:52:28.703516 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f2ce490e0c5d445ab84989aae77c2f19" } new_table_name: "default.Table_2a25c4a0088d463684aa33e76ad0dd66" modify_external_catalogs: true
I20251211 20:52:28.704434 876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5453f68789fd4e89b69e98860050865b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.704766 893 tablet_service.cc:1505] Processing CreateTablet for tablet f22e4d34cde24b37a025c1b5a1046b1b (DEFAULT_TABLE table=default.Table_60e84f16bbb3462682f8bdaffd7d1542 [id=981d70bcd97a42bc8a5b7cd3d0e5f0e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.705135 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f22e4d34cde24b37a025c1b5a1046b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.705411 886 tablet_service.cc:1505] Processing CreateTablet for tablet bbe3529c301a4a65b58b634a443e1fb6 (DEFAULT_TABLE table=default.Table_274e759f68cb487da862431c34237c5c [id=3b8e637fa85244c1998d139c436c51cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.705670 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_687a33fe231c48689b4e08ef5d2d087b" } modify_external_catalogs: true
I20251211 20:52:28.705672 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 04086eaa32a74edd84235d3b28f9f965 (DEFAULT_TABLE table=default.Table_2c7bb884f8b5488e9a80e5d20e75cc95 [id=4942c719263148deb2c5cb57a605672a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.705825 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_687a33fe231c48689b4e08ef5d2d087b" } modify_external_catalogs: true
I20251211 20:52:28.705840 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbe3529c301a4a65b58b634a443e1fb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.706043 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04086eaa32a74edd84235d3b28f9f965. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.706554 1164 raft_consensus.cc:2272] T 33e1d8a576cd41b1a34778b1458df396 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:28.706972 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_d73d156809724a399ad27846bfcc460b" } modify_external_catalogs: true
I20251211 20:52:28.707130 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_d73d156809724a399ad27846bfcc460b" } modify_external_catalogs: true
I20251211 20:52:28.707569 1164 ts_tablet_manager.cc:1916] T 33e1d8a576cd41b1a34778b1458df396 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:28.708248 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_59fd4cd22d6b4065bc70dc11849fa628" } modify_external_catalogs: true
I20251211 20:52:28.708426 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_59fd4cd22d6b4065bc70dc11849fa628" } modify_external_catalogs: true
I20251211 20:52:28.709504 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_021d4fb1667a475d87424f23aa0d5e28" } modify_external_catalogs: true
I20251211 20:52:28.709650 1166 tablet_bootstrap.cc:654] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:28.709653 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_021d4fb1667a475d87424f23aa0d5e28" } modify_external_catalogs: true
I20251211 20:52:28.711556 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_47b57806fd3149c8ab84567d76177fce"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:28.712078 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_47b57806fd3149c8ab84567d76177fce in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.712699 1013 tablet_service.cc:1505] Processing CreateTablet for tablet eb9da081cb9d43fd9a35c4bc51ea133e (DEFAULT_TABLE table=default.Table_a17987402c844d2cbef2310a1375db03 [id=e0cd4de0a942462d9e3b50079f6b757d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.716105 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 3562b5effa124378beac7ccf0d27ee7e (DEFAULT_TABLE table=default.Table_7f5df390c11943918af61066a7c67c66 [id=157a0b5022e445e68ac805168ebc586f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.717005 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb9da081cb9d43fd9a35c4bc51ea133e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.717880 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_afd3c9ee35ad41caa1717a299443409e" } modify_external_catalogs: true
I20251211 20:52:28.718055 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_afd3c9ee35ad41caa1717a299443409e" } modify_external_catalogs: true
I20251211 20:52:28.719537 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_29c42f496ae04f19b277dabacd834892" } new_table_name: "default.Table_bf722f720fc64f2c875ab1edbcaebaf9" modify_external_catalogs: true
I20251211 20:52:28.729836 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet fb2d30147cd74d1893783be1e1694d1f
I20251211 20:52:28.729969 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce
I20251211 20:52:28.730525 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet d5f0ba3ab16240f9822be69f941f5da2
I20251211 20:52:28.730532 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 4d2b7e2a0abe4592bc806d5ef3f894d8
I20251211 20:52:28.731315 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_70c84990b84847979bb2408ce8aaea00" } new_table_name: "default.Table_16b2ebd2f97940acbda1e91061ef1cf6" modify_external_catalogs: true
I20251211 20:52:28.731814 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9ab1d237fd114e33b86d1df0630ee7b9" } new_table_name: "default.Table_568264fe4a14485bb77966a76aefdf9a" modify_external_catalogs: true
I20251211 20:52:28.731863 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_737b2e537604423ab32fd1cc85baab78"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:28.732384 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_737b2e537604423ab32fd1cc85baab78 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.733565 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_3542335cc1af47b3817527896d07f03a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:28.734069 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3542335cc1af47b3817527896d07f03a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.735935 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_654982f288d34c08bcf3ef3426638abd" } new_table_name: "default.Table_63cd6bb6292e4535afd64845a12e0149" modify_external_catalogs: true
I20251211 20:52:28.738049 892 tablet_service.cc:1505] Processing CreateTablet for tablet 1578887b2f5748cd83a6fbdec2f3fc99 (DEFAULT_TABLE table=default.Table_c0ed4563b1d44eafa92065d3ec8adb93 [id=9e7c08f833e14a90b17a9b9f60b702de]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.738531 877 tablet_service.cc:1505] Processing CreateTablet for tablet e484d499af1b4947adb36c32f8ea5704 (DEFAULT_TABLE table=default.Table_07f1881614084afd936866d115aecbe7 [id=eed7dfa66ab545dca02bd10c0a6a0e71]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.742651 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e484d499af1b4947adb36c32f8ea5704. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.743824 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1578887b2f5748cd83a6fbdec2f3fc99. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.749325 880 tablet_service.cc:1505] Processing CreateTablet for tablet b85af79c2574443998e847bc19c53b69 (DEFAULT_TABLE table=default.Table_c4fc3456c46345c3955a0c7f79556c05 [id=15fad5e1cf8d4ea08dae8275a29a11dd]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:28.749842 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0aac6fb3972e4aba8950828c9e49d507 consensus state has no leader
I20251211 20:52:28.750238 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 68be21a25c6743f5b86d518d98fbc35e
W20251211 20:52:28.752638 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ee7b93b0b87541d9af6eb9e68b4cea41 consensus state has no leader
W20251211 20:52:28.752287 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0b267c7feaac43f6aacb7229759c7ec3 consensus state has no leader
I20251211 20:52:28.754683 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_fac9e7762d1a492f87a0e5b18868f2ea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:28.755702 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fac9e7762d1a492f87a0e5b18868f2ea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.759644 881 tablet_service.cc:1505] Processing CreateTablet for tablet 0aac6fb3972e4aba8950828c9e49d507 (DEFAULT_TABLE table=default.Table_29c42f496ae04f19b277dabacd834892 [id=fdebf1e48cc24020867e6b24ef2b55b4]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:28.761431 591 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b529b38d754f46ba8cb60b19e9705256 consensus state has no leader
I20251211 20:52:28.722136 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 4a39870c7eea4a379b284ea2e5ddf760 (DEFAULT_TABLE table=default.Table_4813c06395dd41c78eceebb97804cdad [id=b595480f79174541ba491a21a2082692]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.765841 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 75dd170c5828409fa69bd68fdf28a72b (DEFAULT_TABLE table=default.Table_92ff976cb6724b6199d251d20d56b101 [id=3c8c4257d5834939af83e366994f096d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.766764 1016 tablet_service.cc:1505] Processing CreateTablet for tablet 5453f68789fd4e89b69e98860050865b (DEFAULT_TABLE table=default.Table_424b336a2ac84471bac647eab716d19c [id=50d88b663a4d4891b113aee9c5906699]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.767243 1023 tablet_service.cc:1505] Processing CreateTablet for tablet f22e4d34cde24b37a025c1b5a1046b1b (DEFAULT_TABLE table=default.Table_60e84f16bbb3462682f8bdaffd7d1542 [id=981d70bcd97a42bc8a5b7cd3d0e5f0e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.767549 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_441ecee3b8204f9fa8d22146340e5303"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:28.768100 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_441ecee3b8204f9fa8d22146340e5303 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.768533 876 tablet_service.cc:1505] Processing CreateTablet for tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce (DEFAULT_TABLE table=default.Table_021d4fb1667a475d87424f23aa0d5e28 [id=6bbfcc2561f349519a69ae1f56016cc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.769023 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 0aac6fb3972e4aba8950828c9e49d507 (DEFAULT_TABLE table=default.Table_29c42f496ae04f19b277dabacd834892 [id=fdebf1e48cc24020867e6b24ef2b55b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.769502 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_df56cd18522b410fade2a880c087eff4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:28.769809 1021 tablet_service.cc:1505] Processing CreateTablet for tablet 4d2b7e2a0abe4592bc806d5ef3f894d8 (DEFAULT_TABLE table=default.Table_687a33fe231c48689b4e08ef5d2d087b [id=126810ac1b5c435ba83fc41c093cc2ff]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:28.770016 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_df56cd18522b410fade2a880c087eff4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.766446 1028 tablet_service.cc:1505] Processing CreateTablet for tablet 6b7dd8232d974a43b99ec9b28075dc8d (DEFAULT_TABLE table=default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.770982 1011 tablet_service.cc:1505] Processing CreateTablet for tablet d37a7d0d662e45079b0cfc02765d2e01 (DEFAULT_TABLE table=default.Table_b426c9f38ff44d76af3480168e4003cf [id=8a12299efe714617a970fc05c3bb6232]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.773593 1018 tablet_service.cc:1505] Processing CreateTablet for tablet bbe3529c301a4a65b58b634a443e1fb6 (DEFAULT_TABLE table=default.Table_274e759f68cb487da862431c34237c5c [id=3b8e637fa85244c1998d139c436c51cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.774601 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3562b5effa124378beac7ccf0d27ee7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.779170 875 tablet_service.cc:1552] Processing DeleteTablet for tablet fb2d30147cd74d1893783be1e1694d1f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:28.781000 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0aac6fb3972e4aba8950828c9e49d507. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.784662 1024 tablet_service.cc:1505] Processing CreateTablet for tablet e484d499af1b4947adb36c32f8ea5704 (DEFAULT_TABLE table=default.Table_07f1881614084afd936866d115aecbe7 [id=eed7dfa66ab545dca02bd10c0a6a0e71]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.786538 1015 tablet_service.cc:1505] Processing CreateTablet for tablet 1578887b2f5748cd83a6fbdec2f3fc99 (DEFAULT_TABLE table=default.Table_c0ed4563b1d44eafa92065d3ec8adb93 [id=9e7c08f833e14a90b17a9b9f60b702de]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.790271 1026 tablet_service.cc:1505] Processing CreateTablet for tablet fa79b67b9eee45e2b41c4a90967606cc (DEFAULT_TABLE table=default.Table_8f7e347b9ac64793b1073dfa955ebe21 [id=a6ab64c3e0d94462b716f2d153bc3314]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.790997 1029 tablet_service.cc:1505] Processing CreateTablet for tablet ee7b93b0b87541d9af6eb9e68b4cea41 (DEFAULT_TABLE table=default.Table_9ab1d237fd114e33b86d1df0630ee7b9 [id=a4f984b4263a4bc38f6248628f643ae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.792156 1014 tablet_service.cc:1505] Processing CreateTablet for tablet efe4d6bb07374294b8ab50a9fe605f87 (DEFAULT_TABLE table=default.Table_987db791387d4fa79e285fae8bddce52 [id=9538422c0e9e48a5ae065754c84ba581]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.792409 1027 tablet_service.cc:1505] Processing CreateTablet for tablet b85af79c2574443998e847bc19c53b69 (DEFAULT_TABLE table=default.Table_c4fc3456c46345c3955a0c7f79556c05 [id=15fad5e1cf8d4ea08dae8275a29a11dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.810420 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_05f224658edf4b369163c847787e14bf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:28.811051 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_05f224658edf4b369163c847787e14bf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.812326 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_2edfd010f1ef40e8be655908fdc4800e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:28.812817 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2edfd010f1ef40e8be655908fdc4800e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.826493 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_37520ef8d4194d8aa3eadef57196a291"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:28.827118 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_37520ef8d4194d8aa3eadef57196a291 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.830571 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_6a2c9aead0a640b79bb4c9d6f2258e67"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:28.831135 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6a2c9aead0a640b79bb4c9d6f2258e67 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:28.835148 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 4d2b7e2a0abe4592bc806d5ef3f894d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:28.836342 893 tablet_service.cc:1505] Processing CreateTablet for tablet 0763a475527546bb83cee8fb823893ac (DEFAULT_TABLE table=default.Table_227fbf051d5643528315ca00194583bc [id=1515561ab78d46f6b5aa37a20d90d3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.841236 875 tablet_service.cc:1552] Processing DeleteTablet for tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:28.846989 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 28ddb8a59a214576a25d613d42c2fef5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 28ddb8a59a214576a25d613d42c2fef5 already in progress: creating tablet
W20251211 20:52:28.847338 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet c7147571483e41ad970e573a08c368ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c7147571483e41ad970e573a08c368ca already in progress: creating tablet
W20251211 20:52:28.847612 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 7ec261837cf04bf88a48ff8add1b0d66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7ec261837cf04bf88a48ff8add1b0d66 already in progress: creating tablet
I20251211 20:52:28.849427 1012 tablet_service.cc:1505] Processing CreateTablet for tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce (DEFAULT_TABLE table=default.Table_021d4fb1667a475d87424f23aa0d5e28 [id=6bbfcc2561f349519a69ae1f56016cc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.851007 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f22e4d34cde24b37a025c1b5a1046b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.852833 1166 tablet_bootstrap.cc:492] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:28.853230 891 tablet_service.cc:1505] Processing CreateTablet for tablet fa79b67b9eee45e2b41c4a90967606cc (DEFAULT_TABLE table=default.Table_8f7e347b9ac64793b1073dfa955ebe21 [id=a6ab64c3e0d94462b716f2d153bc3314]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.855805 1011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d37a7d0d662e45079b0cfc02765d2e01. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.856925 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa79b67b9eee45e2b41c4a90967606cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.866312 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 68be21a25c6743f5b86d518d98fbc35e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:28.871523 1016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5453f68789fd4e89b69e98860050865b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.872771 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a39870c7eea4a379b284ea2e5ddf760. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.873039 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b85af79c2574443998e847bc19c53b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.873546 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0aac6fb3972e4aba8950828c9e49d507. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.874019 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b7dd8232d974a43b99ec9b28075dc8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.874102 1166 ts_tablet_manager.cc:1403] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.225s user 0.011s sys 0.008s
I20251211 20:52:28.879746 884 tablet_service.cc:1552] Processing DeleteTablet for tablet d5f0ba3ab16240f9822be69f941f5da2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:28.880224 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0763a475527546bb83cee8fb823893ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.875056 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75dd170c5828409fa69bd68fdf28a72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.888485 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1578887b2f5748cd83a6fbdec2f3fc99. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.891736 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d2b7e2a0abe4592bc806d5ef3f894d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.892007 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b85af79c2574443998e847bc19c53b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.893819 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e484d499af1b4947adb36c32f8ea5704. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.895745 1014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efe4d6bb07374294b8ab50a9fe605f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.896178 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbe3529c301a4a65b58b634a443e1fb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.901729 1012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.904733 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:28.905052 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa79b67b9eee45e2b41c4a90967606cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.905643 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet fb2d30147cd74d1893783be1e1694d1f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:28.906154 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet d5f0ba3ab16240f9822be69f941f5da2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:28.906673 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 4d2b7e2a0abe4592bc806d5ef3f894d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:28.907137 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 68be21a25c6743f5b86d518d98fbc35e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:28.883229 1166 raft_consensus.cc:359] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:28.908596 1166 raft_consensus.cc:385] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:28.908666 1166 raft_consensus.cc:740] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:28.908936 1166 consensus_queue.cc:260] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:28.874984 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 0763a475527546bb83cee8fb823893ac (DEFAULT_TABLE table=default.Table_227fbf051d5643528315ca00194583bc [id=1515561ab78d46f6b5aa37a20d90d3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.914811 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee7b93b0b87541d9af6eb9e68b4cea41. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.921557 1166 ts_tablet_manager.cc:1434] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.043s user 0.001s sys 0.001s
I20251211 20:52:28.923782 1166 tablet_bootstrap.cc:492] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:28.927076 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0763a475527546bb83cee8fb823893ac. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:28.927490 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce because the tablet was not found. No further retry: Not found: Tablet not found: 3ecf0ea8ce19486c81d1d1a91d57a7ce
I20251211 20:52:28.933673 1166 tablet_bootstrap.cc:654] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:28.936344 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.StartTabletCopy -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 48111us
I20251211 20:52:28.938525 745 tablet_service.cc:1505] Processing CreateTablet for tablet 4d2b7e2a0abe4592bc806d5ef3f894d8 (DEFAULT_TABLE table=default.Table_687a33fe231c48689b4e08ef5d2d087b [id=126810ac1b5c435ba83fc41c093cc2ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.789641 749 tablet_service.cc:1505] Processing CreateTablet for tablet efe4d6bb07374294b8ab50a9fe605f87 (DEFAULT_TABLE table=default.Table_987db791387d4fa79e285fae8bddce52 [id=9538422c0e9e48a5ae065754c84ba581]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.941601 753 tablet_service.cc:1505] Processing CreateTablet for tablet b85af79c2574443998e847bc19c53b69 (DEFAULT_TABLE table=default.Table_c4fc3456c46345c3955a0c7f79556c05 [id=15fad5e1cf8d4ea08dae8275a29a11dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.941895 755 tablet_service.cc:1505] Processing CreateTablet for tablet bbe3529c301a4a65b58b634a443e1fb6 (DEFAULT_TABLE table=default.Table_274e759f68cb487da862431c34237c5c [id=3b8e637fa85244c1998d139c436c51cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.944160 742 tablet_service.cc:1552] Processing DeleteTablet for tablet d5f0ba3ab16240f9822be69f941f5da2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:28.942205 746 tablet_service.cc:1505] Processing CreateTablet for tablet f22e4d34cde24b37a025c1b5a1046b1b (DEFAULT_TABLE table=default.Table_60e84f16bbb3462682f8bdaffd7d1542 [id=981d70bcd97a42bc8a5b7cd3d0e5f0e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.947099 747 tablet_service.cc:1505] Processing CreateTablet for tablet fa79b67b9eee45e2b41c4a90967606cc (DEFAULT_TABLE table=default.Table_8f7e347b9ac64793b1073dfa955ebe21 [id=a6ab64c3e0d94462b716f2d153bc3314]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.947347 741 tablet_service.cc:1505] Processing CreateTablet for tablet 5453f68789fd4e89b69e98860050865b (DEFAULT_TABLE table=default.Table_424b336a2ac84471bac647eab716d19c [id=50d88b663a4d4891b113aee9c5906699]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.948719 750 tablet_service.cc:1505] Processing CreateTablet for tablet ee7b93b0b87541d9af6eb9e68b4cea41 (DEFAULT_TABLE table=default.Table_9ab1d237fd114e33b86d1df0630ee7b9 [id=a4f984b4263a4bc38f6248628f643ae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.949438 754 tablet_service.cc:1505] Processing CreateTablet for tablet e484d499af1b4947adb36c32f8ea5704 (DEFAULT_TABLE table=default.Table_07f1881614084afd936866d115aecbe7 [id=eed7dfa66ab545dca02bd10c0a6a0e71]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.949961 738 tablet_service.cc:1505] Processing CreateTablet for tablet 75dd170c5828409fa69bd68fdf28a72b (DEFAULT_TABLE table=default.Table_92ff976cb6724b6199d251d20d56b101 [id=3c8c4257d5834939af83e366994f096d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.950228 757 tablet_service.cc:1505] Processing CreateTablet for tablet 1578887b2f5748cd83a6fbdec2f3fc99 (DEFAULT_TABLE table=default.Table_c0ed4563b1d44eafa92065d3ec8adb93 [id=9e7c08f833e14a90b17a9b9f60b702de]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.951570 743 tablet_service.cc:1505] Processing CreateTablet for tablet 6b7dd8232d974a43b99ec9b28075dc8d (DEFAULT_TABLE table=default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.959542 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efe4d6bb07374294b8ab50a9fe605f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.959798 1168 tablet_bootstrap.cc:654] T f00f7c1f6a4e44ff89341b4a55cc93fb P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:28.969228 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee7b93b0b87541d9af6eb9e68b4cea41. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.971717 738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75dd170c5828409fa69bd68fdf28a72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.975816 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5453f68789fd4e89b69e98860050865b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.979719 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e484d499af1b4947adb36c32f8ea5704. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.962172 1583 tablet.cc:1722] T 4870da0224e048ed82f227d7ba676a94 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:28.962231 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 68be21a25c6743f5b86d518d98fbc35e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:28.962302 740 tablet_service.cc:1505] Processing CreateTablet for tablet 0763a475527546bb83cee8fb823893ac (DEFAULT_TABLE table=default.Table_227fbf051d5643528315ca00194583bc [id=1515561ab78d46f6b5aa37a20d90d3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.962378 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:28.962481 748 tablet_service.cc:1552] Processing DeleteTablet for tablet fb2d30147cd74d1893783be1e1694d1f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:28.963132 752 tablet_service.cc:1552] Processing DeleteTablet for tablet 4d2b7e2a0abe4592bc806d5ef3f894d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:28.963086 756 tablet_service.cc:1505] Processing CreateTablet for tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce (DEFAULT_TABLE table=default.Table_021d4fb1667a475d87424f23aa0d5e28 [id=6bbfcc2561f349519a69ae1f56016cc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:28.699064 1582 rpcz_store.cc:276] 1211 20:52:27.493902 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:27.494691 (+ 789us) service_pool.cc:225] Handling call
1211 20:52:28.654991 (+1160300us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:27.938851 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
1211 20:52:27.939301 (+ 450us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x60c0002b3700. stack: 00007fc66457bee6 00007fc66457bca0 00007fc6635573ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fc67813dcc5 00007fc66fcf1210 00007fc6645a0890 0000000000a76196 00007fc6645bfeaa 00007fc6645bb8cc 00007fc6645bad1e 00007fc6717fb113 00007fc67179d79f 00007fc67428e563
1211 20:52:28.283253 (+343952us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
1211 20:52:28.283264 (+ 11us) alter_schema_op.cc:66] Acquired schema lock
1211 20:52:28.283274 (+ 10us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
1211 20:52:28.283331 (+ 57us) alter_schema_op.cc:128] START. Timestamp: 1765486348283292
1211 20:52:28.283335 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:28.481229 (+197894us) log.cc:844] Serialized 173 byte log entry
1211 20:52:28.481556 (+ 327us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x6160000d1554. stack: 00007fc66457bee6 00007fc66457bca0 00007fc6635573ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fc67041b922 00007fc6717fe875 00007fc6717fd399 00007fc6718057c0 00007fc6718055cd 0000000000a8970c 00007fc6645c220f 00007fc6645c552b 00007fc6645c53a8 0000000000a8970c
1211 20:52:28.513216 (+ 31660us) op_driver.cc:464] REPLICATION: finished
1211 20:52:28.603795 (+ 90579us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
1211 20:52:28.640512 (+ 36717us) log.cc:844] Serialized 5 byte log entry
1211 20:52:28.654355 (+ 13843us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:28.654628 (+ 273us) log.cc:844] Serialized 14 byte log entry
1211 20:52:28.654945 (+ 317us) alter_schema_op.cc:73] Released schema lock
1211 20:52:28.654950 (+ 5us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":90499,"prepare.queue_time_us":439959,"prepare.run_cpu_time_us":2518,"prepare.run_wall_time_us":542763,"raft.queue_time_us":89,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":229,"replication_time_us":229847,"spinlock_wait_cycles":462734464,"thread_start_us":530968,"threads_started":2}]]}
I20251211 20:52:28.990645 743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b7dd8232d974a43b99ec9b28075dc8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.991003 1574 raft_consensus.cc:493] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:28.991515 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f22e4d34cde24b37a025c1b5a1046b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:28.996065 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d2b7e2a0abe4592bc806d5ef3f894d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.002592 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b85af79c2574443998e847bc19c53b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.007627 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa79b67b9eee45e2b41c4a90967606cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.009524 1574 raft_consensus.cc:515] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.010257 1574 leader_election.cc:290] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:29.010828 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1578887b2f5748cd83a6fbdec2f3fc99. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.018528 1165 ts_tablet_manager.cc:1929] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:29.018677 1165 log.cc:1199] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/4ef506349e5b434db7a78809dc44014e
I20251211 20:52:29.019050 1165 ts_tablet_manager.cc:1950] T 4ef506349e5b434db7a78809dc44014e P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:29.019963 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbe3529c301a4a65b58b634a443e1fb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.021482 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0763a475527546bb83cee8fb823893ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.022359 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:29.036391 556 rpcz_store.cc:267] Call kudu.server.GenericService.CheckLeaks from 127.0.0.1:57216 (request call id 27) took 908 ms (client timeout 1000 ms). Trace:
W20251211 20:52:29.036679 556 rpcz_store.cc:269] 1211 20:52:28.127960 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:28.128046 (+ 86us) service_pool.cc:225] Handling call
1211 20:52:29.036308 (+908262us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20251211 20:52:29.040076 416 external_mini_cluster.cc:1658] Killing /tmp/dist-test-task5TWsQc/build/asan/bin/kudu with pid 492
I20251211 20:52:29.075809 1586 raft_consensus.cc:493] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251211 20:52:29.082208 710 leader_election.cc:343] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 1c04187b8ed146c289a6b17d62b07199
I20251211 20:52:29.083046 1583 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:37886 (request call id 318) took 1589 ms. Trace:
I20251211 20:52:29.086010 1583 rpcz_store.cc:276] 1211 20:52:27.528315 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:27.529114 (+ 799us) service_pool.cc:225] Handling call
1211 20:52:29.082985 (+1553871us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:27.947727 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
1211 20:52:28.089304 (+141577us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
1211 20:52:28.089313 (+ 9us) alter_schema_op.cc:66] Acquired schema lock
1211 20:52:28.089321 (+ 8us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
1211 20:52:28.089367 (+ 46us) alter_schema_op.cc:128] START. Timestamp: 1765486348089307
1211 20:52:28.089370 (+ 3us) op_driver.cc:348] REPLICATION: starting
1211 20:52:28.091257 (+ 1887us) log.cc:844] Serialized 173 byte log entry
1211 20:52:28.940325 (+849068us) op_driver.cc:464] REPLICATION: finished
1211 20:52:28.962125 (+ 21800us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
1211 20:52:29.021048 (+ 58923us) log.cc:844] Serialized 5 byte log entry
1211 20:52:29.072506 (+ 51458us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:29.072828 (+ 322us) log.cc:844] Serialized 14 byte log entry
1211 20:52:29.082937 (+ 10109us) alter_schema_op.cc:73] Released schema lock
1211 20:52:29.082943 (+ 6us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":21750,"prepare.queue_time_us":414199,"prepare.run_cpu_time_us":14708,"prepare.run_wall_time_us":599772,"replication_time_us":850918,"thread_start_us":721494,"threads_started":2,"wal-append.queue_time_us":356324}]]}
I20251211 20:52:29.077477 1586 raft_consensus.cc:515] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.089370 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee893e33255745debf73c486d8c70267" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:29.082211 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 312) took 1588 ms. Trace:
I20251211 20:52:29.089769 1586 leader_election.cc:290] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:29.089749 1165 rpcz_store.cc:276] 1211 20:52:27.493389 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:27.494828 (+ 1439us) service_pool.cc:225] Handling call
1211 20:52:28.639644 (+1144816us) ts_tablet_manager.cc:1151] Deleting tablet 4ef506349e5b434db7a78809dc44014e
1211 20:52:28.639653 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:28.937640 (+297987us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:28.959331 (+ 21691us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:29.082149 (+122818us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":597,"tablet-delete.queue_time_us":1059511}
W20251211 20:52:29.090211 982 leader_election.cc:343] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:29.090451 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 347) took 1447 ms. Trace:
I20251211 20:52:29.090708 1165 rpcz_store.cc:276] 1211 20:52:27.643092 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:27.932529 (+289437us) service_pool.cc:225] Handling call
1211 20:52:29.090225 (+1157696us) ts_tablet_manager.cc:1151] Deleting tablet 22273287aa9245a2a66131fed2c8861c
1211 20:52:29.090235 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:29.090419 (+ 184us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1153562}
I20251211 20:52:29.091375 1165 tablet_replica.cc:333] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:29.091789 1165 raft_consensus.cc:2243] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:29.092548 1165 pending_rounds.cc:70] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919: Trying to abort 1 pending ops.
I20251211 20:52:29.093050 1165 pending_rounds.cc:77] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432076313030656 op_type: NO_OP noop_request { }
I20251211 20:52:29.093160 1165 raft_consensus.cc:2889] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:29.093278 1165 raft_consensus.cc:2272] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:29.093791 1165 ts_tablet_manager.cc:1916] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:29.097838 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_ec893136f41e41f6a6db70f193b8065e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:29.098872 876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.098984 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_cedad3ae249f4796b8c3756c4f4d6f74" } modify_external_catalogs: true
I20251211 20:52:29.100664 739 tablet_service.cc:1505] Processing CreateTablet for tablet 0aac6fb3972e4aba8950828c9e49d507 (DEFAULT_TABLE table=default.Table_29c42f496ae04f19b277dabacd834892 [id=fdebf1e48cc24020867e6b24ef2b55b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.101148 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0aac6fb3972e4aba8950828c9e49d507. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.108330 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_2915f90c5dbb4d868402e18273217109" } new_table_name: "default.Table_28744eba003244878ff0195d5d38643e" modify_external_catalogs: true
I20251211 20:52:29.116088 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_40b5190e58a440e9a13d6c9221572914" } modify_external_catalogs: true
I20251211 20:52:29.116310 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_40b5190e58a440e9a13d6c9221572914" } modify_external_catalogs: true
I20251211 20:52:29.117340 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_893c9504152b412eb7177a511fcedc04" } modify_external_catalogs: true
I20251211 20:52:29.108331 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_cedad3ae249f4796b8c3756c4f4d6f74" } modify_external_catalogs: true
I20251211 20:52:29.120242 1166 tablet_bootstrap.cc:492] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:29.120405 1166 ts_tablet_manager.cc:1403] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.199s user 0.048s sys 0.084s
I20251211 20:52:29.120545 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8b8ed719985f42128e56bc26105e7a9c" } modify_external_catalogs: true
I20251211 20:52:29.120771 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8b8ed719985f42128e56bc26105e7a9c" } modify_external_catalogs: true
I20251211 20:52:29.121198 1166 raft_consensus.cc:359] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.121361 1166 raft_consensus.cc:385] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:29.121443 1166 raft_consensus.cc:740] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:29.121663 1166 consensus_queue.cc:260] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.122738 1585 raft_consensus.cc:493] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:29.122902 1585 raft_consensus.cc:515] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.123553 1585 leader_election.cc:290] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:29.124068 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_893c9504152b412eb7177a511fcedc04" } modify_external_catalogs: true
W20251211 20:52:29.124393 710 leader_election.cc:343] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 5fc8d485cc234fd490c487a8bc881bf4
I20251211 20:52:29.124586 1166 ts_tablet_manager.cc:1434] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:29.124868 1166 tablet_bootstrap.cc:492] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:29.126688 1166 tablet_bootstrap.cc:654] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:29.134001 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_dd84a1d3fe964c76b146f798f6781baa" } modify_external_catalogs: true
I20251211 20:52:29.134199 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_dd84a1d3fe964c76b146f798f6781baa" } modify_external_catalogs: true
I20251211 20:52:29.143854 1166 tablet_bootstrap.cc:492] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:29.144101 1166 ts_tablet_manager.cc:1403] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.002s
I20251211 20:52:29.145015 1166 raft_consensus.cc:359] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.145341 1166 raft_consensus.cc:385] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:29.145422 1166 raft_consensus.cc:740] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:29.145730 1166 consensus_queue.cc:260] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.147346 1166 ts_tablet_manager.cc:1434] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:29.147735 1166 tablet_bootstrap.cc:492] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:29.149382 1166 tablet_bootstrap.cc:654] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:29.151188 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee893e33255745debf73c486d8c70267" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:29.153357 983 leader_election.cc:343] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:29.153589 983 leader_election.cc:304] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:29.154027 1586 raft_consensus.cc:2749] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:29.161154 1168 tablet_bootstrap.cc:492] T f00f7c1f6a4e44ff89341b4a55cc93fb P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:29.161334 1168 ts_tablet_manager.cc:1403] T f00f7c1f6a4e44ff89341b4a55cc93fb P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.499s user 0.010s sys 0.004s
I20251211 20:52:29.161700 1166 tablet_bootstrap.cc:492] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:29.161844 1166 ts_tablet_manager.cc:1403] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.003s
I20251211 20:52:29.162432 1553 tablet.cc:1722] T 4870da0224e048ed82f227d7ba676a94 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:29.163080 1549 mvcc.cc:204] Tried to move back new op lower bound from 7231432081773801472 to 7231432077557858304. Current Snapshot: MvccSnapshot[applied={T|T < 7231432081773801472}]
I20251211 20:52:29.163564 1166 raft_consensus.cc:359] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.163786 1166 raft_consensus.cc:385] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:29.163874 1166 raft_consensus.cc:740] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:29.164110 1166 consensus_queue.cc:260] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.164592 1166 ts_tablet_manager.cc:1434] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251211 20:52:29.164883 1166 tablet_bootstrap.cc:492] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:29.166466 1166 tablet_bootstrap.cc:654] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:29.170869 1165 ts_tablet_manager.cc:1929] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20251211 20:52:29.176151 1138 log.cc:927] Time spent T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Append to log took a long time: real 0.238s user 0.000s sys 0.000s
W20251211 20:52:29.176250 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet fb2d30147cd74d1893783be1e1694d1f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fb2d30147cd74d1893783be1e1694d1f already in progress: creating tablet
W20251211 20:52:29.176403 581 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 243857us
W20251211 20:52:29.176668 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet d5f0ba3ab16240f9822be69f941f5da2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d5f0ba3ab16240f9822be69f941f5da2 already in progress: creating tablet
W20251211 20:52:29.176914 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 4d2b7e2a0abe4592bc806d5ef3f894d8 because the tablet was not found. No further retry: Not found: Tablet not found: 4d2b7e2a0abe4592bc806d5ef3f894d8
W20251211 20:52:29.177127 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 68be21a25c6743f5b86d518d98fbc35e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 68be21a25c6743f5b86d518d98fbc35e already in progress: creating tablet
W20251211 20:52:29.177654 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ec893136f41e41f6a6db70f193b8065e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:29.179659 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 46cb1d16a0e94b3a833f32c635213fd2 (DEFAULT_TABLE table=default.Table_737b2e537604423ab32fd1cc85baab78 [id=1bdaf5ce0f6b4b99a038f9f9efb828f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.180127 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46cb1d16a0e94b3a833f32c635213fd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.181983 1166 tablet_bootstrap.cc:492] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:29.181977 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ed11d3864a3b4e04a8c20b0c3a42d684" } new_table_name: "default.Table_320e1edc81a842849ac04e448aa2e2d4" modify_external_catalogs: true
I20251211 20:52:29.183146 1166 ts_tablet_manager.cc:1403] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.004s
I20251211 20:52:29.183844 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 4ef506349e5b434db7a78809dc44014e (table default.Table_fd4577173dc44659b95db8e463e0131c [id=143faf67e1204e7bb2015f06392e48a6]) successfully deleted
I20251211 20:52:29.184101 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 52ef1707f6f2425aafd454f303af0333
I20251211 20:52:29.184818 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_880a8408f1fb4d68a3f9d30b26acffa5" } new_table_name: "default.Table_8c6f9204fa234ae7a9bc18811d75af5d" modify_external_catalogs: true
W20251211 20:52:29.185156 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 22273287aa9245a2a66131fed2c8861c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 22273287aa9245a2a66131fed2c8861c already in progress: creating tablet
I20251211 20:52:29.183884 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet e28d66ce4711430cb09a9650e99fbe9e
I20251211 20:52:29.186800 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 7452dd736b0440fcb4626858e794676f (DEFAULT_TABLE table=default.Table_3542335cc1af47b3817527896d07f03a [id=d1e2bcd20b06470bb87adf24e7b8e7fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.187214 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7452dd736b0440fcb4626858e794676f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.187597 595 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 4870da0224e048ed82f227d7ba676a94 (table default.Table_595d8c100e69422ab5e6d44cce9e733c [id=28f3ebe9b7a64b5e89404b0407b0e5ae]). Expected version 2 got 0
I20251211 20:52:29.188113 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet b631e6b3048e428f901af1b5b0b1ab3c
I20251211 20:52:29.188719 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_86450313ee0040bb87db3b340566266b" } new_table_name: "default.Table_b022d39850dc4b8f92b79b8a747f0409" modify_external_catalogs: true
I20251211 20:52:29.189008 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 605172f066d242f2abea133c08a145ca
I20251211 20:52:29.189716 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8adf3dfe2fb0445385c730198775d1e9" } new_table_name: "default.Table_58802d3fc36742ffb050db7bb7f0c064" modify_external_catalogs: true
I20251211 20:52:29.191098 890 tablet_service.cc:1505] Processing CreateTablet for tablet 46cb1d16a0e94b3a833f32c635213fd2 (DEFAULT_TABLE table=default.Table_737b2e537604423ab32fd1cc85baab78 [id=1bdaf5ce0f6b4b99a038f9f9efb828f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.191092 588 catalog_manager.cc:5654] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 reported cstate change: term changed from 0 to 2, leader changed from <none> to eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2). New cstate: current_term: 2 leader_uuid: "eb0f1933d9a74fccaa4618047fc05c41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:29.193248 1029 tablet_service.cc:1505] Processing CreateTablet for tablet c26c3d0170f64a41ae8944c6668cc5f3 (DEFAULT_TABLE table=default.Table_47b57806fd3149c8ab84567d76177fce [id=a1149606f150473ab2d908bf42316081]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.193652 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c26c3d0170f64a41ae8944c6668cc5f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.194404 1166 raft_consensus.cc:359] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.195407 1012 tablet_service.cc:1552] Processing DeleteTablet for tablet e28d66ce4711430cb09a9650e99fbe9e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:29.197278 1160 tablet_bootstrap.cc:492] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:29.197747 883 tablet_service.cc:1505] Processing CreateTablet for tablet 7452dd736b0440fcb4626858e794676f (DEFAULT_TABLE table=default.Table_3542335cc1af47b3817527896d07f03a [id=d1e2bcd20b06470bb87adf24e7b8e7fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.197806 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5fc8d485cc234fd490c487a8bc881bf4" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:29.197605 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet e28d66ce4711430cb09a9650e99fbe9e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e28d66ce4711430cb09a9650e99fbe9e already in progress: creating tablet
I20251211 20:52:29.198912 879 tablet_service.cc:1505] Processing CreateTablet for tablet c26c3d0170f64a41ae8944c6668cc5f3 (DEFAULT_TABLE table=default.Table_47b57806fd3149c8ab84567d76177fce [id=a1149606f150473ab2d908bf42316081]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.197686 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1c04187b8ed146c289a6b17d62b07199" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:29.199755 881 tablet_service.cc:1505] Processing CreateTablet for tablet 71dc8d20d93243e289330d7b8c635511 (DEFAULT_TABLE table=default.Table_fac9e7762d1a492f87a0e5b18868f2ea [id=38840bab0bbf4cc49b74c2e13614069c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.200057 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46cb1d16a0e94b3a833f32c635213fd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.202805 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 52ef1707f6f2425aafd454f303af0333 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:29.203145 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7452dd736b0440fcb4626858e794676f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.204586 1014 tablet_service.cc:1552] Processing DeleteTablet for tablet b631e6b3048e428f901af1b5b0b1ab3c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:29.205250 1012 tablet_service.cc:1552] Processing DeleteTablet for tablet 605172f066d242f2abea133c08a145ca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:29.205951 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet b631e6b3048e428f901af1b5b0b1ab3c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b631e6b3048e428f901af1b5b0b1ab3c already in progress: creating tablet
W20251211 20:52:29.206216 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 605172f066d242f2abea133c08a145ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 605172f066d242f2abea133c08a145ca already in progress: creating tablet
I20251211 20:52:29.206519 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet 52ef1707f6f2425aafd454f303af0333 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:29.206447 1166 raft_consensus.cc:385] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20251211 20:52:29.207473 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 52ef1707f6f2425aafd454f303af0333 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 52ef1707f6f2425aafd454f303af0333 already in progress: creating tablet
I20251211 20:52:29.206959 1166 raft_consensus.cc:740] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:29.208492 1166 consensus_queue.cc:260] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.209798 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet b30d2054316b49fa8096345527a45d87
W20251211 20:52:29.209898 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c4e048ca28824329b924845ccf5067cf consensus state has no leader
I20251211 20:52:29.212170 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_6814df0b3dd640f58f421a6ed9010686"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:29.212721 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6814df0b3dd640f58f421a6ed9010686 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:29.213331 1160 ts_tablet_manager.cc:1403] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.663s user 0.035s sys 0.054s
I20251211 20:52:29.213845 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_c1f789cbcb1d4ddcbe038bc2ef5e2b92"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:29.214399 882 tablet_service.cc:1552] Processing DeleteTablet for tablet b631e6b3048e428f901af1b5b0b1ab3c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:29.216015 882 tablet_service.cc:1552] Processing DeleteTablet for tablet b30d2054316b49fa8096345527a45d87 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:29.216181 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 605172f066d242f2abea133c08a145ca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:29.216782 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c26c3d0170f64a41ae8944c6668cc5f3. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:29.217504 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c1f789cbcb1d4ddcbe038bc2ef5e2b92 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:29.217559 877 tablet_service.cc:1552] Processing DeleteTablet for tablet e28d66ce4711430cb09a9650e99fbe9e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:29.224341 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71dc8d20d93243e289330d7b8c635511. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.227916 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_43e9cb840d48444b9360e2fa8671dfb8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:29.228449 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_43e9cb840d48444b9360e2fa8671dfb8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:29.229955 595 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_c2f4d6a8642a44b1b5566b7631b598e6 [id=bee5829ed7d44761a9ae57d3134d38dc] alter complete (version 2)
I20251211 20:52:29.230603 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_6412ddf2aa9540e29cf2e6aa2ca8ae47"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:29.231112 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6412ddf2aa9540e29cf2e6aa2ca8ae47 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:29.232707 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3acfef6f082641a199b80eefcfc772ac consensus state has no leader
I20251211 20:52:29.233363 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_33ba9c5d5329493ba57d012d9344420d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:29.233857 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_33ba9c5d5329493ba57d012d9344420d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:29.246170 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_58da4dae54de4c8e9d471de1711d3d6d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:29.246742 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_58da4dae54de4c8e9d471de1711d3d6d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:29.247879 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bc32a778e5424850b23ccaf74bb9df07 consensus state has no leader
W20251211 20:52:29.248337 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8b793241ee3b42b1a38e9f2f05e1173b consensus state has no leader
I20251211 20:52:29.248781 1165 log.cc:1199] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/4d98967714184fe6a3e38dd4ca7979dc
I20251211 20:52:29.249327 1165 ts_tablet_manager.cc:1950] T 4d98967714184fe6a3e38dd4ca7979dc P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:29.251737 1164 ts_tablet_manager.cc:1929] T 33e1d8a576cd41b1a34778b1458df396 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251211 20:52:29.251881 1164 log.cc:1199] T 33e1d8a576cd41b1a34778b1458df396 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/33e1d8a576cd41b1a34778b1458df396
I20251211 20:52:29.250293 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 348) took 1605 ms. Trace:
I20251211 20:52:29.252668 1165 rpcz_store.cc:276] 1211 20:52:27.644734 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:27.936686 (+291952us) service_pool.cc:225] Handling call
1211 20:52:29.091318 (+1154632us) ts_tablet_manager.cc:1151] Deleting tablet 4d98967714184fe6a3e38dd4ca7979dc
1211 20:52:29.091327 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:29.120024 (+ 28697us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:29.170735 (+ 50711us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:29.250252 (+ 79517us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1154431}
I20251211 20:52:29.253283 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 4d98967714184fe6a3e38dd4ca7979dc (table default.Table_ebec874d437748c487f8bf9d0ccaa7fa [id=aea22a96d5b840399f2140486953c1ed]) successfully deleted
I20251211 20:52:29.253796 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 349) took 1571 ms. Trace:
I20251211 20:52:29.253945 1165 rpcz_store.cc:276] 1211 20:52:27.682630 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:27.936917 (+254287us) service_pool.cc:225] Handling call
1211 20:52:29.253624 (+1316707us) ts_tablet_manager.cc:1151] Deleting tablet 88066d94d5d34e0fab20da9fb6ef178e
1211 20:52:29.253636 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:29.253764 (+ 128us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1316530}
W20251211 20:52:29.254423 582 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.104.61:40749 (error 108) [suppressed 8 similar messages]
I20251211 20:52:29.255126 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 350) took 1568 ms. Trace:
I20251211 20:52:29.257853 1165 rpcz_store.cc:276] 1211 20:52:27.688691 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:27.937107 (+248416us) service_pool.cc:225] Handling call
1211 20:52:29.254983 (+1317876us) ts_tablet_manager.cc:1151] Deleting tablet bb112b6431af42a6a7b328dfdefbc2a2
1211 20:52:29.254994 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:29.255090 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1316968}
W20251211 20:52:29.258407 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet bb112b6431af42a6a7b328dfdefbc2a2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb112b6431af42a6a7b328dfdefbc2a2 already in progress: creating tablet
I20251211 20:52:29.258869 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 351) took 1569 ms. Trace:
I20251211 20:52:29.263242 1165 rpcz_store.cc:276] 1211 20:52:27.691227 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:27.938040 (+246813us) service_pool.cc:225] Handling call
1211 20:52:29.258714 (+1320674us) ts_tablet_manager.cc:1151] Deleting tablet e6ba12f94f5f4d688f4222a4c2fb5436
1211 20:52:29.258724 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:29.258837 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1320457}
W20251211 20:52:29.263859 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet e6ba12f94f5f4d688f4222a4c2fb5436 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e6ba12f94f5f4d688f4222a4c2fb5436 already in progress: creating tablet
I20251211 20:52:29.264384 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 353) took 1489 ms. Trace:
I20251211 20:52:29.264544 1165 rpcz_store.cc:276] 1211 20:52:27.774585 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:27.939342 (+164757us) service_pool.cc:225] Handling call
1211 20:52:29.264244 (+1324902us) ts_tablet_manager.cc:1151] Deleting tablet f3855ffb51a74101b26fd28429e65bc2
1211 20:52:29.264255 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:29.264352 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1268897}
W20251211 20:52:29.265012 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet f3855ffb51a74101b26fd28429e65bc2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f3855ffb51a74101b26fd28429e65bc2 already in progress: creating tablet
I20251211 20:52:29.265352 1165 tablet_replica.cc:333] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:29.266898 1165 raft_consensus.cc:2243] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:29.268589 1165 raft_consensus.cc:2272] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:29.268960 1164 ts_tablet_manager.cc:1950] T 33e1d8a576cd41b1a34778b1458df396 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:29.269301 1165 ts_tablet_manager.cc:1916] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:29.273970 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 4d98967714184fe6a3e38dd4ca7979dc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:29.278167 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 71dc8d20d93243e289330d7b8c635511 (DEFAULT_TABLE table=default.Table_fac9e7762d1a492f87a0e5b18868f2ea [id=38840bab0bbf4cc49b74c2e13614069c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.279871 876 tablet_service.cc:1552] Processing DeleteTablet for tablet 8dff13e22899452c8d4f2731b8dac7d5 with delete_type TABLET_DATA_DELETED (replaced by tablet dc5de12701714fcd81b9dba6973a28b5) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:29.281388 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 33e1d8a576cd41b1a34778b1458df396 (table default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]) successfully deleted
W20251211 20:52:29.281663 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet c85624ad640041c29bb87416e1992a9e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c85624ad640041c29bb87416e1992a9e already in progress: creating tablet
W20251211 20:52:29.282040 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 3cd7a465249246c28f07b8fb2e42675f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3cd7a465249246c28f07b8fb2e42675f already in progress: creating tablet
W20251211 20:52:29.282274 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet fb2d30147cd74d1893783be1e1694d1f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fb2d30147cd74d1893783be1e1694d1f already in progress: creating tablet
W20251211 20:52:29.282502 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 4d2b7e2a0abe4592bc806d5ef3f894d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4d2b7e2a0abe4592bc806d5ef3f894d8 already in progress: creating tablet
W20251211 20:52:29.282737 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce already in progress: creating tablet
W20251211 20:52:29.282986 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 68be21a25c6743f5b86d518d98fbc35e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 68be21a25c6743f5b86d518d98fbc35e already in progress: creating tablet
W20251211 20:52:29.283200 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet d5f0ba3ab16240f9822be69f941f5da2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d5f0ba3ab16240f9822be69f941f5da2 already in progress: creating tablet
W20251211 20:52:29.283439 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 52ef1707f6f2425aafd454f303af0333 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 52ef1707f6f2425aafd454f303af0333 already in progress: creating tablet
W20251211 20:52:29.283670 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet b631e6b3048e428f901af1b5b0b1ab3c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b631e6b3048e428f901af1b5b0b1ab3c already in progress: creating tablet
W20251211 20:52:29.283923 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet b30d2054316b49fa8096345527a45d87 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b30d2054316b49fa8096345527a45d87 already in progress: creating tablet
W20251211 20:52:29.284134 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 605172f066d242f2abea133c08a145ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 605172f066d242f2abea133c08a145ca already in progress: creating tablet
W20251211 20:52:29.284353 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet e28d66ce4711430cb09a9650e99fbe9e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e28d66ce4711430cb09a9650e99fbe9e already in progress: creating tablet
W20251211 20:52:29.284606 582 catalog_manager.cc:4952] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 4d98967714184fe6a3e38dd4ca7979dc because the tablet was not found. No further retry: Not found: Tablet not found: 4d98967714184fe6a3e38dd4ca7979dc
W20251211 20:52:29.284974 582 consensus_peers.cc:597] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 -> Peer 11f5f94732f049d0be4c97cebfffcb05 (127.0.104.61:40749): Couldn't send request to peer 11f5f94732f049d0be4c97cebfffcb05. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.61:40749: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:29.284950 564 master_runner.cc:186] Error getting master registration for 127.0.104.61:40749: Network error: Client connection negotiation failed: client connection to 127.0.104.61:40749: connect: Connection refused (error 111),
I20251211 20:52:29.285176 564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20251211 20:52:29.285344 1164 tablet_replica.cc:333] T 8dff13e22899452c8d4f2731b8dac7d5 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:29.285625 1164 raft_consensus.cc:2243] T 8dff13e22899452c8d4f2731b8dac7d5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:29.285842 1164 raft_consensus.cc:2272] T 8dff13e22899452c8d4f2731b8dac7d5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:29.290237 1166 ts_tablet_manager.cc:1434] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.095s user 0.003s sys 0.000s
I20251211 20:52:29.292001 1165 ts_tablet_manager.cc:1929] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:29.292280 1165 log.cc:1199] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/c5b164dc0ce1461fae98d950f7c7a726
I20251211 20:52:29.293085 1165 ts_tablet_manager.cc:1950] T c5b164dc0ce1461fae98d950f7c7a726 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:29.294296 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 357) took 1367 ms. Trace:
I20251211 20:52:29.296443 1164 ts_tablet_manager.cc:1916] T 8dff13e22899452c8d4f2731b8dac7d5 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:29.181200 757 tablet_service.cc:1505] Processing CreateTablet for tablet 46cb1d16a0e94b3a833f32c635213fd2 (DEFAULT_TABLE table=default.Table_737b2e537604423ab32fd1cc85baab78 [id=1bdaf5ce0f6b4b99a038f9f9efb828f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.297706 754 tablet_service.cc:1505] Processing CreateTablet for tablet d43210ec61af423ebde624397dea9f2e (DEFAULT_TABLE table=default.Table_df56cd18522b410fade2a880c087eff4 [id=7c5b2e33fa1c4b12b3941577360908a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.297963 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46cb1d16a0e94b3a833f32c635213fd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.298062 738 tablet_service.cc:1505] Processing CreateTablet for tablet d69fb1f6d9594afcbfb844860b41a377 (DEFAULT_TABLE table=default.Table_2edfd010f1ef40e8be655908fdc4800e [id=b24d18520fdd40e8bb721899ce7ac734]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.298401 738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d69fb1f6d9594afcbfb844860b41a377. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.300017 592 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_595d8c100e69422ab5e6d44cce9e733c [id=28f3ebe9b7a64b5e89404b0407b0e5ae] alter complete (version 2)
I20251211 20:52:29.182382 755 tablet_service.cc:1505] Processing CreateTablet for tablet 7452dd736b0440fcb4626858e794676f (DEFAULT_TABLE table=default.Table_3542335cc1af47b3817527896d07f03a [id=d1e2bcd20b06470bb87adf24e7b8e7fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.301422 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7452dd736b0440fcb4626858e794676f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.301589 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c04187b8ed146c289a6b17d62b07199 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:29.301714 883 tablet_service.cc:1505] Processing CreateTablet for tablet d69fb1f6d9594afcbfb844860b41a377 (DEFAULT_TABLE table=default.Table_2edfd010f1ef40e8be655908fdc4800e [id=b24d18520fdd40e8bb721899ce7ac734]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.301824 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d43210ec61af423ebde624397dea9f2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.186023 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 52ef1707f6f2425aafd454f303af0333 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:29.191890 746 tablet_service.cc:1552] Processing DeleteTablet for tablet e28d66ce4711430cb09a9650e99fbe9e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:29.305029 892 tablet_service.cc:1505] Processing CreateTablet for tablet 47779034ad89466e9e1801987128b4dd (DEFAULT_TABLE table=default.Table_441ecee3b8204f9fa8d22146340e5303 [id=a5b6a2883bda49738a07b6daea5e7310]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.305493 876 tablet_service.cc:1505] Processing CreateTablet for tablet d43210ec61af423ebde624397dea9f2e (DEFAULT_TABLE table=default.Table_df56cd18522b410fade2a880c087eff4 [id=7c5b2e33fa1c4b12b3941577360908a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.307868 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d69fb1f6d9594afcbfb844860b41a377. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.197961 753 tablet_service.cc:1552] Processing DeleteTablet for tablet b631e6b3048e428f901af1b5b0b1ab3c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:29.197822 743 tablet_service.cc:1552] Processing DeleteTablet for tablet 605172f066d242f2abea133c08a145ca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:29.198328 711 leader_election.cc:343] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:29.309468 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47779034ad89466e9e1801987128b4dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.309549 711 leader_election.cc:304] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:29.309865 711 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 111553us
W20251211 20:52:29.310266 711 connection.cc:537] client connection to 127.0.104.61:40749 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251211 20:52:29.310585 711 leader_election.cc:343] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:29.310709 711 leader_election.cc:304] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:29.311064 1585 raft_consensus.cc:2749] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:29.311239 1585 raft_consensus.cc:2749] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:29.311388 588 master_service.cc:954] ReplaceTablet: received request to replace tablet 33addf3d158941098f240cfcdec45d28 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:29.312880 876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d43210ec61af423ebde624397dea9f2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.317296 890 tablet_service.cc:1505] Processing CreateTablet for tablet f25bde14792248b188660036f07ae902 (DEFAULT_TABLE table=default.Table_05f224658edf4b369163c847787e14bf [id=0e372ea5c45443a896aca76f52b48a5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.317755 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f25bde14792248b188660036f07ae902. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.319155 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_7c081945e59345a7830dedbfd3f8f3ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:29.320156 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7c081945e59345a7830dedbfd3f8f3ee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:29.322407 879 tablet_service.cc:1505] Processing CreateTablet for tablet e1aeae8c18b6499e9dc1accabdc3f633 (DEFAULT_TABLE table=default.Table_58da4dae54de4c8e9d471de1711d3d6d [id=d67eb639ab9b456195ce69ac993349ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.322826 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1aeae8c18b6499e9dc1accabdc3f633. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.322907 743 tablet_service.cc:1505] Processing CreateTablet for tablet e1aeae8c18b6499e9dc1accabdc3f633 (DEFAULT_TABLE table=default.Table_58da4dae54de4c8e9d471de1711d3d6d [id=d67eb639ab9b456195ce69ac993349ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.323312 743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1aeae8c18b6499e9dc1accabdc3f633. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.324272 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 33addf3d158941098f240cfcdec45d28
I20251211 20:52:29.324859 588 catalog_manager.cc:6724] ReplaceTablet: tablet 33addf3d158941098f240cfcdec45d28 deleted and replaced by tablet 7be44df39a8f4d6da793479d425fa684
I20251211 20:52:29.325387 746 tablet_service.cc:1505] Processing CreateTablet for tablet 41bf5b5111f64dfa91ed76ed4ab4f6f7 (DEFAULT_TABLE table=default.Table_6814df0b3dd640f58f421a6ed9010686 [id=aa4a2053a7cf478884770f8e36a02fc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.325745 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41bf5b5111f64dfa91ed76ed4ab4f6f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.327219 891 tablet_service.cc:1505] Processing CreateTablet for tablet 373d3f03d7e0408cbe06955f8f60b16c (DEFAULT_TABLE table=default.Table_43e9cb840d48444b9360e2fa8671dfb8 [id=f6cab1da32324fe6afc1df2007f0bdbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.327610 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 373d3f03d7e0408cbe06955f8f60b16c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.327771 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 33addf3d158941098f240cfcdec45d28 with delete_type TABLET_DATA_DELETED (replaced by tablet 7be44df39a8f4d6da793479d425fa684) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:29.328572 877 tablet_service.cc:1505] Processing CreateTablet for tablet d391d4e6a101462d8d1220f1e17d0daa (DEFAULT_TABLE table=default.Table_c1f789cbcb1d4ddcbe038bc2ef5e2b92 [id=e154f2c3021f400c80ba23bcd269883f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.328939 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d391d4e6a101462d8d1220f1e17d0daa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.331230 893 tablet_service.cc:1505] Processing CreateTablet for tablet 41bf5b5111f64dfa91ed76ed4ab4f6f7 (DEFAULT_TABLE table=default.Table_6814df0b3dd640f58f421a6ed9010686 [id=aa4a2053a7cf478884770f8e36a02fc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.331301 741 tablet_service.cc:1505] Processing CreateTablet for tablet d391d4e6a101462d8d1220f1e17d0daa (DEFAULT_TABLE table=default.Table_c1f789cbcb1d4ddcbe038bc2ef5e2b92 [id=e154f2c3021f400c80ba23bcd269883f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.331645 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41bf5b5111f64dfa91ed76ed4ab4f6f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.331705 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d391d4e6a101462d8d1220f1e17d0daa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.334371 1166 tablet_bootstrap.cc:492] T 27a08a64d92342d08ab4406e8006a97b P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:29.335608 882 tablet_service.cc:1505] Processing CreateTablet for tablet 44e2d2bdecf247baaac26e06bccc94de (DEFAULT_TABLE table=default.Table_6a2c9aead0a640b79bb4c9d6f2258e67 [id=bde705b86af14b11be4a2e1fc236b37a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.336151 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 33addf3d158941098f240cfcdec45d28 with delete_type TABLET_DATA_DELETED (replaced by tablet 7be44df39a8f4d6da793479d425fa684) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:29.336480 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44e2d2bdecf247baaac26e06bccc94de. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.343477 592 master_service.cc:954] ReplaceTablet: received request to replace tablet 793f543d6b614ad7b3ab8e717db2ceaf from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:29.343861 880 tablet_service.cc:1505] Processing CreateTablet for tablet 18b977f463d5420186c007b38110e07e (DEFAULT_TABLE table=default.Table_37520ef8d4194d8aa3eadef57196a291 [id=a7e8389b5a9a4b01b7a6c327fd2022b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.344269 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18b977f463d5420186c007b38110e07e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.344738 752 tablet_service.cc:1505] Processing CreateTablet for tablet 18b977f463d5420186c007b38110e07e (DEFAULT_TABLE table=default.Table_37520ef8d4194d8aa3eadef57196a291 [id=a7e8389b5a9a4b01b7a6c327fd2022b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.345134 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18b977f463d5420186c007b38110e07e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.347859 748 tablet_service.cc:1505] Processing CreateTablet for tablet ce1799e6604f4ed7875ddedecab6f327 (DEFAULT_TABLE table=default.Table_6412ddf2aa9540e29cf2e6aa2ca8ae47 [id=a6849726434d41d29dce12f47066a685]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.348223 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce1799e6604f4ed7875ddedecab6f327. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.349617 878 tablet_service.cc:1505] Processing CreateTablet for tablet ce1799e6604f4ed7875ddedecab6f327 (DEFAULT_TABLE table=default.Table_6412ddf2aa9540e29cf2e6aa2ca8ae47 [id=a6849726434d41d29dce12f47066a685]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.350195 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce1799e6604f4ed7875ddedecab6f327. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.351331 884 tablet_service.cc:1505] Processing CreateTablet for tablet 92efda2bd0c84d07a0af6b537c0cd882 (DEFAULT_TABLE table=default.Table_ec893136f41e41f6a6db70f193b8065e [id=c9f87468f622412a8fcd50d311f99117]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:29.351600 582 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet bc32a778e5424850b23ccaf74bb9df07 (table default.Table_8c6f9204fa234ae7a9bc18811d75af5d [id=db38368266194271a02ad265fd991439], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet bc32a778e5424850b23ccaf74bb9df07 consensus state has no leader
I20251211 20:52:29.351804 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92efda2bd0c84d07a0af6b537c0cd882. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.351863 744 tablet_service.cc:1505] Processing CreateTablet for tablet 92efda2bd0c84d07a0af6b537c0cd882 (DEFAULT_TABLE table=default.Table_ec893136f41e41f6a6db70f193b8065e [id=c9f87468f622412a8fcd50d311f99117]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.174286 1168 raft_consensus.cc:359] T f00f7c1f6a4e44ff89341b4a55cc93fb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.215358 745 tablet_service.cc:1552] Processing DeleteTablet for tablet b30d2054316b49fa8096345527a45d87 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:29.297521 739 tablet_service.cc:1505] Processing CreateTablet for tablet f25bde14792248b188660036f07ae902 (DEFAULT_TABLE table=default.Table_05f224658edf4b369163c847787e14bf [id=0e372ea5c45443a896aca76f52b48a5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.301355 749 tablet_service.cc:1505] Processing CreateTablet for tablet 47779034ad89466e9e1801987128b4dd (DEFAULT_TABLE table=default.Table_441ecee3b8204f9fa8d22146340e5303 [id=a5b6a2883bda49738a07b6daea5e7310]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.352208 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92efda2bd0c84d07a0af6b537c0cd882. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.353158 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f25bde14792248b188660036f07ae902. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.356251 756 tablet_service.cc:1505] Processing CreateTablet for tablet 44e2d2bdecf247baaac26e06bccc94de (DEFAULT_TABLE table=default.Table_6a2c9aead0a640b79bb4c9d6f2258e67 [id=bde705b86af14b11be4a2e1fc236b37a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.356706 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44e2d2bdecf247baaac26e06bccc94de. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.359724 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47779034ad89466e9e1801987128b4dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.360785 740 tablet_service.cc:1505] Processing CreateTablet for tablet 373d3f03d7e0408cbe06955f8f60b16c (DEFAULT_TABLE table=default.Table_43e9cb840d48444b9360e2fa8671dfb8 [id=f6cab1da32324fe6afc1df2007f0bdbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.361157 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 373d3f03d7e0408cbe06955f8f60b16c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.370051 1168 raft_consensus.cc:385] T f00f7c1f6a4e44ff89341b4a55cc93fb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:29.371296 1168 raft_consensus.cc:740] T f00f7c1f6a4e44ff89341b4a55cc93fb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:29.371764 1168 consensus_queue.cc:260] T f00f7c1f6a4e44ff89341b4a55cc93fb P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.372004 1160 raft_consensus.cc:359] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:29.376470 1168 ts_tablet_manager.cc:1434] T f00f7c1f6a4e44ff89341b4a55cc93fb P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.215s user 0.002s sys 0.000s
I20251211 20:52:29.379611 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_01868fec1884449da230c4236ab2d506"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:29.380293 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_01868fec1884449da230c4236ab2d506 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:29.198277 747 tablet_service.cc:1505] Processing CreateTablet for tablet 71dc8d20d93243e289330d7b8c635511 (DEFAULT_TABLE table=default.Table_fac9e7762d1a492f87a0e5b18868f2ea [id=38840bab0bbf4cc49b74c2e13614069c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.382655 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71dc8d20d93243e289330d7b8c635511. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.384264 1160 raft_consensus.cc:385] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:29.370563 753 tablet_service.cc:1505] Processing CreateTablet for tablet 5bcc94ec9630484cb7a3e8bf6d4f378a (DEFAULT_TABLE table=default.Table_33ba9c5d5329493ba57d012d9344420d [id=c1d4515be510468fb7d972514e6c87d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.386811 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bcc94ec9630484cb7a3e8bf6d4f378a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.389499 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_43474d2631e0443d9eaecae68a1c110d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:29.390174 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_43474d2631e0443d9eaecae68a1c110d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:29.391752 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_da9a07e5af4a433c8e02a128e8843084" } modify_external_catalogs: true
I20251211 20:52:29.391961 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_da9a07e5af4a433c8e02a128e8843084" } modify_external_catalogs: true
I20251211 20:52:29.393385 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_bf722f720fc64f2c875ab1edbcaebaf9" } modify_external_catalogs: true
I20251211 20:52:29.393545 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_bf722f720fc64f2c875ab1edbcaebaf9" } modify_external_catalogs: true
I20251211 20:52:29.393911 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_491fd0036d9e4f9fa410f3517c2c8286" } modify_external_catalogs: true
I20251211 20:52:29.394060 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_491fd0036d9e4f9fa410f3517c2c8286" } modify_external_catalogs: true
I20251211 20:52:29.198602 750 tablet_service.cc:1505] Processing CreateTablet for tablet c26c3d0170f64a41ae8944c6668cc5f3 (DEFAULT_TABLE table=default.Table_47b57806fd3149c8ab84567d76177fce [id=a1149606f150473ab2d908bf42316081]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.394665 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c26c3d0170f64a41ae8944c6668cc5f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.396909 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ae0290b97d234621bbd1ff141b5c2400" } modify_external_catalogs: true
I20251211 20:52:29.397078 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ae0290b97d234621bbd1ff141b5c2400" } modify_external_catalogs: true
I20251211 20:52:29.399410 892 tablet_service.cc:1505] Processing CreateTablet for tablet 5bcc94ec9630484cb7a3e8bf6d4f378a (DEFAULT_TABLE table=default.Table_33ba9c5d5329493ba57d012d9344420d [id=c1d4515be510468fb7d972514e6c87d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.399699 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71dc8d20d93243e289330d7b8c635511. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.399935 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bcc94ec9630484cb7a3e8bf6d4f378a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.404410 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet b30d2054316b49fa8096345527a45d87 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:29.405596 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 5bcc94ec9630484cb7a3e8bf6d4f378a (DEFAULT_TABLE table=default.Table_33ba9c5d5329493ba57d012d9344420d [id=c1d4515be510468fb7d972514e6c87d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.405787 1014 tablet_service.cc:1505] Processing CreateTablet for tablet d69fb1f6d9594afcbfb844860b41a377 (DEFAULT_TABLE table=default.Table_2edfd010f1ef40e8be655908fdc4800e [id=b24d18520fdd40e8bb721899ce7ac734]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.406070 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 41bf5b5111f64dfa91ed76ed4ab4f6f7 (DEFAULT_TABLE table=default.Table_6814df0b3dd640f58f421a6ed9010686 [id=aa4a2053a7cf478884770f8e36a02fc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.406214 1015 tablet_service.cc:1505] Processing CreateTablet for tablet 47779034ad89466e9e1801987128b4dd (DEFAULT_TABLE table=default.Table_441ecee3b8204f9fa8d22146340e5303 [id=a5b6a2883bda49738a07b6daea5e7310]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.406603 1011 tablet_service.cc:1505] Processing CreateTablet for tablet e1aeae8c18b6499e9dc1accabdc3f633 (DEFAULT_TABLE table=default.Table_58da4dae54de4c8e9d471de1711d3d6d [id=d67eb639ab9b456195ce69ac993349ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.409476 1028 tablet_service.cc:1505] Processing CreateTablet for tablet d391d4e6a101462d8d1220f1e17d0daa (DEFAULT_TABLE table=default.Table_c1f789cbcb1d4ddcbe038bc2ef5e2b92 [id=e154f2c3021f400c80ba23bcd269883f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.411827 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_00bd643f7bb84217970178aefb485a69" } modify_external_catalogs: true
I20251211 20:52:29.412053 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_00bd643f7bb84217970178aefb485a69" } modify_external_catalogs: true
I20251211 20:52:29.409337 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 793f543d6b614ad7b3ab8e717db2ceaf
I20251211 20:52:29.414513 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_85e0192a945545faad36b2be67d76e2a" } new_table_name: "default.Table_1c6adfd0450b41b5be8a7575406a2149" modify_external_catalogs: true
I20251211 20:52:29.415354 592 catalog_manager.cc:6724] ReplaceTablet: tablet 793f543d6b614ad7b3ab8e717db2ceaf deleted and replaced by tablet 06b8f8e46e524545aeab6c931efaf116
I20251211 20:52:29.415745 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 793f543d6b614ad7b3ab8e717db2ceaf with delete_type TABLET_DATA_DELETED (replaced by tablet 06b8f8e46e524545aeab6c931efaf116) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:29.416215 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet ee893e33255745debf73c486d8c70267
I20251211 20:52:29.416847 1165 rpcz_store.cc:276] 1211 20:52:27.927219 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:27.983297 (+ 56078us) service_pool.cc:225] Handling call
1211 20:52:29.265292 (+1281995us) ts_tablet_manager.cc:1151] Deleting tablet c5b164dc0ce1461fae98d950f7c7a726
1211 20:52:29.265301 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:29.281689 (+ 16388us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:29.291836 (+ 10147us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:29.294239 (+ 2403us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1204746}
I20251211 20:52:29.417498 1165 tablet_replica.cc:333] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:29.417750 1165 raft_consensus.cc:2243] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:29.418012 1165 raft_consensus.cc:2272] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:29.418648 1165 ts_tablet_manager.cc:1916] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:29.418503 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet c5b164dc0ce1461fae98d950f7c7a726 (table default.Table_01b77d8f2f0c4c52b733bec1855fba9c [id=9c24845e4efe475fa792a51ce8a66be6]) successfully deleted
I20251211 20:52:29.419427 1012 tablet_service.cc:1505] Processing CreateTablet for tablet f25bde14792248b188660036f07ae902 (DEFAULT_TABLE table=default.Table_05f224658edf4b369163c847787e14bf [id=0e372ea5c45443a896aca76f52b48a5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.420459 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d391d4e6a101462d8d1220f1e17d0daa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.426018 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41bf5b5111f64dfa91ed76ed4ab4f6f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.430896 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_a726aa1257194ddbb21fdf77d7d489d8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:29.431612 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a726aa1257194ddbb21fdf77d7d489d8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:29.431874 1014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d69fb1f6d9594afcbfb844860b41a377. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.400473 1160 raft_consensus.cc:740] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:29.433023 1160 consensus_queue.cc:260] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:29.437211 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 33addf3d158941098f240cfcdec45d28 with delete_type TABLET_DATA_DELETED (replaced by tablet 7be44df39a8f4d6da793479d425fa684) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:29.437831 595 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_5d36b67ed10545eb97ac64602292ca7b" } new_table_name: "default.Table_1c2d691fc52b40dcb94acbe811aefe97" modify_external_catalogs: true
I20251211 20:52:29.438431 1609 tablet.cc:1722] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:29.440475 1021 tablet_service.cc:1505] Processing CreateTablet for tablet 373d3f03d7e0408cbe06955f8f60b16c (DEFAULT_TABLE table=default.Table_43e9cb840d48444b9360e2fa8671dfb8 [id=f6cab1da32324fe6afc1df2007f0bdbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.441675 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bcc94ec9630484cb7a3e8bf6d4f378a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.443106 1011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1aeae8c18b6499e9dc1accabdc3f633. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.452883 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_424b336a2ac84471bac647eab716d19c" } new_table_name: "default.Table_b7bf8c5ad5304175a1f2f6a03887f725" modify_external_catalogs: true
I20251211 20:52:29.453014 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ea25e2f3ec124ff28f04f72f6f7121e1" } new_table_name: "default.Table_468ef643730a494baa1b40f6e81b0a11" modify_external_catalogs: true
I20251211 20:52:29.454449 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_03b10e1dfcd94a08a73fd396ce0b6f98" } new_table_name: "default.Table_59072235d93b41c0b5cd5e3debcb8ddf" modify_external_catalogs: true
I20251211 20:52:29.464322 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47779034ad89466e9e1801987128b4dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.465119 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 793f543d6b614ad7b3ab8e717db2ceaf with delete_type TABLET_DATA_DELETED (replaced by tablet 06b8f8e46e524545aeab6c931efaf116) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:29.465270 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet ee893e33255745debf73c486d8c70267 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:29.465365 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 18b977f463d5420186c007b38110e07e (DEFAULT_TABLE table=default.Table_37520ef8d4194d8aa3eadef57196a291 [id=a7e8389b5a9a4b01b7a6c327fd2022b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.465775 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18b977f463d5420186c007b38110e07e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.467231 1163 tablet_replica.cc:333] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
W20251211 20:52:29.468387 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet b30d2054316b49fa8096345527a45d87 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b30d2054316b49fa8096345527a45d87 already in progress: creating tablet
I20251211 20:52:29.468818 1164 ts_tablet_manager.cc:1929] T 8dff13e22899452c8d4f2731b8dac7d5 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20251211 20:52:29.468891 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 33addf3d158941098f240cfcdec45d28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33addf3d158941098f240cfcdec45d28 already in progress: creating tablet
I20251211 20:52:29.468884 1165 ts_tablet_manager.cc:1929] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:29.469028 1165 log.cc:1199] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/2260850f32b541aab1128818f144b5a7
I20251211 20:52:29.469030 1164 log.cc:1199] T 8dff13e22899452c8d4f2731b8dac7d5 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/8dff13e22899452c8d4f2731b8dac7d5
W20251211 20:52:29.469242 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 793f543d6b614ad7b3ab8e717db2ceaf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 793f543d6b614ad7b3ab8e717db2ceaf already in progress: creating tablet
I20251211 20:52:29.469446 1165 ts_tablet_manager.cc:1950] T 2260850f32b541aab1128818f144b5a7 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:29.469538 1164 ts_tablet_manager.cc:1950] T 8dff13e22899452c8d4f2731b8dac7d5 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:29.470957 1163 raft_consensus.cc:2243] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:29.473985 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 365) took 1506 ms. Trace:
I20251211 20:52:29.474135 1165 rpcz_store.cc:276] 1211 20:52:27.967394 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:28.133381 (+165987us) service_pool.cc:225] Handling call
1211 20:52:29.417404 (+1284023us) ts_tablet_manager.cc:1151] Deleting tablet 2260850f32b541aab1128818f144b5a7
1211 20:52:29.417414 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:29.451005 (+ 33591us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:29.459494 (+ 8489us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:29.473940 (+ 14446us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1210555}
I20251211 20:52:29.474740 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 2260850f32b541aab1128818f144b5a7 (table default.Table_d8c241666f7840b8bf0cb7672482aa55 [id=81bc0640c53f4710895d487f7c6be071]) successfully deleted
I20251211 20:52:29.475346 1166 tablet_bootstrap.cc:654] T 27a08a64d92342d08ab4406e8006a97b P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:29.475597 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 92efda2bd0c84d07a0af6b537c0cd882 (DEFAULT_TABLE table=default.Table_ec893136f41e41f6a6db70f193b8065e [id=c9f87468f622412a8fcd50d311f99117]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.478116 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 366) took 1493 ms. Trace:
I20251211 20:52:29.478467 1165 rpcz_store.cc:276] 1211 20:52:27.984913 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:28.206883 (+221970us) service_pool.cc:225] Handling call
1211 20:52:29.474420 (+1267537us) ts_tablet_manager.cc:1151] Deleting tablet 6a61cf8f5de14adaa4a1de5bddde9a2f
1211 20:52:29.477414 (+ 2994us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x61800000c518. stack: 00007fc66457bee6 00007fc66457bca0 00007fc6635573ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fc66456481d 00007fc66456588e 00007fc6781a5bdd 00007fc678124193 00007fc674352b06 00007fc674353e99 00007fc674387744 00007fc674368240 00007fc674367fcd 0000000000a8970c
1211 20:52:29.477425 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:29.478071 (+ 646us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":6396672,"tablet-delete.queue_time_us":1234363}
I20251211 20:52:29.479275 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 367) took 1454 ms. Trace:
W20251211 20:52:29.479375 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 6a61cf8f5de14adaa4a1de5bddde9a2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a61cf8f5de14adaa4a1de5bddde9a2f already in progress: creating tablet
I20251211 20:52:29.479825 1165 rpcz_store.cc:276] 1211 20:52:28.024357 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:28.240101 (+215744us) service_pool.cc:225] Handling call
1211 20:52:29.479114 (+1239013us) ts_tablet_manager.cc:1151] Deleting tablet e4b5a7553110424d901788c7a2ecdce6
1211 20:52:29.479123 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:29.479246 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1233445}
I20251211 20:52:29.480484 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 368) took 1392 ms. Trace:
W20251211 20:52:29.480556 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet e4b5a7553110424d901788c7a2ecdce6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e4b5a7553110424d901788c7a2ecdce6 already in progress: creating tablet
I20251211 20:52:29.483259 1163 raft_consensus.cc:2272] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:29.483510 1165 rpcz_store.cc:276] 1211 20:52:28.087881 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:28.245706 (+157825us) service_pool.cc:225] Handling call
1211 20:52:29.480357 (+1234651us) ts_tablet_manager.cc:1151] Deleting tablet 25454e67437541d9ac2bfab29bad55f9
1211 20:52:29.480367 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:29.480453 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1222913}
W20251211 20:52:29.483369 711 consensus_peers.cc:597] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 33e1d8a576cd41b1a34778b1458df396. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:29.484390 1165 tablet_replica.cc:333] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:29.484804 1165 raft_consensus.cc:2243] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
W20251211 20:52:29.484824 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 25454e67437541d9ac2bfab29bad55f9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 25454e67437541d9ac2bfab29bad55f9 already in progress: creating tablet
I20251211 20:52:29.485026 1025 tablet_service.cc:1505] Processing CreateTablet for tablet ce1799e6604f4ed7875ddedecab6f327 (DEFAULT_TABLE table=default.Table_6412ddf2aa9540e29cf2e6aa2ca8ae47 [id=a6849726434d41d29dce12f47066a685]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.485373 1012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f25bde14792248b188660036f07ae902. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.485381 1165 raft_consensus.cc:2272] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:29.486490 1165 ts_tablet_manager.cc:1916] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:29.499962 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92efda2bd0c84d07a0af6b537c0cd882. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.501569 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 44e2d2bdecf247baaac26e06bccc94de (DEFAULT_TABLE table=default.Table_6a2c9aead0a640b79bb4c9d6f2258e67 [id=bde705b86af14b11be4a2e1fc236b37a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.503455 1163 ts_tablet_manager.cc:1916] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:29.503623 1165 ts_tablet_manager.cc:1929] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:29.503764 1165 log.cc:1199] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/5fc8d485cc234fd490c487a8bc881bf4
I20251211 20:52:29.504101 1165 ts_tablet_manager.cc:1950] T 5fc8d485cc234fd490c487a8bc881bf4 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:29.505893 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 373d3f03d7e0408cbe06955f8f60b16c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.506908 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 371) took 1142 ms. Trace:
I20251211 20:52:29.507958 1165 rpcz_store.cc:276] 1211 20:52:28.364191 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:28.364373 (+ 182us) service_pool.cc:225] Handling call
1211 20:52:29.484168 (+1119795us) ts_tablet_manager.cc:1151] Deleting tablet 5fc8d485cc234fd490c487a8bc881bf4
1211 20:52:29.484327 (+ 159us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:29.490707 (+ 6380us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:29.503463 (+ 12756us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:29.506851 (+ 3388us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1117893}
I20251211 20:52:29.508694 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 5fc8d485cc234fd490c487a8bc881bf4 (table default.Table_88dcd36b4447481487bc918a05511862 [id=d9453cea23a8430c98a3317856b3cec3]) successfully deleted
W20251211 20:52:29.509598 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet c7147571483e41ad970e573a08c368ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c7147571483e41ad970e573a08c368ca already in progress: creating tablet
W20251211 20:52:29.510526 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 28ddb8a59a214576a25d613d42c2fef5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 28ddb8a59a214576a25d613d42c2fef5 already in progress: creating tablet
I20251211 20:52:29.511034 1165 tablet_replica.cc:333] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:29.511448 1165 raft_consensus.cc:2243] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:29.512072 1165 raft_consensus.cc:2272] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:29.513859 1165 ts_tablet_manager.cc:1916] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:29.513542 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce1799e6604f4ed7875ddedecab6f327. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.525799 1165 ts_tablet_manager.cc:1929] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251211 20:52:29.527751 1165 log.cc:1199] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/33e1d8a576cd41b1a34778b1458df396
I20251211 20:52:29.528349 1165 ts_tablet_manager.cc:1950] T 33e1d8a576cd41b1a34778b1458df396 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:29.529901 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 33e1d8a576cd41b1a34778b1458df396 (table default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]) successfully deleted
W20251211 20:52:29.531733 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 4ce495233a5e4f3792faf44680df54ba because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ce495233a5e4f3792faf44680df54ba already in progress: creating tablet
W20251211 20:52:29.532756 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 7ec261837cf04bf88a48ff8add1b0d66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7ec261837cf04bf88a48ff8add1b0d66 already in progress: creating tablet
W20251211 20:52:29.533704 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 4201da4241aa48dfb670a713e4453ca5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4201da4241aa48dfb670a713e4453ca5 already in progress: creating tablet
W20251211 20:52:29.534590 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 3cd7a465249246c28f07b8fb2e42675f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3cd7a465249246c28f07b8fb2e42675f already in progress: creating tablet
W20251211 20:52:29.535310 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet c85624ad640041c29bb87416e1992a9e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c85624ad640041c29bb87416e1992a9e already in progress: creating tablet
W20251211 20:52:29.537843 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet d5f0ba3ab16240f9822be69f941f5da2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d5f0ba3ab16240f9822be69f941f5da2 already in progress: creating tablet
W20251211 20:52:29.538661 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 68be21a25c6743f5b86d518d98fbc35e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 68be21a25c6743f5b86d518d98fbc35e already in progress: creating tablet
W20251211 20:52:29.539459 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce already in progress: creating tablet
W20251211 20:52:29.540266 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet fb2d30147cd74d1893783be1e1694d1f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fb2d30147cd74d1893783be1e1694d1f already in progress: creating tablet
W20251211 20:52:29.541024 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 4d2b7e2a0abe4592bc806d5ef3f894d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4d2b7e2a0abe4592bc806d5ef3f894d8 already in progress: creating tablet
I20251211 20:52:29.541550 1165 tablet_replica.cc:333] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:29.542047 1165 raft_consensus.cc:2243] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:29.542582 1165 raft_consensus.cc:2272] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:29.543414 1165 ts_tablet_manager.cc:1916] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:29.552196 750 tablet_service.cc:1505] Processing CreateTablet for tablet 95866de91f9d437d80f40b7962253610 (DEFAULT_TABLE table=default.Table_7c081945e59345a7830dedbfd3f8f3ee [id=ec0bb10933fa4529bef3580bee6409e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.552618 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95866de91f9d437d80f40b7962253610. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.552582 912 raft_consensus.cc:1240] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [term 2 LEADER]: Rejecting Update request from peer 6f8d784fd1cd492bbe3809403ec34919 for earlier term 1. Current term is 2. Ops: []
I20251211 20:52:29.553079 740 tablet_service.cc:1505] Processing CreateTablet for tablet 7be44df39a8f4d6da793479d425fa684 (DEFAULT_TABLE table=default.Table_f6a10556ff104db5bd5ecc9fc380e228 [id=f75813add54e46189ff1bbed82d83ba7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.553434 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7be44df39a8f4d6da793479d425fa684. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.555735 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 33e1d8a576cd41b1a34778b1458df396 with delete_type TABLET_DATA_DELETED (replaced by tablet 6b7dd8232d974a43b99ec9b28075dc8d) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:29.556391 1574 consensus_queue.cc:1059] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:29.556617 1574 consensus_queue.cc:1243] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Peer eb0f1933d9a74fccaa4618047fc05c41 log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20251211 20:52:29.556766 1574 raft_consensus.cc:3055] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Stepping down as leader of term 1
I20251211 20:52:29.556841 1574 raft_consensus.cc:740] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:29.557133 1574 consensus_queue.cc:260] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:29.557432 1574 raft_consensus.cc:3060] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Advancing to term 2
I20251211 20:52:29.560565 589 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 5fd2a6cf80a04f20aef8ff9c84bc88db (table default.Table_370d3069ceee4b8e8856bd74fe3a1815 [id=10fd0368198c447c95b692b1f3e6a023]). Expected version 2 got 0
I20251211 20:52:29.561023 589 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3eac21a07ba34e1fa275e25ee9dcabb1 (table default.Table_fbe890176f3d42768b00d57c928f38c4 [id=9700f9fe6e3044748e34954bef8a0038]). Expected version 1 got 0
I20251211 20:52:29.565304 1024 tablet_service.cc:1505] Processing CreateTablet for tablet d43210ec61af423ebde624397dea9f2e (DEFAULT_TABLE table=default.Table_df56cd18522b410fade2a880c087eff4 [id=7c5b2e33fa1c4b12b3941577360908a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.566839 1165 ts_tablet_manager.cc:1929] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:29.567992 741 tablet_service.cc:1552] Processing DeleteTablet for tablet ee893e33255745debf73c486d8c70267 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:29.570897 592 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5453f68789fd4e89b69e98860050865b consensus state has no leader
I20251211 20:52:29.570935 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 68792ccef54944a09af9b5456e9ff1fe
I20251211 20:52:29.571354 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_a3a395764457481b9b9082279213a9ef"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:29.571656 1165 log.cc:1199] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/1c04187b8ed146c289a6b17d62b07199
W20251211 20:52:29.571988 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a3a395764457481b9b9082279213a9ef in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:29.572253 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 68792ccef54944a09af9b5456e9ff1fe with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:29.572821 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_3bccb9c2f8e04bc3aff742806d2b634a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:29.572950 1165 ts_tablet_manager.cc:1950] T 1c04187b8ed146c289a6b17d62b07199 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
W20251211 20:52:29.573287 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c7a2610f20df4c1ca81eae2fc7dba6bb consensus state has no leader
W20251211 20:52:29.573711 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3bccb9c2f8e04bc3aff742806d2b634a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:29.573913 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8229b704e7464b30b7d2902a938ce33b consensus state has no leader
I20251211 20:52:29.574242 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_d11ab1dc4ef24e4aa5a57ac9876f5eb9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:29.574836 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d11ab1dc4ef24e4aa5a57ac9876f5eb9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:29.575299 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 5710fc2f24764f818f7e5c86c6fd985e
I20251211 20:52:29.576028 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_608d54a047d5494d8ea7e335dd734192"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:29.576490 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_608d54a047d5494d8ea7e335dd734192 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:29.576942 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_8351c303d217491e889c60fba86aebba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:29.577183 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 5710fc2f24764f818f7e5c86c6fd985e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:29.577498 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8351c303d217491e889c60fba86aebba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:29.577963 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_dd85571b35b14e7b8fa57c09d6bfa1b1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:29.578471 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dd85571b35b14e7b8fa57c09d6bfa1b1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:29.579785 595 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8308ad46551447439a248d20ed1e5922 consensus state has no leader
I20251211 20:52:29.580286 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_1021d997007b41208355c31d22f4e26c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:29.580673 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1021d997007b41208355c31d22f4e26c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:29.581622 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 1c04187b8ed146c289a6b17d62b07199 (table default.Table_2bdd381fdf6443da869b647ddb92699b [id=70ed017f68a140ceaa39557fbe86aa0b]) successfully deleted
W20251211 20:52:29.583916 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 52ef1707f6f2425aafd454f303af0333 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 52ef1707f6f2425aafd454f303af0333 already in progress: creating tablet
W20251211 20:52:29.585067 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet e28d66ce4711430cb09a9650e99fbe9e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e28d66ce4711430cb09a9650e99fbe9e already in progress: creating tablet
W20251211 20:52:29.585767 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet b631e6b3048e428f901af1b5b0b1ab3c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b631e6b3048e428f901af1b5b0b1ab3c already in progress: creating tablet
W20251211 20:52:29.586500 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 605172f066d242f2abea133c08a145ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 605172f066d242f2abea133c08a145ca already in progress: creating tablet
W20251211 20:52:29.587278 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 33addf3d158941098f240cfcdec45d28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33addf3d158941098f240cfcdec45d28 already in progress: creating tablet
I20251211 20:52:29.587775 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 5710fc2f24764f818f7e5c86c6fd985e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:29.588078 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet b30d2054316b49fa8096345527a45d87 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b30d2054316b49fa8096345527a45d87 already in progress: creating tablet
I20251211 20:52:29.588248 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 88066d94d5d34e0fab20da9fb6ef178e with delete_type TABLET_DATA_DELETED (replaced by tablet 6a61cf8f5de14adaa4a1de5bddde9a2f) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:29.588940 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 793f543d6b614ad7b3ab8e717db2ceaf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 793f543d6b614ad7b3ab8e717db2ceaf already in progress: creating tablet
I20251211 20:52:29.589391 1028 tablet_service.cc:1505] Processing CreateTablet for tablet 7be44df39a8f4d6da793479d425fa684 (DEFAULT_TABLE table=default.Table_f6a10556ff104db5bd5ecc9fc380e228 [id=f75813add54e46189ff1bbed82d83ba7]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:29.589704 581 catalog_manager.cc:4952] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 33e1d8a576cd41b1a34778b1458df396 because the tablet was not found. No further retry: Not found: Tablet not found: 33e1d8a576cd41b1a34778b1458df396
I20251211 20:52:29.590018 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet c4b76a7e84d344268567c551236ce772 with delete_type TABLET_DATA_DELETED (replaced by tablet 68be21a25c6743f5b86d518d98fbc35e) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:29.590413 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 19d0aa259ac94f36b7f6a24d11a3d420 with delete_type TABLET_DATA_DELETED (replaced by tablet b9e5ad8030a44e03acfe73124b156ac8) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:29.590473 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet ee893e33255745debf73c486d8c70267 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ee893e33255745debf73c486d8c70267 already in progress: creating tablet
W20251211 20:52:29.591105 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 68792ccef54944a09af9b5456e9ff1fe because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 68792ccef54944a09af9b5456e9ff1fe already in progress: creating tablet
W20251211 20:52:29.591857 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 5710fc2f24764f818f7e5c86c6fd985e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5710fc2f24764f818f7e5c86c6fd985e already in progress: creating tablet
I20251211 20:52:29.593698 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44e2d2bdecf247baaac26e06bccc94de. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.593871 1014 tablet_service.cc:1505] Processing CreateTablet for tablet 95866de91f9d437d80f40b7962253610 (DEFAULT_TABLE table=default.Table_7c081945e59345a7830dedbfd3f8f3ee [id=ec0bb10933fa4529bef3580bee6409e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.602139 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 4f807596fa6a4e66b89566b5d5dd1080
I20251211 20:52:29.603796 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 4f807596fa6a4e66b89566b5d5dd1080 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:29.604238 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 0aac6fb3972e4aba8950828c9e49d507
I20251211 20:52:29.605206 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 0aac6fb3972e4aba8950828c9e49d507 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:29.605798 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_3357d689c6fa4890bcaa97d99990c9fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:29.605890 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 4f807596fa6a4e66b89566b5d5dd1080 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4f807596fa6a4e66b89566b5d5dd1080 already in progress: creating tablet
W20251211 20:52:29.606534 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3357d689c6fa4890bcaa97d99990c9fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:29.606673 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 0aac6fb3972e4aba8950828c9e49d507 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0aac6fb3972e4aba8950828c9e49d507 already in progress: creating tablet
I20251211 20:52:29.608860 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet 68792ccef54944a09af9b5456e9ff1fe with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:29.613497 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d43210ec61af423ebde624397dea9f2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.615919 1011 tablet_service.cc:1552] Processing DeleteTablet for tablet 4f807596fa6a4e66b89566b5d5dd1080 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:29.616196 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7be44df39a8f4d6da793479d425fa684. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.620462 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_91453272b5a341baab304eedbc5b2b34" } modify_external_catalogs: true
I20251211 20:52:29.620782 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_91453272b5a341baab304eedbc5b2b34" } modify_external_catalogs: true
I20251211 20:52:29.621673 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_ccd52fe248db4491820429e77b22f465"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:29.622367 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ccd52fe248db4491820429e77b22f465 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:29.625844 1014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95866de91f9d437d80f40b7962253610. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.633040 1586 raft_consensus.cc:493] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:29.633257 1586 raft_consensus.cc:515] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.634693 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet 0aac6fb3972e4aba8950828c9e49d507 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:29.636065 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c4b76a7e84d344268567c551236ce772" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:29.636929 982 leader_election.cc:343] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:29.636946 1586 leader_election.cc:290] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:29.637091 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c4b76a7e84d344268567c551236ce772" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:29.643373 1557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9230f94a6a1a49cead51aeea9f069087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } } }
I20251211 20:52:29.643761 1557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [sys.catalog]: This master's current role is: LEADER
I20251211 20:52:29.381734 738 tablet_service.cc:1552] Processing DeleteTablet for tablet f00f7c1f6a4e44ff89341b4a55cc93fb with delete_type TABLET_DATA_DELETED (replaced by tablet 88066d94d5d34e0fab20da9fb6ef178e) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:29.663077 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet f00f7c1f6a4e44ff89341b4a55cc93fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f00f7c1f6a4e44ff89341b4a55cc93fb already in progress: creating tablet
I20251211 20:52:29.668742 1163 ts_tablet_manager.cc:1929] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20251211 20:52:29.669176 983 leader_election.cc:343] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:29.670053 1163 log.cc:1199] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/ee893e33255745debf73c486d8c70267
I20251211 20:52:29.670462 983 leader_election.cc:304] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:29.670917 1586 raft_consensus.cc:2749] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:29.674482 1168 ts_tablet_manager.cc:1467] T f00f7c1f6a4e44ff89341b4a55cc93fb P 6f8d784fd1cd492bbe3809403ec34919: Tablet startup took 512ms
W20251211 20:52:29.671546 836 kernel_stack_watchdog.cc:198] Thread 847 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 166ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f8ffc641980 (unknown) at ??:0
@ 0x93e827 __sanitizer::internal_sched_yield() at ??:?
@ 0x8a3edd __interceptor_pthread_create at ??:?
@ 0x7f8ff902a0f1 kudu::Thread::StartThread() at ??:0
@ 0xa76197 kudu::Thread::Create() at ??:?
@ 0x7f8ff9049eab kudu::ThreadPool::CreateThread() at ??:0
@ 0x7f8ff90458cd kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7f8ff9044d1f kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7f9004dad3a8 kudu::consensus::Peer::ProcessResponse() at ??:0
@ 0x7f9004dadbd5 kudu::consensus::Peer::ProcessSingleResponse() at ??:0
@ 0x7f9004db5761 kudu::consensus::Peer::SendNextRequest()::$_6::operator()() at ??:0
@ 0x7f9004db556e std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
@ 0x7f8ffff142ef kudu::rpc::Proxy::AsyncRequest()::$_1::operator()() at ??:0
@ 0x7f8ffff13ece std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
I20251211 20:52:29.702620 1586 raft_consensus.cc:493] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:29.702842 1586 raft_consensus.cc:515] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.703562 1586 leader_election.cc:290] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:29.704326 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "19d0aa259ac94f36b7f6a24d11a3d420" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:29.704941 982 leader_election.cc:343] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:29.674753 1168 ts_tablet_manager.cc:1469] T f00f7c1f6a4e44ff89341b4a55cc93fb P 6f8d784fd1cd492bbe3809403ec34919: Trace:
1211 20:52:23.107559 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:23.107634 (+ 75us) service_pool.cc:225] Handling call
1211 20:52:23.108119 (+ 485us) ts_tablet_manager.cc:669] Acquired tablet manager lock
1211 20:52:23.108147 (+ 28us) ts_tablet_manager.cc:682] Creating new metadata...
1211 20:52:23.122992 (+ 14845us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:23.128683 (+ 5691us) tablet_replica.cc:206] Creating consensus instance
1211 20:52:23.128851 (+ 168us) inbound_call.cc:173] Queueing success response
1211 20:52:28.661843 (+5532992us) ts_tablet_manager.cc:1379] Bootstrapping tablet
1211 20:52:29.161407 (+499564us) ts_tablet_manager.cc:1435] Starting tablet replica
1211 20:52:29.161587 (+ 180us) tablet_replica.cc:261] Starting instrumentation
1211 20:52:29.173838 (+ 12251us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":42,"raft.run_cpu_time_us":68,"raft.run_wall_time_us":70,"tablet-open.queue_time_us":5533040}
I20251211 20:52:29.707403 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "19d0aa259ac94f36b7f6a24d11a3d420" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:29.707773 1168 tablet_bootstrap.cc:492] T 8dff13e22899452c8d4f2731b8dac7d5 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:29.712229 738 tablet_service.cc:1552] Processing DeleteTablet for tablet f00f7c1f6a4e44ff89341b4a55cc93fb with delete_type TABLET_DATA_DELETED (replaced by tablet 88066d94d5d34e0fab20da9fb6ef178e) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:29.712505 1165 tablet_replica.cc:333] T f00f7c1f6a4e44ff89341b4a55cc93fb P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:29.712687 1165 raft_consensus.cc:2243] T f00f7c1f6a4e44ff89341b4a55cc93fb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:29.712873 1165 raft_consensus.cc:2272] T f00f7c1f6a4e44ff89341b4a55cc93fb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:29.713384 1165 ts_tablet_manager.cc:1916] T f00f7c1f6a4e44ff89341b4a55cc93fb P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:29.719413 1165 ts_tablet_manager.cc:1929] T f00f7c1f6a4e44ff89341b4a55cc93fb P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:29.719537 1165 log.cc:1199] T f00f7c1f6a4e44ff89341b4a55cc93fb P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/f00f7c1f6a4e44ff89341b4a55cc93fb
I20251211 20:52:29.719910 1165 ts_tablet_manager.cc:1950] T f00f7c1f6a4e44ff89341b4a55cc93fb P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:29.721016 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet f00f7c1f6a4e44ff89341b4a55cc93fb (table default.Table_b06f9587f78645bb9712c3b080f3c0d4 [id=ea12bdaa894f446da22635f95d97ac2a]) successfully deleted
W20251211 20:52:29.729682 983 leader_election.cc:343] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:29.732349 983 leader_election.cc:304] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:29.732739 1586 raft_consensus.cc:2749] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:29.734140 1163 ts_tablet_manager.cc:1950] T ee893e33255745debf73c486d8c70267 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:29.743023 590 master_service.cc:954] ReplaceTablet: received request to replace tablet 4ce495233a5e4f3792faf44680df54ba from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:29.750605 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_c9b4200ff6244586a61f2f90bd35aa83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:29.750762 847 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.1:42177, user_credentials={real_user=slave}} blocked reactor thread for 431112us
W20251211 20:52:29.751252 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c9b4200ff6244586a61f2f90bd35aa83 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:29.752529 776 raft_consensus.cc:1275] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [term 2 FOLLOWER]: Refusing update from remote peer eb0f1933d9a74fccaa4618047fc05c41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20251211 20:52:29.752650 776 pending_rounds.cc:85] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919: Aborting all ops after (but not including) 0
I20251211 20:52:29.752770 776 pending_rounds.cc:107] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20251211 20:52:29.752890 776 raft_consensus.cc:2889] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20251211 20:52:29.762037 1168 tablet_bootstrap.cc:654] T 8dff13e22899452c8d4f2731b8dac7d5 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:29.765272 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 8dff13e22899452c8d4f2731b8dac7d5 (table default.Table_f572100667d148e8a69e7930596f0f52 [id=cb97ccf262fd4c41a180267fd0678cec]) successfully deleted
W20251211 20:52:29.765525 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 33addf3d158941098f240cfcdec45d28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33addf3d158941098f240cfcdec45d28 already in progress: creating tablet
I20251211 20:52:29.765837 741 tablet_service.cc:1505] Processing CreateTablet for tablet 58e35de0792a464a818170886ca8d8c9 (DEFAULT_TABLE table=default.Table_43474d2631e0443d9eaecae68a1c110d [id=e50846ecd18341919707dfa446765ce1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.766271 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58e35de0792a464a818170886ca8d8c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.767284 740 tablet_service.cc:1505] Processing CreateTablet for tablet 06b8f8e46e524545aeab6c931efaf116 (DEFAULT_TABLE table=default.Table_bf0900ef3fb8447c834a43e509caa105 [id=861b00629ceb42588a1d1be34153b792]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.768088 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06b8f8e46e524545aeab6c931efaf116. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.772667 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_01240e3a77b44d43a567846655dbc681" } new_table_name: "default.Table_e157133a993643a586e90be0e591ea6f" modify_external_catalogs: true
I20251211 20:52:29.775823 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_01b77d8f2f0c4c52b733bec1855fba9c" } new_table_name: "default.Table_4de1e61d3b1c4ddebf22cdebca28beac" modify_external_catalogs: true
I20251211 20:52:29.775802 1612 consensus_queue.cc:1048] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20251211 20:52:29.776419 892 tablet_service.cc:1552] Processing DeleteTablet for tablet 793f543d6b614ad7b3ab8e717db2ceaf with delete_type TABLET_DATA_DELETED (replaced by tablet 06b8f8e46e524545aeab6c931efaf116) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:29.778410 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8351c303d217491e889c60fba86aebba" } modify_external_catalogs: true
I20251211 20:52:29.778995 588 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8c6f9204fa234ae7a9bc18811d75af5d" } new_table_name: "default.Table_e6252813479645a98ce2709dc7aba798" modify_external_catalogs: true
I20251211 20:52:29.779421 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a17987402c844d2cbef2310a1375db03" } new_table_name: "default.Table_b43dda4aa0674e3f8ef0d55e0e2431e3" modify_external_catalogs: true
I20251211 20:52:29.779568 1586 raft_consensus.cc:493] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:29.779754 1586 raft_consensus.cc:515] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.781073 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_7a09ac6e8f3e441f83b5738731d01544" } new_table_name: "default.Table_17e2482cdad5493a888dd8bd1ba85a7f" modify_external_catalogs: true
I20251211 20:52:29.784106 892 tablet_service.cc:1552] Processing DeleteTablet for tablet ee893e33255745debf73c486d8c70267 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:29.785218 892 tablet_service.cc:1505] Processing CreateTablet for tablet 95866de91f9d437d80f40b7962253610 (DEFAULT_TABLE table=default.Table_7c081945e59345a7830dedbfd3f8f3ee [id=ec0bb10933fa4529bef3580bee6409e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.785653 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95866de91f9d437d80f40b7962253610. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.787827 890 tablet_service.cc:1505] Processing CreateTablet for tablet 7be44df39a8f4d6da793479d425fa684 (DEFAULT_TABLE table=default.Table_f6a10556ff104db5bd5ecc9fc380e228 [id=f75813add54e46189ff1bbed82d83ba7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.787886 1609 tablet.cc:1722] T 694518e6c1ae4032b06b99e9bde2e5dd P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:29.788213 876 tablet_service.cc:1552] Processing DeleteTablet for tablet 68792ccef54944a09af9b5456e9ff1fe with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:29.788717 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7be44df39a8f4d6da793479d425fa684. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.790535 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8351c303d217491e889c60fba86aebba" } modify_external_catalogs: true
I20251211 20:52:29.791550 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 4f807596fa6a4e66b89566b5d5dd1080 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:29.792274 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 5710fc2f24764f818f7e5c86c6fd985e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:29.795476 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_2edfd010f1ef40e8be655908fdc4800e" } modify_external_catalogs: true
I20251211 20:52:29.795673 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_2edfd010f1ef40e8be655908fdc4800e" } modify_external_catalogs: true
I20251211 20:52:29.796128 1160 ts_tablet_manager.cc:1434] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.572s user 0.028s sys 0.046s
I20251211 20:52:29.798645 1012 tablet_service.cc:1505] Processing CreateTablet for tablet 58e35de0792a464a818170886ca8d8c9 (DEFAULT_TABLE table=default.Table_43474d2631e0443d9eaecae68a1c110d [id=e50846ecd18341919707dfa446765ce1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.800341 738 tablet_service.cc:1505] Processing CreateTablet for tablet 3621735fcb294e3a8c0180ba9ef86442 (DEFAULT_TABLE table=default.Table_01868fec1884449da230c4236ab2d506 [id=d303c5c8fd81400ab473845a942ec140]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.800848 738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3621735fcb294e3a8c0180ba9ef86442. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.805351 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_987db791387d4fa79e285fae8bddce52" } modify_external_catalogs: true
I20251211 20:52:29.805569 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_987db791387d4fa79e285fae8bddce52" } modify_external_catalogs: true
W20251211 20:52:29.806545 1160 ts_tablet_manager.cc:1467] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41: Tablet startup took 582ms
W20251211 20:52:29.807792 1160 ts_tablet_manager.cc:1469] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41: Trace:
1211 20:52:22.996941 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:23.036443 (+ 39502us) service_pool.cc:225] Handling call
1211 20:52:23.045695 (+ 9252us) ts_tablet_manager.cc:669] Acquired tablet manager lock
1211 20:52:23.045724 (+ 29us) ts_tablet_manager.cc:682] Creating new metadata...
1211 20:52:23.125276 (+ 79552us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:23.136517 (+ 11241us) tablet_replica.cc:206] Creating consensus instance
1211 20:52:23.136684 (+ 167us) inbound_call.cc:173] Queueing success response
1211 20:52:28.534893 (+5398209us) ts_tablet_manager.cc:1379] Bootstrapping tablet
1211 20:52:29.215194 (+680301us) ts_tablet_manager.cc:1435] Starting tablet replica
1211 20:52:29.295552 (+ 80358us) tablet_replica.cc:261] Starting instrumentation
1211 20:52:29.295733 (+ 181us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":29813,"raft.run_cpu_time_us":25,"raft.run_wall_time_us":26,"tablet-open.queue_time_us":5398263,"thread_start_us":351535,"threads_started":1}
I20251211 20:52:29.808777 1160 tablet_bootstrap.cc:492] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:29.809260 1626 tablet.cc:1722] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:29.809432 747 tablet_service.cc:1505] Processing CreateTablet for tablet a7979798e92e45eb969197c8c7ac2580 (DEFAULT_TABLE table=default.Table_a726aa1257194ddbb21fdf77d7d489d8 [id=1c86f943adfb4ed8bfbbcacf4f9fe125]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.810817 1160 tablet_bootstrap.cc:654] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:29.812304 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7979798e92e45eb969197c8c7ac2580. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:29.818054 421 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 80628us
I20251211 20:52:29.821503 1168 tablet_bootstrap.cc:492] T 8dff13e22899452c8d4f2731b8dac7d5 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:29.821662 1168 ts_tablet_manager.cc:1403] T 8dff13e22899452c8d4f2731b8dac7d5 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.114s user 0.007s sys 0.000s
I20251211 20:52:29.822438 1168 raft_consensus.cc:359] T 8dff13e22899452c8d4f2731b8dac7d5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.822636 1168 raft_consensus.cc:385] T 8dff13e22899452c8d4f2731b8dac7d5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:29.822726 1168 raft_consensus.cc:740] T 8dff13e22899452c8d4f2731b8dac7d5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:29.822903 1628 tablet.cc:1722] T 694518e6c1ae4032b06b99e9bde2e5dd P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:29.822989 1168 consensus_queue.cc:260] T 8dff13e22899452c8d4f2731b8dac7d5 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.824592 1168 ts_tablet_manager.cc:1434] T 8dff13e22899452c8d4f2731b8dac7d5 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:29.825699 1168 tablet_bootstrap.cc:492] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:29.827987 1168 tablet_bootstrap.cc:654] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:29.831815 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e991174a45694a1fa40bd58fe9dc40e4" } modify_external_catalogs: true
I20251211 20:52:29.832041 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e991174a45694a1fa40bd58fe9dc40e4" } modify_external_catalogs: true
I20251211 20:52:29.835405 890 tablet_service.cc:1552] Processing DeleteTablet for tablet 0aac6fb3972e4aba8950828c9e49d507 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:29.836956 892 tablet_service.cc:1505] Processing CreateTablet for tablet a7979798e92e45eb969197c8c7ac2580 (DEFAULT_TABLE table=default.Table_a726aa1257194ddbb21fdf77d7d489d8 [id=1c86f943adfb4ed8bfbbcacf4f9fe125]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.837425 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7979798e92e45eb969197c8c7ac2580. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.844678 879 tablet_service.cc:1505] Processing CreateTablet for tablet 58e35de0792a464a818170886ca8d8c9 (DEFAULT_TABLE table=default.Table_43474d2631e0443d9eaecae68a1c110d [id=e50846ecd18341919707dfa446765ce1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.845093 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58e35de0792a464a818170886ca8d8c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.851182 890 tablet_service.cc:1505] Processing CreateTablet for tablet 3621735fcb294e3a8c0180ba9ef86442 (DEFAULT_TABLE table=default.Table_01868fec1884449da230c4236ab2d506 [id=d303c5c8fd81400ab473845a942ec140]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.852370 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3621735fcb294e3a8c0180ba9ef86442. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:29.855366 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 793f543d6b614ad7b3ab8e717db2ceaf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 793f543d6b614ad7b3ab8e717db2ceaf already in progress: creating tablet
I20251211 20:52:29.855919 879 tablet_service.cc:1505] Processing CreateTablet for tablet 06b8f8e46e524545aeab6c931efaf116 (DEFAULT_TABLE table=default.Table_bf0900ef3fb8447c834a43e509caa105 [id=861b00629ceb42588a1d1be34153b792]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.856280 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06b8f8e46e524545aeab6c931efaf116. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:29.860071 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet ee893e33255745debf73c486d8c70267 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ee893e33255745debf73c486d8c70267 already in progress: creating tablet
W20251211 20:52:29.863878 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 68792ccef54944a09af9b5456e9ff1fe because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 68792ccef54944a09af9b5456e9ff1fe already in progress: creating tablet
W20251211 20:52:29.864542 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 4f807596fa6a4e66b89566b5d5dd1080 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4f807596fa6a4e66b89566b5d5dd1080 already in progress: creating tablet
W20251211 20:52:29.864984 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 5710fc2f24764f818f7e5c86c6fd985e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5710fc2f24764f818f7e5c86c6fd985e already in progress: creating tablet
W20251211 20:52:29.867839 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 0aac6fb3972e4aba8950828c9e49d507 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0aac6fb3972e4aba8950828c9e49d507 already in progress: creating tablet
I20251211 20:52:29.870453 1609 tablet.cc:1722] T 4870da0224e048ed82f227d7ba676a94 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:29.877384 1168 tablet_bootstrap.cc:492] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:29.877559 1168 ts_tablet_manager.cc:1403] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.052s user 0.005s sys 0.014s
I20251211 20:52:29.878351 1168 raft_consensus.cc:359] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.878546 1168 raft_consensus.cc:385] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:29.878605 1168 raft_consensus.cc:740] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:29.878818 1168 consensus_queue.cc:260] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.880393 1168 ts_tablet_manager.cc:1434] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:29.881582 1168 tablet_bootstrap.cc:492] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:29.881989 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "88066d94d5d34e0fab20da9fb6ef178e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:29.883019 1045 raft_consensus.cc:1275] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7 [term 2 FOLLOWER]: Refusing update from remote peer eb0f1933d9a74fccaa4618047fc05c41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20251211 20:52:29.883307 1626 tablet.cc:1722] T 01ada50100bc48e39d1bc67082da750e P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20251211 20:52:29.884119 1163 tablet_replica.cc:333] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:29.884341 1163 raft_consensus.cc:2243] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:29.884521 1163 raft_consensus.cc:2272] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:29.884654 1586 leader_election.cc:290] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:29.886677 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet ee893e33255745debf73c486d8c70267 (table default.Table_da9a07e5af4a433c8e02a128e8843084 [id=3940282e8e794b21b982be1f171b677d]) successfully deleted
W20251211 20:52:29.886976 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 5710fc2f24764f818f7e5c86c6fd985e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5710fc2f24764f818f7e5c86c6fd985e already in progress: creating tablet
I20251211 20:52:29.888115 1012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58e35de0792a464a818170886ca8d8c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.888176 1168 tablet_bootstrap.cc:654] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:29.888566 1024 tablet_service.cc:1505] Processing CreateTablet for tablet 06b8f8e46e524545aeab6c931efaf116 (DEFAULT_TABLE table=default.Table_bf0900ef3fb8447c834a43e509caa105 [id=861b00629ceb42588a1d1be34153b792]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.889123 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06b8f8e46e524545aeab6c931efaf116. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:29.890949 982 leader_election.cc:343] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:29.894032 1163 ts_tablet_manager.cc:1916] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:29.895515 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 3621735fcb294e3a8c0180ba9ef86442 (DEFAULT_TABLE table=default.Table_01868fec1884449da230c4236ab2d506 [id=d303c5c8fd81400ab473845a942ec140]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.895936 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3621735fcb294e3a8c0180ba9ef86442. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.898650 1166 tablet_bootstrap.cc:492] T 27a08a64d92342d08ab4406e8006a97b P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:29.898805 1166 ts_tablet_manager.cc:1403] T 27a08a64d92342d08ab4406e8006a97b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.608s user 0.012s sys 0.004s
I20251211 20:52:29.900995 1166 raft_consensus.cc:359] T 27a08a64d92342d08ab4406e8006a97b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.901189 1166 raft_consensus.cc:385] T 27a08a64d92342d08ab4406e8006a97b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:29.901278 1166 raft_consensus.cc:740] T 27a08a64d92342d08ab4406e8006a97b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:29.902276 1017 tablet_service.cc:1505] Processing CreateTablet for tablet a7979798e92e45eb969197c8c7ac2580 (DEFAULT_TABLE table=default.Table_a726aa1257194ddbb21fdf77d7d489d8 [id=1c86f943adfb4ed8bfbbcacf4f9fe125]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.902736 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7979798e92e45eb969197c8c7ac2580. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.914577 1168 tablet_bootstrap.cc:492] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:29.914527 1586 raft_consensus.cc:493] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:29.914741 1168 ts_tablet_manager.cc:1403] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.033s user 0.025s sys 0.002s
I20251211 20:52:29.914743 1586 raft_consensus.cc:515] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.915438 1586 leader_election.cc:290] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:29.915534 1168 raft_consensus.cc:359] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.915757 1168 raft_consensus.cc:385] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:29.915838 1168 raft_consensus.cc:740] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:29.916029 1168 consensus_queue.cc:260] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.916308 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3eac21a07ba34e1fa275e25ee9dcabb1" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:29.916877 982 leader_election.cc:343] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:29.917187 1168 ts_tablet_manager.cc:1434] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:29.917472 1168 tablet_bootstrap.cc:492] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:29.919777 1168 tablet_bootstrap.cc:654] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:29.923787 1636 tablet.cc:1722] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:29.924748 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "88066d94d5d34e0fab20da9fb6ef178e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:29.925292 983 leader_election.cc:343] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:29.925421 983 leader_election.cc:304] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:29.925586 983 raft_consensus.cc:2686] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251211 20:52:29.930814 1163 ts_tablet_manager.cc:1929] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:29.930923 1163 log.cc:1199] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/88066d94d5d34e0fab20da9fb6ef178e
I20251211 20:52:29.931241 1163 ts_tablet_manager.cc:1950] T 88066d94d5d34e0fab20da9fb6ef178e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:29.933300 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 88066d94d5d34e0fab20da9fb6ef178e (table default.Table_b06f9587f78645bb9712c3b080f3c0d4 [id=ea12bdaa894f446da22635f95d97ac2a]) successfully deleted
I20251211 20:52:29.933367 1163 tablet_replica.cc:333] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:29.933579 1163 raft_consensus.cc:2243] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:29.933751 1163 raft_consensus.cc:2272] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:29.934109 1168 tablet_bootstrap.cc:492] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:29.934262 1168 ts_tablet_manager.cc:1403] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.006s
I20251211 20:52:29.934262 1163 ts_tablet_manager.cc:1916] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:29.934973 1168 raft_consensus.cc:359] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.935665 1168 raft_consensus.cc:385] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:29.935775 1168 raft_consensus.cc:740] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:29.936045 1168 consensus_queue.cc:260] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.937044 1168 ts_tablet_manager.cc:1434] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20251211 20:52:29.937445 1168 tablet_bootstrap.cc:492] T 27466459d8e6411ab87ac474fb617735 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:29.939651 1168 tablet_bootstrap.cc:654] T 27466459d8e6411ab87ac474fb617735 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:29.940194 1574 raft_consensus.cc:493] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:29.940326 1574 raft_consensus.cc:515] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.940979 1574 leader_election.cc:290] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:29.939087 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3eac21a07ba34e1fa275e25ee9dcabb1" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:29.941470 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8308ad46551447439a248d20ed1e5922" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:29.941684 1044 raft_consensus.cc:2468] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:29.942076 710 leader_election.cc:304] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:29.942332 1574 raft_consensus.cc:2804] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:29.942432 1574 raft_consensus.cc:493] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:29.942514 1574 raft_consensus.cc:3060] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:29.945391 1574 raft_consensus.cc:515] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.946579 1574 leader_election.cc:290] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:29.947144 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8308ad46551447439a248d20ed1e5922" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:29.947332 1044 raft_consensus.cc:3060] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:29.931797 1618 consensus_queue.cc:1048] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.057s
I20251211 20:52:29.949067 1160 tablet_bootstrap.cc:492] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:29.950234 1044 raft_consensus.cc:2468] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:29.950696 1160 ts_tablet_manager.cc:1403] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.141s user 0.005s sys 0.004s
I20251211 20:52:29.950691 710 leader_election.cc:304] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:29.951274 874 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:43876 (request call id 90) took 1604 ms. Trace:
I20251211 20:52:29.951385 1574 raft_consensus.cc:2804] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:29.951381 874 rpcz_store.cc:276] 1211 20:52:28.346427 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:28.346487 (+ 60us) service_pool.cc:225] Handling call
1211 20:52:28.738787 (+392300us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x6160000e3254. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:29.871737 (+1132950us) spinlock_profiling.cc:243] Waited 765 ms on lock 0x6160000b1154. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:29.951036 (+ 79299us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2364915200}
I20251211 20:52:29.951529 1574 raft_consensus.cc:697] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:29.951849 1574 consensus_queue.cc:237] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:29.959975 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8308ad46551447439a248d20ed1e5922" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:29.960188 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8308ad46551447439a248d20ed1e5922" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:29.960690 711 leader_election.cc:343] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:29.961021 711 leader_election.cc:343] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:29.962895 1609 tablet.cc:1722] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
W20251211 20:52:29.974727 592 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 940dfd8781de444a9caa78dd363c7451 consensus state has no leader
W20251211 20:52:29.975239 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 49a33084c0fc460eb9592224b6dfc279 consensus state has no leader
I20251211 20:52:29.976289 594 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 4870da0224e048ed82f227d7ba676a94 (table default.Table_595d8c100e69422ab5e6d44cce9e733c [id=28f3ebe9b7a64b5e89404b0407b0e5ae]). Expected version 2 got 0
I20251211 20:52:29.977176 597 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet c120f98529664957bc430f9b8b991345
I20251211 20:52:29.977360 1160 raft_consensus.cc:359] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:29.977514 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_035e31feb8d941e9b9678ac477b28032"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:29.978082 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_035e31feb8d941e9b9678ac477b28032 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:29.978236 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet eb9da081cb9d43fd9a35c4bc51ea133e consensus state has no leader
I20251211 20:52:29.978655 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_088f58bd95b84371baf067edc743167f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:29.979200 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_088f58bd95b84371baf067edc743167f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:29.979387 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c8e17645a26947ab8c01f9b093fa422c consensus state has no leader
I20251211 20:52:29.979790 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_fd6097253c774917bd3fdf776d468873"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:29.980273 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fd6097253c774917bd3fdf776d468873 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:29.980827 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet efe4d6bb07374294b8ab50a9fe605f87
I20251211 20:52:29.981577 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_5c5941c768984fc49f485d17cb1a485d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:29.982024 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5c5941c768984fc49f485d17cb1a485d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:30.032068 421 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 69700.8us
I20251211 20:52:29.985369 870 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:43876 (request call id 91) took 1250 ms. Trace:
I20251211 20:52:30.193949 870 rpcz_store.cc:276] 1211 20:52:28.735190 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:28.735249 (+ 59us) service_pool.cc:225] Handling call
1211 20:52:29.923772 (+1188523us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x6160000b1154. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:29.985300 (+ 61528us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1422483456}
W20251211 20:52:29.999614 983 leader_election.cc:343] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:30.194653 983 leader_election.cc:304] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:30.195020 1649 raft_consensus.cc:2749] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:30.195091 983 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 235498us
I20251211 20:52:30.015082 1586 raft_consensus.cc:493] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:30.016875 1166 consensus_queue.cc:260] T 27a08a64d92342d08ab4406e8006a97b P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.176517 416 external_mini_cluster.cc:1366] Running /tmp/dist-test-task5TWsQc/build/asan/bin/kudu
/tmp/dist-test-task5TWsQc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.61:40749
--webserver_interface=127.0.104.61
--webserver_port=42143
--builtin_ntp_servers=127.0.104.20:41333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.62:41123,127.0.104.61:40749,127.0.104.60:42087
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20251211 20:52:29.984107 588 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bc32a778e5424850b23ccaf74bb9df07 consensus state has no leader
I20251211 20:52:29.985673 892 tablet_service.cc:1552] Processing DeleteTablet for tablet efe4d6bb07374294b8ab50a9fe605f87 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:29.984659 741 tablet_service.cc:1552] Processing DeleteTablet for tablet efe4d6bb07374294b8ab50a9fe605f87 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:29.985750 1160 raft_consensus.cc:385] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:29.984443 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet a2058b67ddf1413db338bfa2f64e7a77
I20251211 20:52:30.196071 1160 raft_consensus.cc:740] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:30.196197 1166 ts_tablet_manager.cc:1434] T 27a08a64d92342d08ab4406e8006a97b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.297s user 0.003s sys 0.000s
I20251211 20:52:30.196386 1160 consensus_queue.cc:260] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:30.196564 1166 tablet_bootstrap.cc:492] T 49a33084c0fc460eb9592224b6dfc279 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:30.196931 1160 ts_tablet_manager.cc:1434] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.246s user 0.002s sys 0.000s
I20251211 20:52:30.197244 1160 tablet_bootstrap.cc:492] T f00f7c1f6a4e44ff89341b4a55cc93fb P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:30.198297 1166 tablet_bootstrap.cc:654] T 49a33084c0fc460eb9592224b6dfc279 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:30.198446 1160 tablet_bootstrap.cc:654] T f00f7c1f6a4e44ff89341b4a55cc93fb P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:30.026649 1012 tablet_service.cc:1552] Processing DeleteTablet for tablet efe4d6bb07374294b8ab50a9fe605f87 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:30.027516 1168 tablet_bootstrap.cc:492] T 27466459d8e6411ab87ac474fb617735 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:30.048935 740 tablet_service.cc:1505] Processing CreateTablet for tablet 6029c1fd563d4023a8cf7cfb8ffa0876 (DEFAULT_TABLE table=default.Table_d11ab1dc4ef24e4aa5a57ac9876f5eb9 [id=8f7b8c70bd6f42b8817de464b5b7aac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.199946 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6029c1fd563d4023a8cf7cfb8ffa0876. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.049307 750 tablet_service.cc:1505] Processing CreateTablet for tablet 94658fcf512a48d0a992a7e2e0f7c176 (DEFAULT_TABLE table=default.Table_a3a395764457481b9b9082279213a9ef [id=97377a8ef56c4cdebe95d15955207404]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.201117 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94658fcf512a48d0a992a7e2e0f7c176. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.045789 890 tablet_service.cc:1505] Processing CreateTablet for tablet fd43a27db69044b68826cda87e16a086 (DEFAULT_TABLE table=default.Table_608d54a047d5494d8ea7e335dd734192 [id=f1d32002c612403a82bdff5baf209b9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.987462 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet d69fb1f6d9594afcbfb844860b41a377
I20251211 20:52:30.203943 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd43a27db69044b68826cda87e16a086. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.206015 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_5e842888ea5549bb8f88b2bd74e1cfc8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.206616 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5e842888ea5549bb8f88b2bd74e1cfc8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:29.991065 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_8d1a1564105844ed8c61257caa9e03e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.208642 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8d1a1564105844ed8c61257caa9e03e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.045804 879 tablet_service.cc:1505] Processing CreateTablet for tablet e9ead34065e240eeb1f946319537d32a (DEFAULT_TABLE table=default.Table_dd85571b35b14e7b8fa57c09d6bfa1b1 [id=6addaf2424e9450e9a11ee71c17f055d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.209101 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9ead34065e240eeb1f946319537d32a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.046595 883 tablet_service.cc:1505] Processing CreateTablet for tablet 6029c1fd563d4023a8cf7cfb8ffa0876 (DEFAULT_TABLE table=default.Table_d11ab1dc4ef24e4aa5a57ac9876f5eb9 [id=8f7b8c70bd6f42b8817de464b5b7aac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.209887 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6029c1fd563d4023a8cf7cfb8ffa0876. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.991326 592 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_2a25c4a0088d463684aa33e76ad0dd66 [id=f78e1da479af4d23a552af1c4fc55f81] alter complete (version 2)
I20251211 20:52:30.210848 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_162bb61ae9c74b42854b233dab7191ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.211325 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_162bb61ae9c74b42854b233dab7191ee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.034016 1636 tablet.cc:1722] T 694518e6c1ae4032b06b99e9bde2e5dd P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:30.049608 756 tablet_service.cc:1505] Processing CreateTablet for tablet 497c10742f01476db885bd70c4d2c491 (DEFAULT_TABLE table=default.Table_3357d689c6fa4890bcaa97d99990c9fb [id=602d7eee703948d99a0edcb7cac0651b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.211934 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 497c10742f01476db885bd70c4d2c491. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.991636 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_464acf89dba243ae925fa3b824ceb333"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.212666 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_464acf89dba243ae925fa3b824ceb333 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.050779 1163 ts_tablet_manager.cc:1929] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:30.213647 1163 log.cc:1199] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/c4b76a7e84d344268567c551236ce772
I20251211 20:52:30.214056 1163 ts_tablet_manager.cc:1950] T c4b76a7e84d344268567c551236ce772 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:30.215076 1163 tablet_replica.cc:333] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:30.215281 1163 raft_consensus.cc:2243] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:30.215464 1163 raft_consensus.cc:2272] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:30.046839 884 tablet_service.cc:1505] Processing CreateTablet for tablet 94658fcf512a48d0a992a7e2e0f7c176 (DEFAULT_TABLE table=default.Table_a3a395764457481b9b9082279213a9ef [id=97377a8ef56c4cdebe95d15955207404]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.215888 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94658fcf512a48d0a992a7e2e0f7c176. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:29.991851 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_310ce452c2a54f069cf6c65f233c0603"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:30.047449 876 tablet_service.cc:1505] Processing CreateTablet for tablet 497c10742f01476db885bd70c4d2c491 (DEFAULT_TABLE table=default.Table_3357d689c6fa4890bcaa97d99990c9fb [id=602d7eee703948d99a0edcb7cac0651b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.216414 876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 497c10742f01476db885bd70c4d2c491. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:30.216555 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_310ce452c2a54f069cf6c65f233c0603 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.051056 1024 tablet_service.cc:1505] Processing CreateTablet for tablet 94658fcf512a48d0a992a7e2e0f7c176 (DEFAULT_TABLE table=default.Table_a3a395764457481b9b9082279213a9ef [id=97377a8ef56c4cdebe95d15955207404]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.992357 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_2cd4051c3df74bbc9b5d715b770ee926"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.218267 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2cd4051c3df74bbc9b5d715b770ee926 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.051433 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 497c10742f01476db885bd70c4d2c491 (DEFAULT_TABLE table=default.Table_3357d689c6fa4890bcaa97d99990c9fb [id=602d7eee703948d99a0edcb7cac0651b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.055931 1651 tablet.cc:1722] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20251211 20:52:30.062738 878 tablet_service.cc:1505] Processing CreateTablet for tablet 845b5ac4a5984c0cb4ab188925103dc9 (DEFAULT_TABLE table=default.Table_3bccb9c2f8e04bc3aff742806d2b634a [id=08a6f3ec06a84050998d659db08ac087]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:29.995815 593 master_service.cc:954] ReplaceTablet: received request to replace tablet fb030a62796248869d46b682656eaeb0 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:30.057353 1028 tablet_service.cc:1505] Processing CreateTablet for tablet fd43a27db69044b68826cda87e16a086 (DEFAULT_TABLE table=default.Table_608d54a047d5494d8ea7e335dd734192 [id=f1d32002c612403a82bdff5baf209b9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.064821 877 tablet_service.cc:1505] Processing CreateTablet for tablet a295a24e81f14e5fa126a4a58ffd08cf (DEFAULT_TABLE table=default.Table_1021d997007b41208355c31d22f4e26c [id=8c644777215b4617a8ce57691c34ce8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.220014 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a295a24e81f14e5fa126a4a58ffd08cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.220135 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c9b4200ff6244586a61f2f90bd35aa83" } new_table_name: "default.Table_0e1298b5dbe74fdebb9368056208f205" modify_external_catalogs: true
I20251211 20:52:30.055972 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 41c30bded6504bbc9f0af8f34dc81e8d
I20251211 20:52:30.221665 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_96a88507426c4a45b670fa5002d8df08" } new_table_name: "default.Table_51db250005ed4b3d947f32acd1d62563" modify_external_catalogs: true
I20251211 20:52:30.070703 1616 raft_consensus.cc:493] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:30.223212 1616 raft_consensus.cc:515] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:30.103955 891 tablet_service.cc:1505] Processing CreateTablet for tablet 01104975fd8b470aa026578b97ff6b8d (DEFAULT_TABLE table=default.Table_5c5941c768984fc49f485d17cb1a485d [id=433155eccb834498a3b4e213051f8bbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.104997 881 tablet_service.cc:1505] Processing CreateTablet for tablet 1e13965b6361488eaeb7dd13b467ca65 (DEFAULT_TABLE table=default.Table_035e31feb8d941e9b9678ac477b28032 [id=c6de27bf635a42f19cc08dad18c47f4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.106401 893 tablet_service.cc:1505] Processing CreateTablet for tablet 79cc81e91e1846038621af7a0a9f7a2e (DEFAULT_TABLE table=default.Table_088f58bd95b84371baf067edc743167f [id=8bb3e5b7881b4c24b425addc10a063b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.108006 880 tablet_service.cc:1505] Processing CreateTablet for tablet 84b2a67c836844ceb3628cc3b7d8ba10 (DEFAULT_TABLE table=default.Table_c9b4200ff6244586a61f2f90bd35aa83 [id=652317214bae4ccf9d178793ac21bb28]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.109288 882 tablet_service.cc:1505] Processing CreateTablet for tablet bd2bf1113dfe4b67b263b56f3d3db07d (DEFAULT_TABLE table=default.Table_ccd52fe248db4491820429e77b22f465 [id=a3f079fc1d054841b369c45cdae7bdfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.110723 887 tablet_service.cc:1505] Processing CreateTablet for tablet 98d575ac680d49d0990877666638f013 (DEFAULT_TABLE table=default.Table_fd6097253c774917bd3fdf776d468873 [id=cc99459a5039424dbc192ff748b6708b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.223984 1616 leader_election.cc:290] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:30.057757 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 6029c1fd563d4023a8cf7cfb8ffa0876 (DEFAULT_TABLE table=default.Table_d11ab1dc4ef24e4aa5a57ac9876f5eb9 [id=8f7b8c70bd6f42b8817de464b5b7aac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.058073 1014 tablet_service.cc:1505] Processing CreateTablet for tablet e9ead34065e240eeb1f946319537d32a (DEFAULT_TABLE table=default.Table_dd85571b35b14e7b8fa57c09d6bfa1b1 [id=6addaf2424e9450e9a11ee71c17f055d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.062871 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 845b5ac4a5984c0cb4ab188925103dc9 (DEFAULT_TABLE table=default.Table_3bccb9c2f8e04bc3aff742806d2b634a [id=08a6f3ec06a84050998d659db08ac087]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.065042 1021 tablet_service.cc:1505] Processing CreateTablet for tablet a295a24e81f14e5fa126a4a58ffd08cf (DEFAULT_TABLE table=default.Table_1021d997007b41208355c31d22f4e26c [id=8c644777215b4617a8ce57691c34ce8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.104362 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 01104975fd8b470aa026578b97ff6b8d (DEFAULT_TABLE table=default.Table_5c5941c768984fc49f485d17cb1a485d [id=433155eccb834498a3b4e213051f8bbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.106099 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 1e13965b6361488eaeb7dd13b467ca65 (DEFAULT_TABLE table=default.Table_035e31feb8d941e9b9678ac477b28032 [id=c6de27bf635a42f19cc08dad18c47f4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.107641 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 79cc81e91e1846038621af7a0a9f7a2e (DEFAULT_TABLE table=default.Table_088f58bd95b84371baf067edc743167f [id=8bb3e5b7881b4c24b425addc10a063b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.108978 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 84b2a67c836844ceb3628cc3b7d8ba10 (DEFAULT_TABLE table=default.Table_c9b4200ff6244586a61f2f90bd35aa83 [id=652317214bae4ccf9d178793ac21bb28]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.110260 1011 tablet_service.cc:1505] Processing CreateTablet for tablet bd2bf1113dfe4b67b263b56f3d3db07d (DEFAULT_TABLE table=default.Table_ccd52fe248db4491820429e77b22f465 [id=a3f079fc1d054841b369c45cdae7bdfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.111369 1015 tablet_service.cc:1505] Processing CreateTablet for tablet 98d575ac680d49d0990877666638f013 (DEFAULT_TABLE table=default.Table_fd6097253c774917bd3fdf776d468873 [id=cc99459a5039424dbc192ff748b6708b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.224568 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_2a5ff353268344dba84653a0fd41e4a0" } modify_external_catalogs: true
I20251211 20:52:30.224745 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_2a5ff353268344dba84653a0fd41e4a0" } modify_external_catalogs: true
I20251211 20:52:30.049921 753 tablet_service.cc:1505] Processing CreateTablet for tablet 845b5ac4a5984c0cb4ab188925103dc9 (DEFAULT_TABLE table=default.Table_3bccb9c2f8e04bc3aff742806d2b634a [id=08a6f3ec06a84050998d659db08ac087]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.225342 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 845b5ac4a5984c0cb4ab188925103dc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.225405 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01104975fd8b470aa026578b97ff6b8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.226209 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79cc81e91e1846038621af7a0a9f7a2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.058935 738 tablet_service.cc:1505] Processing CreateTablet for tablet e9ead34065e240eeb1f946319537d32a (DEFAULT_TABLE table=default.Table_dd85571b35b14e7b8fa57c09d6bfa1b1 [id=6addaf2424e9450e9a11ee71c17f055d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.227757 741 tablet_service.cc:1552] Processing DeleteTablet for tablet d69fb1f6d9594afcbfb844860b41a377 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:30.227907 738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9ead34065e240eeb1f946319537d32a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.230286 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e13965b6361488eaeb7dd13b467ca65. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.230880 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet c4b76a7e84d344268567c551236ce772 (table default.Table_afd3c9ee35ad41caa1717a299443409e [id=b0ebaab8cd5a4deb9a5ea1e7bd3ccec3]) successfully deleted
W20251211 20:52:30.231125 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet efe4d6bb07374294b8ab50a9fe605f87 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet efe4d6bb07374294b8ab50a9fe605f87 already in progress: creating tablet
W20251211 20:52:30.231427 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet d69fb1f6d9594afcbfb844860b41a377 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d69fb1f6d9594afcbfb844860b41a377 already in progress: creating tablet
I20251211 20:52:30.059398 747 tablet_service.cc:1505] Processing CreateTablet for tablet fd43a27db69044b68826cda87e16a086 (DEFAULT_TABLE table=default.Table_608d54a047d5494d8ea7e335dd734192 [id=f1d32002c612403a82bdff5baf209b9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.231916 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd2bf1113dfe4b67b263b56f3d3db07d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.232105 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd43a27db69044b68826cda87e16a086. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.232549 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 8dff13e22899452c8d4f2731b8dac7d5 with delete_type TABLET_DATA_DELETED (replaced by tablet dc5de12701714fcd81b9dba6973a28b5) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:30.232919 1165 tablet_replica.cc:333] T 8dff13e22899452c8d4f2731b8dac7d5 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:30.233134 1165 raft_consensus.cc:2243] T 8dff13e22899452c8d4f2731b8dac7d5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:30.233314 1165 raft_consensus.cc:2272] T 8dff13e22899452c8d4f2731b8dac7d5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:30.064121 743 tablet_service.cc:1505] Processing CreateTablet for tablet a295a24e81f14e5fa126a4a58ffd08cf (DEFAULT_TABLE table=default.Table_1021d997007b41208355c31d22f4e26c [id=8c644777215b4617a8ce57691c34ce8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.234112 743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a295a24e81f14e5fa126a4a58ffd08cf. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:30.235110 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet efe4d6bb07374294b8ab50a9fe605f87 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet efe4d6bb07374294b8ab50a9fe605f87 already in progress: creating tablet
I20251211 20:52:30.235736 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 41c30bded6504bbc9f0af8f34dc81e8d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:30.235946 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84b2a67c836844ceb3628cc3b7d8ba10. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.237015 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 845b5ac4a5984c0cb4ab188925103dc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.216089 1163 ts_tablet_manager.cc:1916] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:30.217908 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94658fcf512a48d0a992a7e2e0f7c176. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.103646 755 tablet_service.cc:1505] Processing CreateTablet for tablet 01104975fd8b470aa026578b97ff6b8d (DEFAULT_TABLE table=default.Table_5c5941c768984fc49f485d17cb1a485d [id=433155eccb834498a3b4e213051f8bbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.239722 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98d575ac680d49d0990877666638f013. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.239748 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd43a27db69044b68826cda87e16a086. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.239894 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01104975fd8b470aa026578b97ff6b8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.242228 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 497c10742f01476db885bd70c4d2c491. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.104341 748 tablet_service.cc:1505] Processing CreateTablet for tablet 1e13965b6361488eaeb7dd13b467ca65 (DEFAULT_TABLE table=default.Table_035e31feb8d941e9b9678ac477b28032 [id=c6de27bf635a42f19cc08dad18c47f4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.243656 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e13965b6361488eaeb7dd13b467ca65. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.244084 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6029c1fd563d4023a8cf7cfb8ffa0876. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.106372 757 tablet_service.cc:1505] Processing CreateTablet for tablet 79cc81e91e1846038621af7a0a9f7a2e (DEFAULT_TABLE table=default.Table_088f58bd95b84371baf067edc743167f [id=8bb3e5b7881b4c24b425addc10a063b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.248230 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79cc81e91e1846038621af7a0a9f7a2e. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:30.248255 847 leader_election.cc:343] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 2433068ec179490dac3e91d046a42b70
I20251211 20:52:30.248997 591 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_1c6adfd0450b41b5be8a7575406a2149 [id=a24d8c58b9d84cdea2b278ae9c6e0ba5] alter complete (version 4)
I20251211 20:52:30.107981 744 tablet_service.cc:1505] Processing CreateTablet for tablet 84b2a67c836844ceb3628cc3b7d8ba10 (DEFAULT_TABLE table=default.Table_c9b4200ff6244586a61f2f90bd35aa83 [id=652317214bae4ccf9d178793ac21bb28]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.251544 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84b2a67c836844ceb3628cc3b7d8ba10. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.255308 1610 raft_consensus.cc:493] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:30.255532 1610 raft_consensus.cc:515] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:30.256434 1610 leader_election.cc:290] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:30.109277 746 tablet_service.cc:1505] Processing CreateTablet for tablet bd2bf1113dfe4b67b263b56f3d3db07d (DEFAULT_TABLE table=default.Table_ccd52fe248db4491820429e77b22f465 [id=a3f079fc1d054841b369c45cdae7bdfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.256876 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd2bf1113dfe4b67b263b56f3d3db07d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.257776 740 tablet_service.cc:1552] Processing DeleteTablet for tablet a2058b67ddf1413db338bfa2f64e7a77 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:30.110626 754 tablet_service.cc:1505] Processing CreateTablet for tablet 98d575ac680d49d0990877666638f013 (DEFAULT_TABLE table=default.Table_fd6097253c774917bd3fdf776d468873 [id=cc99459a5039424dbc192ff748b6708b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.261495 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98d575ac680d49d0990877666638f013. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.182251 1574 raft_consensus.cc:493] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:30.199604 1168 ts_tablet_manager.cc:1403] T 27466459d8e6411ab87ac474fb617735 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.262s user 0.014s sys 0.022s
I20251211 20:52:30.263780 1574 raft_consensus.cc:515] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.264611 1574 leader_election.cc:290] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:30.264570 1168 raft_consensus.cc:359] T 27466459d8e6411ab87ac474fb617735 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.264791 1168 raft_consensus.cc:385] T 27466459d8e6411ab87ac474fb617735 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:30.264878 1168 raft_consensus.cc:740] T 27466459d8e6411ab87ac474fb617735 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:30.265084 1168 consensus_queue.cc:260] T 27466459d8e6411ab87ac474fb617735 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.265586 1168 ts_tablet_manager.cc:1434] T 27466459d8e6411ab87ac474fb617735 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:30.265895 1168 tablet_bootstrap.cc:492] T ed9ee283045a44978422cd080aa2cda5 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:30.267571 1168 tablet_bootstrap.cc:654] T ed9ee283045a44978422cd080aa2cda5 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:30.272470 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79cc81e91e1846038621af7a0a9f7a2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.274916 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7474a08c51cf4346ae9a518963d56b7a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:30.275031 596 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet dd425601d89148eb95525f796a3abf1e (table default.Table_d4afd6ed615a41ac8f6714c8aac27857 [id=c842792b01cb4c689fcfd3d796151508]). Expected version 1 got 0
I20251211 20:52:30.275461 596 catalog_manager.cc:5654] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:30.276469 596 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 8308ad46551447439a248d20ed1e5922 (table default.Table_1c2d691fc52b40dcb94acbe811aefe97 [id=17b2359416c04980857565ab4df594bd]). Expected version 3 got 0
I20251211 20:52:30.281590 879 tablet_service.cc:1552] Processing DeleteTablet for tablet 41c30bded6504bbc9f0af8f34dc81e8d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:30.281770 711 leader_election.cc:343] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:30.281862 883 tablet_service.cc:1552] Processing DeleteTablet for tablet d69fb1f6d9594afcbfb844860b41a377 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:30.282219 891 tablet_service.cc:1552] Processing DeleteTablet for tablet a2058b67ddf1413db338bfa2f64e7a77 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:30.282974 1168 tablet_bootstrap.cc:492] T ed9ee283045a44978422cd080aa2cda5 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:30.283107 1160 tablet_bootstrap.cc:492] T f00f7c1f6a4e44ff89341b4a55cc93fb P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:30.283197 1168 ts_tablet_manager.cc:1403] T ed9ee283045a44978422cd080aa2cda5 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.000s
I20251211 20:52:30.283254 1160 ts_tablet_manager.cc:1403] T f00f7c1f6a4e44ff89341b4a55cc93fb P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.086s user 0.001s sys 0.001s
I20251211 20:52:30.284041 1168 raft_consensus.cc:359] T ed9ee283045a44978422cd080aa2cda5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.284049 1160 raft_consensus.cc:359] T f00f7c1f6a4e44ff89341b4a55cc93fb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.284226 1160 raft_consensus.cc:385] T f00f7c1f6a4e44ff89341b4a55cc93fb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:30.284334 1168 raft_consensus.cc:385] T ed9ee283045a44978422cd080aa2cda5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:30.284348 1160 raft_consensus.cc:740] T f00f7c1f6a4e44ff89341b4a55cc93fb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:30.284471 1168 raft_consensus.cc:740] T ed9ee283045a44978422cd080aa2cda5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:30.284556 1160 consensus_queue.cc:260] T f00f7c1f6a4e44ff89341b4a55cc93fb P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.284781 1168 consensus_queue.cc:260] T ed9ee283045a44978422cd080aa2cda5 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.285269 1168 ts_tablet_manager.cc:1434] T ed9ee283045a44978422cd080aa2cda5 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:30.285283 1160 ts_tablet_manager.cc:1434] T f00f7c1f6a4e44ff89341b4a55cc93fb P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:30.285611 1160 tablet_bootstrap.cc:492] T 1c04187b8ed146c289a6b17d62b07199 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:30.287276 1160 tablet_bootstrap.cc:654] T 1c04187b8ed146c289a6b17d62b07199 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:30.293088 1165 ts_tablet_manager.cc:1916] T 8dff13e22899452c8d4f2731b8dac7d5 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:30.292925 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 41c30bded6504bbc9f0af8f34dc81e8d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 41c30bded6504bbc9f0af8f34dc81e8d already in progress: creating tablet
W20251211 20:52:30.293931 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet d69fb1f6d9594afcbfb844860b41a377 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d69fb1f6d9594afcbfb844860b41a377 already in progress: creating tablet
I20251211 20:52:30.296615 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_33d44c934d124375827a2987a3ec3d3d" } new_table_name: "default.Table_f51ac924827c4dbea700109489db0963" modify_external_catalogs: true
I20251211 20:52:30.296797 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_1c6adfd0450b41b5be8a7575406a2149" } new_table_name: "default.Table_7ecc6c3df6214115bf3e0aa5d1ffbb48" modify_external_catalogs: true
I20251211 20:52:30.298548 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_35338ef5e24d4a1c9972400654a56d83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.299021 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_35338ef5e24d4a1c9972400654a56d83 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.299862 1160 tablet_bootstrap.cc:492] T 1c04187b8ed146c289a6b17d62b07199 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:30.300019 1160 ts_tablet_manager.cc:1403] T 1c04187b8ed146c289a6b17d62b07199 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.000s
W20251211 20:52:30.300320 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 84b2a67c836844ceb3628cc3b7d8ba10 consensus state has no leader
I20251211 20:52:30.300661 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_6d9aa092c758405fa974ff1675c60a8a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.301131 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6d9aa092c758405fa974ff1675c60a8a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:30.302201 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3e6f62c2deff4c11b137fc0cba5ee234 consensus state has no leader
I20251211 20:52:30.302114 1160 raft_consensus.cc:359] T 1c04187b8ed146c289a6b17d62b07199 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.302335 1160 raft_consensus.cc:385] T 1c04187b8ed146c289a6b17d62b07199 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:30.302412 1160 raft_consensus.cc:740] T 1c04187b8ed146c289a6b17d62b07199 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:30.302529 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_494ee7d3e0dc4a47b47082643a3134ce"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:30.302672 1160 consensus_queue.cc:260] T 1c04187b8ed146c289a6b17d62b07199 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:30.302949 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_494ee7d3e0dc4a47b47082643a3134ce in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.303133 1160 ts_tablet_manager.cc:1434] T 1c04187b8ed146c289a6b17d62b07199 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251211 20:52:30.303411 1160 tablet_bootstrap.cc:492] T 4ddf1bd1e18f4187bf050d6f7767b722 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:30.304421 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_60e84f16bbb3462682f8bdaffd7d1542" } modify_external_catalogs: true
I20251211 20:52:30.304610 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_60e84f16bbb3462682f8bdaffd7d1542" } modify_external_catalogs: true
I20251211 20:52:30.305048 1160 tablet_bootstrap.cc:654] T 4ddf1bd1e18f4187bf050d6f7767b722 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:30.306099 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet a2058b67ddf1413db338bfa2f64e7a77 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a2058b67ddf1413db338bfa2f64e7a77 already in progress: creating tablet
I20251211 20:52:30.307888 1160 tablet_bootstrap.cc:492] T 4ddf1bd1e18f4187bf050d6f7767b722 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:30.308086 1160 ts_tablet_manager.cc:1403] T 4ddf1bd1e18f4187bf050d6f7767b722 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251211 20:52:30.308545 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a295a24e81f14e5fa126a4a58ffd08cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.308835 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e13965b6361488eaeb7dd13b467ca65. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.309252 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 845b5ac4a5984c0cb4ab188925103dc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.310684 1014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9ead34065e240eeb1f946319537d32a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.219033 1166 tablet_bootstrap.cc:492] T 49a33084c0fc460eb9592224b6dfc279 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:30.224023 1012 tablet_service.cc:1552] Processing DeleteTablet for tablet a2058b67ddf1413db338bfa2f64e7a77 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:30.224108 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet d69fb1f6d9594afcbfb844860b41a377 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:30.235095 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 41c30bded6504bbc9f0af8f34dc81e8d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:30.315680 1168 tablet_bootstrap.cc:492] T 91602f30b4464e35b16fa3dcac445b49 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:30.224193 1586 raft_consensus.cc:515] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.323760 1166 ts_tablet_manager.cc:1403] T 49a33084c0fc460eb9592224b6dfc279 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.117s user 0.001s sys 0.008s
I20251211 20:52:30.323971 1168 tablet_bootstrap.cc:654] T 91602f30b4464e35b16fa3dcac445b49 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:30.324743 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98d575ac680d49d0990877666638f013. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.328272 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5fd2a6cf80a04f20aef8ff9c84bc88db" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:30.330200 1011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd2bf1113dfe4b67b263b56f3d3db07d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.330215 1166 raft_consensus.cc:359] T 49a33084c0fc460eb9592224b6dfc279 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.330680 1166 raft_consensus.cc:385] T 49a33084c0fc460eb9592224b6dfc279 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:30.330783 1166 raft_consensus.cc:740] T 49a33084c0fc460eb9592224b6dfc279 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:30.331014 1166 consensus_queue.cc:260] T 49a33084c0fc460eb9592224b6dfc279 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.331525 1166 ts_tablet_manager.cc:1434] T 49a33084c0fc460eb9592224b6dfc279 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20251211 20:52:30.331862 1166 tablet_bootstrap.cc:492] T ae1abef61473443b9922eb13f2ae92fa P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:30.332502 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7474a08c51cf4346ae9a518963d56b7a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:30.332693 1045 raft_consensus.cc:2468] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:30.332979 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84b2a67c836844ceb3628cc3b7d8ba10. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.333210 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01104975fd8b470aa026578b97ff6b8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.332949 1586 leader_election.cc:290] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
W20251211 20:52:30.335708 982 leader_election.cc:343] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:30.350919 1160 raft_consensus.cc:359] T 4ddf1bd1e18f4187bf050d6f7767b722 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:30.351163 1160 raft_consensus.cc:385] T 4ddf1bd1e18f4187bf050d6f7767b722 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:30.351466 1574 raft_consensus.cc:493] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:30.351475 1160 raft_consensus.cc:740] T 4ddf1bd1e18f4187bf050d6f7767b722 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:30.351656 1574 raft_consensus.cc:515] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.352345 1574 leader_election.cc:290] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:30.353041 1160 consensus_queue.cc:260] T 4ddf1bd1e18f4187bf050d6f7767b722 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:30.353649 1160 ts_tablet_manager.cc:1434] T 4ddf1bd1e18f4187bf050d6f7767b722 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.045s user 0.002s sys 0.000s
I20251211 20:52:30.354007 1160 tablet_bootstrap.cc:492] T 0129162969d74f5da3c08a1e6a27ab22 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:30.354214 581 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 84b2a67c836844ceb3628cc3b7d8ba10 (table default.Table_0e1298b5dbe74fdebb9368056208f205 [id=652317214bae4ccf9d178793ac21bb28], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 84b2a67c836844ceb3628cc3b7d8ba10 consensus state has no leader
W20251211 20:52:30.355538 847 leader_election.cc:343] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: c5b164dc0ce1461fae98d950f7c7a726
I20251211 20:52:30.355545 710 leader_election.cc:304] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:30.355818 847 leader_election.cc:343] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 2433068ec179490dac3e91d046a42b70
I20251211 20:52:30.355921 847 leader_election.cc:304] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
W20251211 20:52:30.356267 847 leader_election.cc:343] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: c5b164dc0ce1461fae98d950f7c7a726
I20251211 20:52:30.356365 847 leader_election.cc:304] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:30.356599 1616 raft_consensus.cc:2749] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:30.356806 1616 raft_consensus.cc:2749] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:30.358127 1574 raft_consensus.cc:2804] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:30.358242 1574 raft_consensus.cc:493] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:30.358317 1574 raft_consensus.cc:3060] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:30.360474 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5fd2a6cf80a04f20aef8ff9c84bc88db" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:30.362797 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dd425601d89148eb95525f796a3abf1e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:30.363623 711 leader_election.cc:343] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:30.364804 1160 tablet_bootstrap.cc:654] T 0129162969d74f5da3c08a1e6a27ab22 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:30.367189 1165 ts_tablet_manager.cc:1929] T 8dff13e22899452c8d4f2731b8dac7d5 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:30.367884 1165 log.cc:1199] T 8dff13e22899452c8d4f2731b8dac7d5 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/8dff13e22899452c8d4f2731b8dac7d5
I20251211 20:52:30.368430 1165 ts_tablet_manager.cc:1950] T 8dff13e22899452c8d4f2731b8dac7d5 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:30.369875 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 8dff13e22899452c8d4f2731b8dac7d5 (table default.Table_f572100667d148e8a69e7930596f0f52 [id=cb97ccf262fd4c41a180267fd0678cec]) successfully deleted
W20251211 20:52:30.371559 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 41c30bded6504bbc9f0af8f34dc81e8d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 41c30bded6504bbc9f0af8f34dc81e8d already in progress: creating tablet
W20251211 20:52:30.374372 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet a2058b67ddf1413db338bfa2f64e7a77 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a2058b67ddf1413db338bfa2f64e7a77 already in progress: creating tablet
W20251211 20:52:30.375386 983 leader_election.cc:343] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:30.376820 591 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_b022d39850dc4b8f92b79b8a747f0409 [id=d49d5216afba41d68497f42fc72b1bf0] alter complete (version 3)
I20251211 20:52:30.377255 983 leader_election.cc:304] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:30.377535 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_00616be7e0f84573a75647eacd3a086d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:30.377686 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 05385f20e88c46108e09a78814811209
W20251211 20:52:30.378131 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_00616be7e0f84573a75647eacd3a086d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.378224 588 master_service.cc:954] ReplaceTablet: received request to replace tablet e4b5a7553110424d901788c7a2ecdce6 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:30.378751 588 master_service.cc:954] ReplaceTablet: received request to replace tablet bc32a778e5424850b23ccaf74bb9df07 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:30.379236 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 2433068ec179490dac3e91d046a42b70 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:30.379825 589 master_service.cc:954] ReplaceTablet: received request to replace tablet 822abda3c85848728205b71847ef7029 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:30.380775 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet f22e4d34cde24b37a025c1b5a1046b1b
W20251211 20:52:30.382573 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 60d966c0397c4a4ea3718445e87e9703 consensus state has no leader
I20251211 20:52:30.384404 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dd425601d89148eb95525f796a3abf1e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:30.384647 881 tablet_service.cc:1552] Processing DeleteTablet for tablet 05385f20e88c46108e09a78814811209 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:30.385517 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 2433068ec179490dac3e91d046a42b70 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:30.385572 876 tablet_service.cc:1552] Processing DeleteTablet for tablet f22e4d34cde24b37a025c1b5a1046b1b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:30.385969 1164 tablet_replica.cc:333] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:30.386152 1164 raft_consensus.cc:2243] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:30.386328 1164 raft_consensus.cc:2272] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:30.386500 1586 raft_consensus.cc:2749] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:30.386878 1164 ts_tablet_manager.cc:1916] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:30.389098 1574 raft_consensus.cc:515] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.390172 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7474a08c51cf4346ae9a518963d56b7a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:30.390297 754 tablet_service.cc:1552] Processing DeleteTablet for tablet f22e4d34cde24b37a025c1b5a1046b1b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:30.390705 711 leader_election.cc:343] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:30.390805 755 tablet_service.cc:1552] Processing DeleteTablet for tablet 05385f20e88c46108e09a78814811209 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:30.392048 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 05385f20e88c46108e09a78814811209 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 05385f20e88c46108e09a78814811209 already in progress: creating tablet
I20251211 20:52:30.390338 1574 leader_election.cc:290] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:30.393983 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet f22e4d34cde24b37a025c1b5a1046b1b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:30.394203 1028 tablet_service.cc:1552] Processing DeleteTablet for tablet 27a08a64d92342d08ab4406e8006a97b with delete_type TABLET_DATA_DELETED (replaced by tablet bfb52935296443c58190d4d06e2e6e9c) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:30.392184 1045 raft_consensus.cc:2468] T dd425601d89148eb95525f796a3abf1e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:30.397348 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ddf1bd1e18f4187bf050d6f7767b722 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:30.397580 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c04187b8ed146c289a6b17d62b07199 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:30.397783 876 tablet_service.cc:1552] Processing DeleteTablet for tablet f00f7c1f6a4e44ff89341b4a55cc93fb with delete_type TABLET_DATA_DELETED (replaced by tablet 88066d94d5d34e0fab20da9fb6ef178e) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:30.397922 881 tablet_service.cc:1552] Processing DeleteTablet for tablet c5b164dc0ce1461fae98d950f7c7a726 with delete_type TABLET_DATA_DELETED (replaced by tablet 49a33084c0fc460eb9592224b6dfc279) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:30.405227 1164 ts_tablet_manager.cc:1929] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:30.407431 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 27466459d8e6411ab87ac474fb617735 with delete_type TABLET_DATA_DELETED (replaced by tablet 8feee01fc0aa4b32be926ff2b617b894) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:30.407835 755 tablet_service.cc:1552] Processing DeleteTablet for tablet ed9ee283045a44978422cd080aa2cda5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:30.409545 1164 log.cc:1199] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/2433068ec179490dac3e91d046a42b70
I20251211 20:52:30.410118 1164 ts_tablet_manager.cc:1950] T 2433068ec179490dac3e91d046a42b70 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:30.410840 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet 05385f20e88c46108e09a78814811209 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:30.411346 710 leader_election.cc:304] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:30.411466 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 2433068ec179490dac3e91d046a42b70 (table default.Table_26b3b7e5db73457da38332d4e3d943f6 [id=18edd3deb100406cb5f421ee06216e6b]) successfully deleted
I20251211 20:52:30.411826 1574 raft_consensus.cc:2804] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:30.411921 1574 raft_consensus.cc:493] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:30.412014 1574 raft_consensus.cc:3060] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:30.413044 1164 tablet_replica.cc:333] T 4ddf1bd1e18f4187bf050d6f7767b722 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:30.413239 1164 raft_consensus.cc:2243] T 4ddf1bd1e18f4187bf050d6f7767b722 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:30.413393 1164 raft_consensus.cc:2272] T 4ddf1bd1e18f4187bf050d6f7767b722 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:30.414690 1164 ts_tablet_manager.cc:1916] T 4ddf1bd1e18f4187bf050d6f7767b722 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:30.414599 1574 raft_consensus.cc:515] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.414973 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7474a08c51cf4346ae9a518963d56b7a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:30.415162 1045 raft_consensus.cc:3060] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:30.415729 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dd425601d89148eb95525f796a3abf1e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:30.415818 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet f22e4d34cde24b37a025c1b5a1046b1b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f22e4d34cde24b37a025c1b5a1046b1b already in progress: creating tablet
W20251211 20:52:30.416388 711 leader_election.cc:343] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:30.416740 1574 leader_election.cc:290] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:30.418782 1045 raft_consensus.cc:2468] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:30.420603 710 leader_election.cc:304] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:30.420915 1574 raft_consensus.cc:2804] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:30.421087 1574 raft_consensus.cc:697] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:30.421324 1574 consensus_queue.cc:237] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.425530 1164 ts_tablet_manager.cc:1929] T 4ddf1bd1e18f4187bf050d6f7767b722 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:30.427006 1166 tablet_bootstrap.cc:654] T ae1abef61473443b9922eb13f2ae92fa P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:30.428411 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dd425601d89148eb95525f796a3abf1e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:30.429392 1164 log.cc:1199] T 4ddf1bd1e18f4187bf050d6f7767b722 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/4ddf1bd1e18f4187bf050d6f7767b722
I20251211 20:52:30.429831 1164 ts_tablet_manager.cc:1950] T 4ddf1bd1e18f4187bf050d6f7767b722 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:30.431241 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 4ddf1bd1e18f4187bf050d6f7767b722 (table default.Table_307027e1160747b98c2fb2ea31eb8745 [id=3ade2e0cb4924ec38113bf929b5f554b]) successfully deleted
I20251211 20:52:30.432252 1164 tablet_replica.cc:333] T 1c04187b8ed146c289a6b17d62b07199 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:30.433099 1164 raft_consensus.cc:2243] T 1c04187b8ed146c289a6b17d62b07199 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:30.433876 1164 raft_consensus.cc:2272] T 1c04187b8ed146c289a6b17d62b07199 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:30.435549 1164 ts_tablet_manager.cc:1916] T 1c04187b8ed146c289a6b17d62b07199 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:30.444767 1164 ts_tablet_manager.cc:1929] T 1c04187b8ed146c289a6b17d62b07199 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:30.444916 1164 log.cc:1199] T 1c04187b8ed146c289a6b17d62b07199 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/1c04187b8ed146c289a6b17d62b07199
I20251211 20:52:30.451383 1163 ts_tablet_manager.cc:1929] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:30.451551 1163 log.cc:1199] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/19d0aa259ac94f36b7f6a24d11a3d420
I20251211 20:52:30.452152 1164 ts_tablet_manager.cc:1950] T 1c04187b8ed146c289a6b17d62b07199 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:30.452042 1045 raft_consensus.cc:3060] T dd425601d89148eb95525f796a3abf1e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:30.453367 1163 ts_tablet_manager.cc:1950] T 19d0aa259ac94f36b7f6a24d11a3d420 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:30.454190 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 1c04187b8ed146c289a6b17d62b07199 (table default.Table_2bdd381fdf6443da869b647ddb92699b [id=70ed017f68a140ceaa39557fbe86aa0b]) successfully deleted
I20251211 20:52:30.454567 1164 tablet_replica.cc:333] T f00f7c1f6a4e44ff89341b4a55cc93fb P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:30.455025 1164 raft_consensus.cc:2243] T f00f7c1f6a4e44ff89341b4a55cc93fb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:30.455492 1164 raft_consensus.cc:2272] T f00f7c1f6a4e44ff89341b4a55cc93fb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:30.455704 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet f22e4d34cde24b37a025c1b5a1046b1b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f22e4d34cde24b37a025c1b5a1046b1b already in progress: creating tablet
I20251211 20:52:30.457141 1164 ts_tablet_manager.cc:1916] T f00f7c1f6a4e44ff89341b4a55cc93fb P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:30.457566 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 05385f20e88c46108e09a78814811209 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 05385f20e88c46108e09a78814811209 already in progress: creating tablet
I20251211 20:52:30.460361 1165 tablet_replica.cc:333] T 27466459d8e6411ab87ac474fb617735 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:30.460641 1165 raft_consensus.cc:2243] T 27466459d8e6411ab87ac474fb617735 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:30.461230 1165 raft_consensus.cc:2272] T 27466459d8e6411ab87ac474fb617735 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:30.462118 1165 ts_tablet_manager.cc:1916] T 27466459d8e6411ab87ac474fb617735 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:30.463236 1164 ts_tablet_manager.cc:1929] T f00f7c1f6a4e44ff89341b4a55cc93fb P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:30.463922 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 19d0aa259ac94f36b7f6a24d11a3d420 (table default.Table_ef1c4ca915474d569c91296213346105 [id=9dd03042ff0948b99b39b899255848fe]) successfully deleted
I20251211 20:52:30.464478 1045 raft_consensus.cc:2468] T dd425601d89148eb95525f796a3abf1e P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
W20251211 20:52:30.464661 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 68792ccef54944a09af9b5456e9ff1fe because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 68792ccef54944a09af9b5456e9ff1fe already in progress: creating tablet
I20251211 20:52:30.464973 710 leader_election.cc:304] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:30.465243 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 4f807596fa6a4e66b89566b5d5dd1080 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4f807596fa6a4e66b89566b5d5dd1080 already in progress: creating tablet
I20251211 20:52:30.465412 1574 raft_consensus.cc:2804] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:30.465546 1574 raft_consensus.cc:697] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
W20251211 20:52:30.465919 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 0aac6fb3972e4aba8950828c9e49d507 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0aac6fb3972e4aba8950828c9e49d507 already in progress: creating tablet
I20251211 20:52:30.465822 1574 consensus_queue.cc:237] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:30.466413 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet efe4d6bb07374294b8ab50a9fe605f87 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet efe4d6bb07374294b8ab50a9fe605f87 already in progress: creating tablet
W20251211 20:52:30.466940 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet a2058b67ddf1413db338bfa2f64e7a77 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a2058b67ddf1413db338bfa2f64e7a77 already in progress: creating tablet
W20251211 20:52:30.467442 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet d69fb1f6d9594afcbfb844860b41a377 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d69fb1f6d9594afcbfb844860b41a377 already in progress: creating tablet
I20251211 20:52:30.468986 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_035e31feb8d941e9b9678ac477b28032" } modify_external_catalogs: true
I20251211 20:52:30.469175 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_035e31feb8d941e9b9678ac477b28032" } modify_external_catalogs: true
I20251211 20:52:30.470346 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_16b2ebd2f97940acbda1e91061ef1cf6" } new_table_name: "default.Table_a95581dfcc7a448ea004e1b657388be9" modify_external_catalogs: true
W20251211 20:52:30.470666 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 41c30bded6504bbc9f0af8f34dc81e8d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 41c30bded6504bbc9f0af8f34dc81e8d already in progress: creating tablet
W20251211 20:52:30.471397 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet f22e4d34cde24b37a025c1b5a1046b1b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f22e4d34cde24b37a025c1b5a1046b1b already in progress: creating tablet
I20251211 20:52:30.471740 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_678702734654439b9b10658a0b6b3421" } modify_external_catalogs: true
I20251211 20:52:30.471788 1163 tablet_replica.cc:333] T 27a08a64d92342d08ab4406e8006a97b P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:30.471915 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_678702734654439b9b10658a0b6b3421" } modify_external_catalogs: true
I20251211 20:52:30.472158 1163 raft_consensus.cc:2243] T 27a08a64d92342d08ab4406e8006a97b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:30.472373 1163 raft_consensus.cc:2272] T 27a08a64d92342d08ab4406e8006a97b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:30.473027 1163 ts_tablet_manager.cc:1916] T 27a08a64d92342d08ab4406e8006a97b P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:30.473184 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a726aa1257194ddbb21fdf77d7d489d8" } modify_external_catalogs: true
I20251211 20:52:30.473341 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a726aa1257194ddbb21fdf77d7d489d8" } modify_external_catalogs: true
I20251211 20:52:30.483378 1636 tablet.cc:1722] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20251211 20:52:30.487283 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_20c88681d6824e06ad511ad2ccb67eb4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.487912 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_20c88681d6824e06ad511ad2ccb67eb4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.488571 1164 log.cc:1199] T f00f7c1f6a4e44ff89341b4a55cc93fb P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/f00f7c1f6a4e44ff89341b4a55cc93fb
I20251211 20:52:30.489239 1164 ts_tablet_manager.cc:1950] T f00f7c1f6a4e44ff89341b4a55cc93fb P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:30.489915 1163 ts_tablet_manager.cc:1929] T 27a08a64d92342d08ab4406e8006a97b P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:30.490825 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet f00f7c1f6a4e44ff89341b4a55cc93fb (table default.Table_b06f9587f78645bb9712c3b080f3c0d4 [id=ea12bdaa894f446da22635f95d97ac2a]) successfully deleted
I20251211 20:52:30.491066 1163 log.cc:1199] T 27a08a64d92342d08ab4406e8006a97b P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/27a08a64d92342d08ab4406e8006a97b
I20251211 20:52:30.491390 1168 tablet_bootstrap.cc:492] T 91602f30b4464e35b16fa3dcac445b49 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:30.482496 1640 tablet.cc:1722] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20251211 20:52:30.491657 1163 ts_tablet_manager.cc:1950] T 27a08a64d92342d08ab4406e8006a97b P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:30.491997 1164 tablet_replica.cc:333] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:30.492535 1164 raft_consensus.cc:2243] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:30.493785 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 27a08a64d92342d08ab4406e8006a97b (table default.Table_47249158f89f4c16af084f0e1147ef9f [id=383392d835da4c50848068d83261cecc]) successfully deleted
I20251211 20:52:30.494330 1164 raft_consensus.cc:2272] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:30.495198 1164 ts_tablet_manager.cc:1916] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:30.496268 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 05385f20e88c46108e09a78814811209 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 05385f20e88c46108e09a78814811209 already in progress: creating tablet
I20251211 20:52:30.498281 1168 ts_tablet_manager.cc:1403] T 91602f30b4464e35b16fa3dcac445b49 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.205s user 0.012s sys 0.000s
I20251211 20:52:30.499053 1628 tablet.cc:1722] T 01ada50100bc48e39d1bc67082da750e P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20251211 20:52:30.502660 1164 ts_tablet_manager.cc:1929] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:30.502789 1164 log.cc:1199] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/c5b164dc0ce1461fae98d950f7c7a726
I20251211 20:52:30.503144 1164 ts_tablet_manager.cc:1950] T c5b164dc0ce1461fae98d950f7c7a726 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:30.503473 1168 raft_consensus.cc:359] T 91602f30b4464e35b16fa3dcac445b49 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.503798 1168 raft_consensus.cc:385] T 91602f30b4464e35b16fa3dcac445b49 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:30.503947 1168 raft_consensus.cc:740] T 91602f30b4464e35b16fa3dcac445b49 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:30.504223 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet c5b164dc0ce1461fae98d950f7c7a726 (table default.Table_4de1e61d3b1c4ddebf22cdebca28beac [id=9c24845e4efe475fa792a51ce8a66be6]) successfully deleted
I20251211 20:52:30.504215 1168 consensus_queue.cc:260] T 91602f30b4464e35b16fa3dcac445b49 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.504902 1168 ts_tablet_manager.cc:1434] T 91602f30b4464e35b16fa3dcac445b49 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20251211 20:52:30.505198 1168 tablet_bootstrap.cc:492] T d824bc02b25842d18f28f711eef6669f P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:30.490597 1166 tablet_bootstrap.cc:492] T ae1abef61473443b9922eb13f2ae92fa P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:30.505967 1166 ts_tablet_manager.cc:1403] T ae1abef61473443b9922eb13f2ae92fa P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.174s user 0.006s sys 0.004s
I20251211 20:52:30.506707 1166 raft_consensus.cc:359] T ae1abef61473443b9922eb13f2ae92fa P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.506896 1166 raft_consensus.cc:385] T ae1abef61473443b9922eb13f2ae92fa P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:30.506978 1166 raft_consensus.cc:740] T ae1abef61473443b9922eb13f2ae92fa P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:30.507184 1166 consensus_queue.cc:260] T ae1abef61473443b9922eb13f2ae92fa P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.508332 1166 ts_tablet_manager.cc:1434] T ae1abef61473443b9922eb13f2ae92fa P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:30.512967 1166 tablet_bootstrap.cc:492] T dc5de12701714fcd81b9dba6973a28b5 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:30.514809 1166 tablet_bootstrap.cc:654] T dc5de12701714fcd81b9dba6973a28b5 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:30.520148 1168 tablet_bootstrap.cc:654] T d824bc02b25842d18f28f711eef6669f P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:30.521384 1160 tablet_bootstrap.cc:492] T 0129162969d74f5da3c08a1e6a27ab22 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:30.521649 1160 ts_tablet_manager.cc:1403] T 0129162969d74f5da3c08a1e6a27ab22 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.168s user 0.025s sys 0.031s
I20251211 20:52:30.522506 1160 raft_consensus.cc:359] T 0129162969d74f5da3c08a1e6a27ab22 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:30.522792 1160 raft_consensus.cc:385] T 0129162969d74f5da3c08a1e6a27ab22 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:30.522912 1160 raft_consensus.cc:740] T 0129162969d74f5da3c08a1e6a27ab22 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:30.523303 1166 tablet_bootstrap.cc:492] T dc5de12701714fcd81b9dba6973a28b5 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:30.523214 1160 consensus_queue.cc:260] T 0129162969d74f5da3c08a1e6a27ab22 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:30.523556 1166 ts_tablet_manager.cc:1403] T dc5de12701714fcd81b9dba6973a28b5 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.007s
I20251211 20:52:30.523732 1160 ts_tablet_manager.cc:1434] T 0129162969d74f5da3c08a1e6a27ab22 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:30.524992 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet bc32a778e5424850b23ccaf74bb9df07
I20251211 20:52:30.525457 588 catalog_manager.cc:6724] ReplaceTablet: tablet bc32a778e5424850b23ccaf74bb9df07 deleted and replaced by tablet 6a206abd6ea2420089cb07714e0d19c0
I20251211 20:52:30.525660 588 master_service.cc:954] ReplaceTablet: received request to replace tablet 98d575ac680d49d0990877666638f013 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:30.526839 755 tablet_service.cc:1552] Processing DeleteTablet for tablet bc32a778e5424850b23ccaf74bb9df07 with delete_type TABLET_DATA_DELETED (replaced by tablet 6a206abd6ea2420089cb07714e0d19c0) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:30.527053 1160 tablet_bootstrap.cc:492] T 4ef506349e5b434db7a78809dc44014e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:30.527536 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 822abda3c85848728205b71847ef7029
I20251211 20:52:30.528028 589 catalog_manager.cc:6724] ReplaceTablet: tablet 822abda3c85848728205b71847ef7029 deleted and replaced by tablet f9a3fdf17f274c15ac0bdfa9c174f649
I20251211 20:52:30.528204 881 tablet_service.cc:1552] Processing DeleteTablet for tablet 822abda3c85848728205b71847ef7029 with delete_type TABLET_DATA_DELETED (replaced by tablet f9a3fdf17f274c15ac0bdfa9c174f649) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:30.528357 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_088f58bd95b84371baf067edc743167f" } new_table_name: "default.Table_99d08ab1dfe14c2c9be51f4f6d6498db" modify_external_catalogs: true
W20251211 20:52:30.528509 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0b267c7feaac43f6aacb7229759c7ec3 consensus state has no leader
I20251211 20:52:30.528960 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_6bfa9e94e0b24faaa8240f4280e3d535"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.529490 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6bfa9e94e0b24faaa8240f4280e3d535 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.529532 1160 tablet_bootstrap.cc:654] T 4ef506349e5b434db7a78809dc44014e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:30.529965 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet a7979798e92e45eb969197c8c7ac2580
I20251211 20:52:30.530829 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_aeca38849f1341c4b309630c08331c13"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.531205 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aeca38849f1341c4b309630c08331c13 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.531810 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb
I20251211 20:52:30.532505 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b23d9e7bfa424f92a22e13faaeba571f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.532804 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 822abda3c85848728205b71847ef7029 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 822abda3c85848728205b71847ef7029 already in progress: creating tablet
W20251211 20:52:30.533010 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b23d9e7bfa424f92a22e13faaeba571f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.533043 881 tablet_service.cc:1552] Processing DeleteTablet for tablet a7979798e92e45eb969197c8c7ac2580 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:30.533149 876 tablet_service.cc:1552] Processing DeleteTablet for tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:30.534211 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_903bfd8112e444bcbe3e677773384da5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.534731 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_903bfd8112e444bcbe3e677773384da5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.535836 754 tablet_service.cc:1552] Processing DeleteTablet for tablet a7979798e92e45eb969197c8c7ac2580 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:30.536093 746 tablet_service.cc:1552] Processing DeleteTablet for tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:30.536341 755 tablet_service.cc:1552] Processing DeleteTablet for tablet 822abda3c85848728205b71847ef7029 with delete_type TABLET_DATA_DELETED (replaced by tablet f9a3fdf17f274c15ac0bdfa9c174f649) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:30.536676 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet a7979798e92e45eb969197c8c7ac2580 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a7979798e92e45eb969197c8c7ac2580 already in progress: creating tablet
W20251211 20:52:30.536913 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb already in progress: creating tablet
I20251211 20:52:30.537310 891 tablet_service.cc:1552] Processing DeleteTablet for tablet bc32a778e5424850b23ccaf74bb9df07 with delete_type TABLET_DATA_DELETED (replaced by tablet 6a206abd6ea2420089cb07714e0d19c0) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:30.537894 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet bc32a778e5424850b23ccaf74bb9df07 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc32a778e5424850b23ccaf74bb9df07 already in progress: creating tablet
I20251211 20:52:30.522305 1165 ts_tablet_manager.cc:1929] T 27466459d8e6411ab87ac474fb617735 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:30.538368 1165 log.cc:1199] T 27466459d8e6411ab87ac474fb617735 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/27466459d8e6411ab87ac474fb617735
I20251211 20:52:30.538755 1165 ts_tablet_manager.cc:1950] T 27466459d8e6411ab87ac474fb617735 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:30.526643 1011 tablet_service.cc:1552] Processing DeleteTablet for tablet bc32a778e5424850b23ccaf74bb9df07 with delete_type TABLET_DATA_DELETED (replaced by tablet 6a206abd6ea2420089cb07714e0d19c0) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:30.535445 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 822abda3c85848728205b71847ef7029 with delete_type TABLET_DATA_DELETED (replaced by tablet f9a3fdf17f274c15ac0bdfa9c174f649) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:30.535748 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:30.535861 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet a7979798e92e45eb969197c8c7ac2580 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:30.540040 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 27466459d8e6411ab87ac474fb617735 (table default.Table_265379fa17b74d0f954a2a2c4de113c8 [id=18f8d722faaa4293a112525ff0c96216]) successfully deleted
I20251211 20:52:30.540042 1166 raft_consensus.cc:359] T dc5de12701714fcd81b9dba6973a28b5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.540237 1166 raft_consensus.cc:385] T dc5de12701714fcd81b9dba6973a28b5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:30.540318 1166 raft_consensus.cc:740] T dc5de12701714fcd81b9dba6973a28b5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
W20251211 20:52:30.540555 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet bc32a778e5424850b23ccaf74bb9df07 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc32a778e5424850b23ccaf74bb9df07 already in progress: creating tablet
I20251211 20:52:30.540561 1166 consensus_queue.cc:260] T dc5de12701714fcd81b9dba6973a28b5 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.540823 1165 tablet_replica.cc:333] T ed9ee283045a44978422cd080aa2cda5 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
W20251211 20:52:30.540808 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 822abda3c85848728205b71847ef7029 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 822abda3c85848728205b71847ef7029 already in progress: creating tablet
W20251211 20:52:30.540987 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb already in progress: creating tablet
I20251211 20:52:30.541023 1166 ts_tablet_manager.cc:1434] T dc5de12701714fcd81b9dba6973a28b5 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
W20251211 20:52:30.541222 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet a7979798e92e45eb969197c8c7ac2580 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a7979798e92e45eb969197c8c7ac2580 already in progress: creating tablet
I20251211 20:52:30.542635 1166 tablet_bootstrap.cc:492] T cf1201bba4b6421f8d7ce593f4bd892b P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:30.544425 1166 tablet_bootstrap.cc:654] T cf1201bba4b6421f8d7ce593f4bd892b P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:30.550191 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 79cc81e91e1846038621af7a0a9f7a2e consensus state has no leader
I20251211 20:52:30.550598 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c4fc3456c46345c3955a0c7f79556c05" } new_table_name: "default.Table_fa3f2f115cd043a9aa6be2b070d320cb" modify_external_catalogs: true
I20251211 20:52:30.541018 1165 raft_consensus.cc:2243] T ed9ee283045a44978422cd080aa2cda5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:30.554560 1165 raft_consensus.cc:2272] T ed9ee283045a44978422cd080aa2cda5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:30.559767 594 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 49a33084c0fc460eb9592224b6dfc279 (table default.Table_4de1e61d3b1c4ddebf22cdebca28beac [id=9c24845e4efe475fa792a51ce8a66be6]). Expected version 2 got 0
I20251211 20:52:30.560052 594 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 01ada50100bc48e39d1bc67082da750e (table default.Table_7ecc6c3df6214115bf3e0aa5d1ffbb48 [id=a24d8c58b9d84cdea2b278ae9c6e0ba5]). Expected version 5 got 4
I20251211 20:52:30.560480 755 tablet_service.cc:1505] Processing CreateTablet for tablet de508d79f30f489cb745401f53115165 (DEFAULT_TABLE table=default.Table_8d1a1564105844ed8c61257caa9e03e9 [id=1a74d6608ce342f5ba5547e70ba1c9bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.560976 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de508d79f30f489cb745401f53115165. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.563115 595 catalog_manager.cc:5654] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:30.564735 1026 tablet_service.cc:1505] Processing CreateTablet for tablet de508d79f30f489cb745401f53115165 (DEFAULT_TABLE table=default.Table_8d1a1564105844ed8c61257caa9e03e9 [id=1a74d6608ce342f5ba5547e70ba1c9bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.565179 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de508d79f30f489cb745401f53115165. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.566902 594 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet dc5de12701714fcd81b9dba6973a28b5 (table default.Table_f572100667d148e8a69e7930596f0f52 [id=cb97ccf262fd4c41a180267fd0678cec]). Expected version 2 got 0
I20251211 20:52:30.567720 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 27a08a64d92342d08ab4406e8006a97b with delete_type TABLET_DATA_DELETED (replaced by tablet bfb52935296443c58190d4d06e2e6e9c) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:30.568349 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 27a08a64d92342d08ab4406e8006a97b because the tablet was not found. No further retry: Not found: Tablet not found: 27a08a64d92342d08ab4406e8006a97b
I20251211 20:52:30.569195 891 tablet_service.cc:1505] Processing CreateTablet for tablet de508d79f30f489cb745401f53115165 (DEFAULT_TABLE table=default.Table_8d1a1564105844ed8c61257caa9e03e9 [id=1a74d6608ce342f5ba5547e70ba1c9bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.569619 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de508d79f30f489cb745401f53115165. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:30.570039 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b85af79c2574443998e847bc19c53b69 consensus state has no leader
I20251211 20:52:30.570430 1166 tablet_bootstrap.cc:492] T cf1201bba4b6421f8d7ce593f4bd892b P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:30.573778 1166 ts_tablet_manager.cc:1403] T cf1201bba4b6421f8d7ce593f4bd892b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.031s user 0.003s sys 0.008s
I20251211 20:52:30.574301 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 1e13965b6361488eaeb7dd13b467ca65
I20251211 20:52:30.574647 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 98d575ac680d49d0990877666638f013
I20251211 20:52:30.574627 1166 raft_consensus.cc:359] T cf1201bba4b6421f8d7ce593f4bd892b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.574815 1166 raft_consensus.cc:385] T cf1201bba4b6421f8d7ce593f4bd892b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:30.574910 1166 raft_consensus.cc:740] T cf1201bba4b6421f8d7ce593f4bd892b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:30.575008 588 catalog_manager.cc:6724] ReplaceTablet: tablet 98d575ac680d49d0990877666638f013 deleted and replaced by tablet 834b84a557b04e78b5d2c8c84520a644
I20251211 20:52:30.575126 1166 consensus_queue.cc:260] T cf1201bba4b6421f8d7ce593f4bd892b P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.575279 591 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_7ecc6c3df6214115bf3e0aa5d1ffbb48 [id=a24d8c58b9d84cdea2b278ae9c6e0ba5] alter complete (version 5)
I20251211 20:52:30.575543 1166 ts_tablet_manager.cc:1434] T cf1201bba4b6421f8d7ce593f4bd892b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:30.575850 1166 tablet_bootstrap.cc:492] T bb5c5523e175406d9f6f9cd2736a2045 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:30.576900 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 1e13965b6361488eaeb7dd13b467ca65 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:30.577116 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 98d575ac680d49d0990877666638f013 with delete_type TABLET_DATA_DELETED (replaced by tablet 834b84a557b04e78b5d2c8c84520a644) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:30.579082 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 98d575ac680d49d0990877666638f013 with delete_type TABLET_DATA_DELETED (replaced by tablet 834b84a557b04e78b5d2c8c84520a644) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:30.580312 881 tablet_service.cc:1552] Processing DeleteTablet for tablet 98d575ac680d49d0990877666638f013 with delete_type TABLET_DATA_DELETED (replaced by tablet 834b84a557b04e78b5d2c8c84520a644) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:30.580859 876 tablet_service.cc:1552] Processing DeleteTablet for tablet 1e13965b6361488eaeb7dd13b467ca65 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:30.582333 591 catalog_manager.cc:5654] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:30.582752 591 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet dd425601d89148eb95525f796a3abf1e (table default.Table_d4afd6ed615a41ac8f6714c8aac27857 [id=c842792b01cb4c689fcfd3d796151508]). Expected version 1 got 0
I20251211 20:52:30.583350 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e6252813479645a98ce2709dc7aba798" } modify_external_catalogs: true
I20251211 20:52:30.583516 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e6252813479645a98ce2709dc7aba798" } modify_external_catalogs: true
I20251211 20:52:30.583561 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_063289c4a3514468a2c1fa058dde23f3" } new_table_name: "default.Table_4d70f35e05524680b3d72b2e488adbd5" modify_external_catalogs: true
I20251211 20:52:30.584077 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_17e2482cdad5493a888dd8bd1ba85a7f" } modify_external_catalogs: true
I20251211 20:52:30.584251 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_17e2482cdad5493a888dd8bd1ba85a7f" } modify_external_catalogs: true
I20251211 20:52:30.585312 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_70ec351f0d724592bcfb6bf8b469d84d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.585834 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_70ec351f0d724592bcfb6bf8b469d84d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.586133 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_c14c424a6284406d968649c9991c7a92"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.586553 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c14c424a6284406d968649c9991c7a92 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:30.588147 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 98d575ac680d49d0990877666638f013 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 98d575ac680d49d0990877666638f013 already in progress: creating tablet
I20251211 20:52:30.588213 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_92ff976cb6724b6199d251d20d56b101" } new_table_name: "default.Table_205371a488cd4233a77a9057784ee2dc" modify_external_catalogs: true
W20251211 20:52:30.588456 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 1e13965b6361488eaeb7dd13b467ca65 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1e13965b6361488eaeb7dd13b467ca65 already in progress: creating tablet
I20251211 20:52:30.588893 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_e878c65986704075b805f91d6880bacb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.589318 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e878c65986704075b805f91d6880bacb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.590354 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_bb1778853fbe4b7eaf0117deaf0877b9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.590811 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bb1778853fbe4b7eaf0117deaf0877b9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.591967 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 1e13965b6361488eaeb7dd13b467ca65 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:30.592397 876 tablet_service.cc:1552] Processing DeleteTablet for tablet 0129162969d74f5da3c08a1e6a27ab22 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:30.592633 1164 tablet_replica.cc:333] T 0129162969d74f5da3c08a1e6a27ab22 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:30.592835 1164 raft_consensus.cc:2243] T 0129162969d74f5da3c08a1e6a27ab22 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:30.592978 1164 raft_consensus.cc:2272] T 0129162969d74f5da3c08a1e6a27ab22 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:30.595489 1164 ts_tablet_manager.cc:1916] T 0129162969d74f5da3c08a1e6a27ab22 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:30.596868 1166 tablet_bootstrap.cc:654] T bb5c5523e175406d9f6f9cd2736a2045 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:30.599893 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 98d575ac680d49d0990877666638f013 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 98d575ac680d49d0990877666638f013 already in progress: creating tablet
W20251211 20:52:30.600121 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 1e13965b6361488eaeb7dd13b467ca65 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1e13965b6361488eaeb7dd13b467ca65 already in progress: creating tablet
I20251211 20:52:30.601890 1164 ts_tablet_manager.cc:1929] T 0129162969d74f5da3c08a1e6a27ab22 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:30.602044 1164 log.cc:1199] T 0129162969d74f5da3c08a1e6a27ab22 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/0129162969d74f5da3c08a1e6a27ab22
I20251211 20:52:30.602416 1164 ts_tablet_manager.cc:1950] T 0129162969d74f5da3c08a1e6a27ab22 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:30.603072 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 0129162969d74f5da3c08a1e6a27ab22 (table default.Table_daeaf2f3dd24423e9060e4f09a5f4c99 [id=755ac98365be4def8589dcfd1f3e2cbf]) successfully deleted
I20251211 20:52:30.606648 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 91602f30b4464e35b16fa3dcac445b49 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:30.607466 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 75dd170c5828409fa69bd68fdf28a72b consensus state has no leader
W20251211 20:52:30.607969 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2630ad57845a4c31b20c8a104dd27335 consensus state has no leader
I20251211 20:52:30.613319 1168 tablet_bootstrap.cc:492] T d824bc02b25842d18f28f711eef6669f P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:30.613500 1168 ts_tablet_manager.cc:1403] T d824bc02b25842d18f28f711eef6669f P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.108s user 0.004s sys 0.000s
I20251211 20:52:30.614238 1168 raft_consensus.cc:359] T d824bc02b25842d18f28f711eef6669f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.614414 1168 raft_consensus.cc:385] T d824bc02b25842d18f28f711eef6669f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:30.614517 1168 raft_consensus.cc:740] T d824bc02b25842d18f28f711eef6669f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:30.614740 1168 consensus_queue.cc:260] T d824bc02b25842d18f28f711eef6669f P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.617249 1168 ts_tablet_manager.cc:1434] T d824bc02b25842d18f28f711eef6669f P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:30.617781 1168 tablet_bootstrap.cc:492] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:30.619149 755 tablet_service.cc:1552] Processing DeleteTablet for tablet d824bc02b25842d18f28f711eef6669f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:30.620661 1168 tablet_bootstrap.cc:654] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:30.622885 1160 tablet_bootstrap.cc:492] T 4ef506349e5b434db7a78809dc44014e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:30.623085 1160 ts_tablet_manager.cc:1403] T 4ef506349e5b434db7a78809dc44014e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.096s user 0.012s sys 0.022s
I20251211 20:52:30.624074 1160 raft_consensus.cc:359] T 4ef506349e5b434db7a78809dc44014e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:30.624264 1165 ts_tablet_manager.cc:1916] T ed9ee283045a44978422cd080aa2cda5 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:30.624292 1160 raft_consensus.cc:385] T 4ef506349e5b434db7a78809dc44014e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:30.624380 1160 raft_consensus.cc:740] T 4ef506349e5b434db7a78809dc44014e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:30.624636 1160 consensus_queue.cc:260] T 4ef506349e5b434db7a78809dc44014e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:30.626670 1160 ts_tablet_manager.cc:1434] T 4ef506349e5b434db7a78809dc44014e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:30.627007 1160 tablet_bootstrap.cc:492] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:30.628952 1160 tablet_bootstrap.cc:654] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:30.630558 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ef506349e5b434db7a78809dc44014e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:30.630879 1164 tablet_replica.cc:333] T 4ef506349e5b434db7a78809dc44014e P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:30.631098 1164 raft_consensus.cc:2243] T 4ef506349e5b434db7a78809dc44014e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:30.631315 1164 raft_consensus.cc:2272] T 4ef506349e5b434db7a78809dc44014e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:30.632524 1164 ts_tablet_manager.cc:1916] T 4ef506349e5b434db7a78809dc44014e P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:30.642313 1164 ts_tablet_manager.cc:1929] T 4ef506349e5b434db7a78809dc44014e P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:30.642585 1164 log.cc:1199] T 4ef506349e5b434db7a78809dc44014e P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/4ef506349e5b434db7a78809dc44014e
I20251211 20:52:30.643187 1164 ts_tablet_manager.cc:1950] T 4ef506349e5b434db7a78809dc44014e P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:30.644179 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 4ef506349e5b434db7a78809dc44014e (table default.Table_fd4577173dc44659b95db8e463e0131c [id=143faf67e1204e7bb2015f06392e48a6]) successfully deleted
I20251211 20:52:30.653260 594 master_service.cc:954] ReplaceTablet: received request to replace tablet bb0fc179a337469eae056c5bb3815e6b from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:30.668942 1168 tablet_bootstrap.cc:492] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:30.670207 1168 ts_tablet_manager.cc:1403] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.053s user 0.011s sys 0.005s
I20251211 20:52:30.671380 1168 raft_consensus.cc:359] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.672851 1168 raft_consensus.cc:385] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:30.673120 1168 raft_consensus.cc:740] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:30.673549 1168 consensus_queue.cc:260] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.673831 1165 ts_tablet_manager.cc:1929] T ed9ee283045a44978422cd080aa2cda5 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:30.674388 1165 log.cc:1199] T ed9ee283045a44978422cd080aa2cda5 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/ed9ee283045a44978422cd080aa2cda5
I20251211 20:52:30.674862 1165 ts_tablet_manager.cc:1950] T ed9ee283045a44978422cd080aa2cda5 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:30.681655 755 tablet_service.cc:1552] Processing DeleteTablet for tablet bd8fd6c977204c90b558827fc0b15f43 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:30.682152 1168 ts_tablet_manager.cc:1434] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.012s user 0.000s sys 0.002s
I20251211 20:52:30.683141 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet ed9ee283045a44978422cd080aa2cda5 (table default.Table_786e1caf9f674d48a6aad3ef7edc5b01 [id=33c5cbdd3d004439ab01b4ae69db56d1]) successfully deleted
I20251211 20:52:30.688999 1166 tablet_bootstrap.cc:492] T bb5c5523e175406d9f6f9cd2736a2045 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:30.689179 1166 ts_tablet_manager.cc:1403] T bb5c5523e175406d9f6f9cd2736a2045 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.113s user 0.020s sys 0.049s
W20251211 20:52:30.689813 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet bc32a778e5424850b23ccaf74bb9df07 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc32a778e5424850b23ccaf74bb9df07 already in progress: creating tablet
I20251211 20:52:30.689903 1166 raft_consensus.cc:359] T bb5c5523e175406d9f6f9cd2736a2045 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.690101 1166 raft_consensus.cc:385] T bb5c5523e175406d9f6f9cd2736a2045 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:30.690177 1166 raft_consensus.cc:740] T bb5c5523e175406d9f6f9cd2736a2045 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:30.690390 1166 consensus_queue.cc:260] T bb5c5523e175406d9f6f9cd2736a2045 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.690691 1168 tablet_bootstrap.cc:492] T f19a83b557e24f929b172d926996ae64 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:30.690876 1166 ts_tablet_manager.cc:1434] T bb5c5523e175406d9f6f9cd2736a2045 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:30.692782 593 master_service.cc:954] ReplaceTablet: received request to replace tablet 95866de91f9d437d80f40b7962253610 from {username='slave'} at 127.0.0.1:44772
W20251211 20:52:30.693730 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet a7979798e92e45eb969197c8c7ac2580 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a7979798e92e45eb969197c8c7ac2580 already in progress: creating tablet
I20251211 20:52:30.695833 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 7a0a20f95416425489af94408ac1e97c (DEFAULT_TABLE table=default.Table_903bfd8112e444bcbe3e677773384da5 [id=849873b9731344feb51f5fbefd42576d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.696976 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 6a206abd6ea2420089cb07714e0d19c0
I20251211 20:52:30.696933 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 5df87ea7d28340da84ef141c92efc710 (DEFAULT_TABLE table=default.Table_6bfa9e94e0b24faaa8240f4280e3d535 [id=4358b12fe36941e18e574120e6d892c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.697289 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 32f8d104f6a948d1a97a83769ea82449 (DEFAULT_TABLE table=default.Table_20c88681d6824e06ad511ad2ccb67eb4 [id=4b8f683aba234b40b129ba35b19873ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.697990 1166 tablet_bootstrap.cc:492] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:30.698029 754 tablet_service.cc:1505] Processing CreateTablet for tablet 5df87ea7d28340da84ef141c92efc710 (DEFAULT_TABLE table=default.Table_6bfa9e94e0b24faaa8240f4280e3d535 [id=4358b12fe36941e18e574120e6d892c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.698462 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5df87ea7d28340da84ef141c92efc710. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.700037 1166 tablet_bootstrap.cc:654] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:30.705293 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 940dfd8781de444a9caa78dd363c7451
I20251211 20:52:30.703491 746 tablet_service.cc:1505] Processing CreateTablet for tablet 32f8d104f6a948d1a97a83769ea82449 (DEFAULT_TABLE table=default.Table_20c88681d6824e06ad511ad2ccb67eb4 [id=4b8f683aba234b40b129ba35b19873ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.706056 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32f8d104f6a948d1a97a83769ea82449. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.709406 1160 tablet_bootstrap.cc:492] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:30.709578 1160 ts_tablet_manager.cc:1403] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.083s user 0.015s sys 0.041s
I20251211 20:52:30.710450 1160 raft_consensus.cc:359] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.710659 1160 raft_consensus.cc:385] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:30.710733 1160 raft_consensus.cc:740] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:30.710991 1160 consensus_queue.cc:260] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.712323 1160 ts_tablet_manager.cc:1434] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:30.712638 1160 tablet_bootstrap.cc:492] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:30.714373 1160 tablet_bootstrap.cc:654] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:30.719956 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32f8d104f6a948d1a97a83769ea82449. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.723937 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5df87ea7d28340da84ef141c92efc710. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.732627 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a0a20f95416425489af94408ac1e97c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.739094 1160 tablet_bootstrap.cc:492] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:30.739370 1160 ts_tablet_manager.cc:1403] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.027s user 0.005s sys 0.005s
I20251211 20:52:30.740295 1160 raft_consensus.cc:359] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.740684 1160 raft_consensus.cc:385] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:30.740846 1160 raft_consensus.cc:740] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:30.741503 1160 consensus_queue.cc:260] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.742239 1160 ts_tablet_manager.cc:1434] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251211 20:52:30.743937 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet bb0fc179a337469eae056c5bb3815e6b
I20251211 20:52:30.744287 1160 tablet_bootstrap.cc:492] T 8308ad46551447439a248d20ed1e5922 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:30.744457 594 catalog_manager.cc:6724] ReplaceTablet: tablet bb0fc179a337469eae056c5bb3815e6b deleted and replaced by tablet d53b074865f4490693152a247b0fffdc
I20251211 20:52:30.746170 1160 tablet_bootstrap.cc:654] T 8308ad46551447439a248d20ed1e5922 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:30.749004 757 tablet_service.cc:1505] Processing CreateTablet for tablet be876471f8bf453aa4467c91891cbbde (DEFAULT_TABLE table=default.Table_35338ef5e24d4a1c9972400654a56d83 [id=ad4931c1fa584e83bae93e064eca9621]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.749262 753 tablet_service.cc:1505] Processing CreateTablet for tablet f2ae084982f44fdf8bfecdd2bc4c01b9 (DEFAULT_TABLE table=default.Table_162bb61ae9c74b42854b233dab7191ee [id=43a7e8312fcd4c9cb98abbde71164269]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.749487 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be876471f8bf453aa4467c91891cbbde. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.749861 738 tablet_service.cc:1505] Processing CreateTablet for tablet 962b80ae747b4470abd23643e711fad9 (DEFAULT_TABLE table=default.Table_464acf89dba243ae925fa3b824ceb333 [id=d69ab7378b1d4272b1c2f8df48b96e81]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.750211 738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 962b80ae747b4470abd23643e711fad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.750358 748 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a206abd6ea2420089cb07714e0d19c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:30.750792 756 tablet_service.cc:1505] Processing CreateTablet for tablet c081396b492049b996ac4661e1318388 (DEFAULT_TABLE table=default.Table_310ce452c2a54f069cf6c65f233c0603 [id=95c9ed1ec1834e2dbfd3198887b75369]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.751206 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c081396b492049b996ac4661e1318388. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.752203 750 tablet_service.cc:1505] Processing CreateTablet for tablet 79d92736bb1c46d2bfa5678b2d396ec2 (DEFAULT_TABLE table=default.Table_00616be7e0f84573a75647eacd3a086d [id=41d6f5ec2735416db86dd259e8cb555c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.749004 743 tablet_service.cc:1505] Processing CreateTablet for tablet 04679dfddf564d6f8c16d1c257e29b41 (DEFAULT_TABLE table=default.Table_2cd4051c3df74bbc9b5d715b770ee926 [id=188ac848ba6b41e88df38554246be155]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.752538 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79d92736bb1c46d2bfa5678b2d396ec2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.752803 743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04679dfddf564d6f8c16d1c257e29b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.753034 749 tablet_service.cc:1505] Processing CreateTablet for tablet 6a206abd6ea2420089cb07714e0d19c0 (DEFAULT_TABLE table=default.Table_e6252813479645a98ce2709dc7aba798 [id=db38368266194271a02ad265fd991439]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.753249 742 tablet_service.cc:1552] Processing DeleteTablet for tablet bb0fc179a337469eae056c5bb3815e6b with delete_type TABLET_DATA_DELETED (replaced by tablet d53b074865f4490693152a247b0fffdc) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:30.750243 744 tablet_service.cc:1505] Processing CreateTablet for tablet 903c7f4b312d4677bb35be1aaa46ad6b (DEFAULT_TABLE table=default.Table_6d9aa092c758405fa974ff1675c60a8a [id=78f84aeb9f40475f91cdf41fd040b400]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.753875 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 903c7f4b312d4677bb35be1aaa46ad6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.754268 1612 raft_consensus.cc:493] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:30.754438 1612 raft_consensus.cc:515] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.754539 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2ae084982f44fdf8bfecdd2bc4c01b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.755100 1612 leader_election.cc:290] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:30.755931 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "91602f30b4464e35b16fa3dcac445b49" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:30.756155 776 raft_consensus.cc:2468] T 91602f30b4464e35b16fa3dcac445b49 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 0.
I20251211 20:52:30.759552 755 tablet_service.cc:1505] Processing CreateTablet for tablet 7a0a20f95416425489af94408ac1e97c (DEFAULT_TABLE table=default.Table_903bfd8112e444bcbe3e677773384da5 [id=849873b9731344feb51f5fbefd42576d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.759981 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a0a20f95416425489af94408ac1e97c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.760861 1574 raft_consensus.cc:493] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:30.761831 1015 tablet_service.cc:1505] Processing CreateTablet for tablet be876471f8bf453aa4467c91891cbbde (DEFAULT_TABLE table=default.Table_35338ef5e24d4a1c9972400654a56d83 [id=ad4931c1fa584e83bae93e064eca9621]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.762235 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be876471f8bf453aa4467c91891cbbde. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.762187 1011 tablet_service.cc:1505] Processing CreateTablet for tablet 903c7f4b312d4677bb35be1aaa46ad6b (DEFAULT_TABLE table=default.Table_6d9aa092c758405fa974ff1675c60a8a [id=78f84aeb9f40475f91cdf41fd040b400]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.762423 1028 tablet_service.cc:1505] Processing CreateTablet for tablet 79d92736bb1c46d2bfa5678b2d396ec2 (DEFAULT_TABLE table=default.Table_00616be7e0f84573a75647eacd3a086d [id=41d6f5ec2735416db86dd259e8cb555c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.762559 1011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 903c7f4b312d4677bb35be1aaa46ad6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.762737 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79d92736bb1c46d2bfa5678b2d396ec2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.762804 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 04679dfddf564d6f8c16d1c257e29b41 (DEFAULT_TABLE table=default.Table_2cd4051c3df74bbc9b5d715b770ee926 [id=188ac848ba6b41e88df38554246be155]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.763103 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04679dfddf564d6f8c16d1c257e29b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.763236 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 6a206abd6ea2420089cb07714e0d19c0 (DEFAULT_TABLE table=default.Table_e6252813479645a98ce2709dc7aba798 [id=db38368266194271a02ad265fd991439]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.763615 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a206abd6ea2420089cb07714e0d19c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.763665 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet bb0fc179a337469eae056c5bb3815e6b with delete_type TABLET_DATA_DELETED (replaced by tablet d53b074865f4490693152a247b0fffdc) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:30.764101 1013 tablet_service.cc:1505] Processing CreateTablet for tablet 969a8587019c4202b6e705d126f83d44 (DEFAULT_TABLE table=default.Table_aeca38849f1341c4b309630c08331c13 [id=22347b15537d451d9927a17e8e5a202a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.764480 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 969a8587019c4202b6e705d126f83d44. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.764818 1021 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a206abd6ea2420089cb07714e0d19c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:30.765234 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 4b07f110f25645799ed0c89737cd8279 (DEFAULT_TABLE table=default.Table_5e842888ea5549bb8f88b2bd74e1cfc8 [id=ca09f7f22bfe4a3eb8d69016753a48d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.765579 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b07f110f25645799ed0c89737cd8279. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.767871 1012 tablet_service.cc:1505] Processing CreateTablet for tablet 7b06cd366de5494780b5ac81abbf0702 (DEFAULT_TABLE table=default.Table_b23d9e7bfa424f92a22e13faaeba571f [id=dcd95d2b036045be9b2eb93cdff84982]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.768102 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 962b80ae747b4470abd23643e711fad9 (DEFAULT_TABLE table=default.Table_464acf89dba243ae925fa3b824ceb333 [id=d69ab7378b1d4272b1c2f8df48b96e81]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.768282 1012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b06cd366de5494780b5ac81abbf0702. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.768572 1014 tablet_service.cc:1505] Processing CreateTablet for tablet f2ae084982f44fdf8bfecdd2bc4c01b9 (DEFAULT_TABLE table=default.Table_162bb61ae9c74b42854b233dab7191ee [id=43a7e8312fcd4c9cb98abbde71164269]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.768884 1014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2ae084982f44fdf8bfecdd2bc4c01b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.770215 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 962b80ae747b4470abd23643e711fad9. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:30.771260 847 leader_election.cc:343] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 91602f30b4464e35b16fa3dcac445b49
I20251211 20:52:30.771286 1025 tablet_service.cc:1505] Processing CreateTablet for tablet f9a3fdf17f274c15ac0bdfa9c174f649 (DEFAULT_TABLE table=default.Table_4aad4cf01eff415f86b8446c29977007 [id=01282adb7b834261a7750c350ad0b250]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.771510 847 leader_election.cc:304] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters: e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:30.771664 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9a3fdf17f274c15ac0bdfa9c174f649. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.772013 1612 raft_consensus.cc:2804] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:30.772107 1612 raft_consensus.cc:493] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:30.772198 1612 raft_consensus.cc:3060] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:30.774839 1612 raft_consensus.cc:515] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.775481 1612 leader_election.cc:290] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Requested vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:30.775916 1016 tablet_service.cc:1505] Processing CreateTablet for tablet 3f1840e4b5214ddb878935a79b214eae (DEFAULT_TABLE table=default.Table_494ee7d3e0dc4a47b47082643a3134ce [id=42422ac9805e49cbab71e262e73098b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.776288 1016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f1840e4b5214ddb878935a79b214eae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.779034 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet bb5c5523e175406d9f6f9cd2736a2045 with delete_type TABLET_DATA_DELETED (replaced by tablet cce44b80c97948e1af7ac68e77cccfd6) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:30.779971 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "91602f30b4464e35b16fa3dcac445b49" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:30.767875 1024 tablet_service.cc:1505] Processing CreateTablet for tablet c081396b492049b996ac4661e1318388 (DEFAULT_TABLE table=default.Table_310ce452c2a54f069cf6c65f233c0603 [id=95c9ed1ec1834e2dbfd3198887b75369]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.780205 776 raft_consensus.cc:3060] T 91602f30b4464e35b16fa3dcac445b49 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:30.780385 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c081396b492049b996ac4661e1318388. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:30.781446 847 leader_election.cc:343] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 91602f30b4464e35b16fa3dcac445b49
I20251211 20:52:30.788596 1019 tablet_service.cc:1552] Processing DeleteTablet for tablet 940dfd8781de444a9caa78dd363c7451 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:30.789713 1163 tablet_replica.cc:333] T bb5c5523e175406d9f6f9cd2736a2045 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:30.789902 1163 raft_consensus.cc:2243] T bb5c5523e175406d9f6f9cd2736a2045 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:30.790711 1163 raft_consensus.cc:2272] T bb5c5523e175406d9f6f9cd2736a2045 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:30.791234 747 tablet_service.cc:1505] Processing CreateTablet for tablet 969a8587019c4202b6e705d126f83d44 (DEFAULT_TABLE table=default.Table_aeca38849f1341c4b309630c08331c13 [id=22347b15537d451d9927a17e8e5a202a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.792315 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 969a8587019c4202b6e705d126f83d44. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.792606 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a206abd6ea2420089cb07714e0d19c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.792680 752 tablet_service.cc:1505] Processing CreateTablet for tablet f9a3fdf17f274c15ac0bdfa9c174f649 (DEFAULT_TABLE table=default.Table_4aad4cf01eff415f86b8446c29977007 [id=01282adb7b834261a7750c350ad0b250]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.793359 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 95866de91f9d437d80f40b7962253610
I20251211 20:52:30.793778 593 catalog_manager.cc:6724] ReplaceTablet: tablet 95866de91f9d437d80f40b7962253610 deleted and replaced by tablet 51d12723f5194afb97d2c05fcbaf3776
I20251211 20:52:30.794335 1163 ts_tablet_manager.cc:1916] T bb5c5523e175406d9f6f9cd2736a2045 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:30.795042 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet bb0fc179a337469eae056c5bb3815e6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb0fc179a337469eae056c5bb3815e6b already in progress: creating tablet
W20251211 20:52:30.795264 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 6a206abd6ea2420089cb07714e0d19c0 because the tablet was not found. No further retry: Not found: Tablet not found: 6a206abd6ea2420089cb07714e0d19c0
I20251211 20:52:30.797199 1028 tablet_service.cc:1505] Processing CreateTablet for tablet 4fe5c534b572491c911aae01eaa79bae (DEFAULT_TABLE table=default.Table_e878c65986704075b805f91d6880bacb [id=14fa8debfe5443ce81587f02dfa15847]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.797578 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fe5c534b572491c911aae01eaa79bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.799563 739 tablet_service.cc:1505] Processing CreateTablet for tablet 3f1840e4b5214ddb878935a79b214eae (DEFAULT_TABLE table=default.Table_494ee7d3e0dc4a47b47082643a3134ce [id=42422ac9805e49cbab71e262e73098b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.799432 1160 tablet_bootstrap.cc:492] T 8308ad46551447439a248d20ed1e5922 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:30.800638 1160 ts_tablet_manager.cc:1403] T 8308ad46551447439a248d20ed1e5922 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.056s user 0.002s sys 0.005s
I20251211 20:52:30.801579 1160 raft_consensus.cc:359] T 8308ad46551447439a248d20ed1e5922 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.801999 1160 raft_consensus.cc:385] T 8308ad46551447439a248d20ed1e5922 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:30.802140 1160 raft_consensus.cc:740] T 8308ad46551447439a248d20ed1e5922 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:30.802444 1160 consensus_queue.cc:260] T 8308ad46551447439a248d20ed1e5922 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.804219 1160 ts_tablet_manager.cc:1434] T 8308ad46551447439a248d20ed1e5922 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:30.804612 1160 tablet_bootstrap.cc:492] T 7474a08c51cf4346ae9a518963d56b7a P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:30.806372 1160 tablet_bootstrap.cc:654] T 7474a08c51cf4346ae9a518963d56b7a P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:30.815224 1013 tablet_service.cc:1505] Processing CreateTablet for tablet e59f8afe1834485d9fc64615e7bfe174 (DEFAULT_TABLE table=default.Table_bb1778853fbe4b7eaf0117deaf0877b9 [id=93b01cf646fd4f4dac01b7fecffa5f4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.815737 1014 tablet_service.cc:1505] Processing CreateTablet for tablet 834b84a557b04e78b5d2c8c84520a644 (DEFAULT_TABLE table=default.Table_fd6097253c774917bd3fdf776d468873 [id=cc99459a5039424dbc192ff748b6708b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.816164 1018 tablet_service.cc:1505] Processing CreateTablet for tablet c112714331bf4bd7a1ddcca17f338d4f (DEFAULT_TABLE table=default.Table_c14c424a6284406d968649c9991c7a92 [id=0396911aea9b4a5f860bce4c471e2df7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.820073 1014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 834b84a557b04e78b5d2c8c84520a644. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.820937 1019 tablet_service.cc:1505] Processing CreateTablet for tablet f91487dd7e0f4c8b8483c2b83945da3b (DEFAULT_TABLE table=default.Table_70ec351f0d724592bcfb6bf8b469d84d [id=508da7ce4977453d848dc14a4d8f6758]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.822013 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 95866de91f9d437d80f40b7962253610 with delete_type TABLET_DATA_DELETED (replaced by tablet 51d12723f5194afb97d2c05fcbaf3776) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:30.827776 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f91487dd7e0f4c8b8483c2b83945da3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.830528 1612 raft_consensus.cc:493] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:30.830711 1612 raft_consensus.cc:515] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.831288 1612 leader_election.cc:290] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:30.831367 593 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 8308ad46551447439a248d20ed1e5922 (table default.Table_1c2d691fc52b40dcb94acbe811aefe97 [id=17b2359416c04980857565ab4df594bd]). Expected version 3 got 0
I20251211 20:52:30.838660 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c112714331bf4bd7a1ddcca17f338d4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.840457 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e59f8afe1834485d9fc64615e7bfe174. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.834759 741 tablet_service.cc:1505] Processing CreateTablet for tablet 7b06cd366de5494780b5ac81abbf0702 (DEFAULT_TABLE table=default.Table_b23d9e7bfa424f92a22e13faaeba571f [id=dcd95d2b036045be9b2eb93cdff84982]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:30.847416 582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.61:40749, user_credentials={real_user=slave}} blocked reactor thread for 154216us
I20251211 20:52:30.851485 1025 tablet_service.cc:1505] Processing CreateTablet for tablet d53b074865f4490693152a247b0fffdc (DEFAULT_TABLE table=default.Table_c27589191ac249f4a9460f5bbae2611b [id=461d63ce3b4f4d12b5f395bb8c1cdbb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.852484 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d53b074865f4490693152a247b0fffdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.856827 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 51d12723f5194afb97d2c05fcbaf3776 (DEFAULT_TABLE table=default.Table_7c081945e59345a7830dedbfd3f8f3ee [id=ec0bb10933fa4529bef3580bee6409e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.857156 740 tablet_service.cc:1505] Processing CreateTablet for tablet 4b07f110f25645799ed0c89737cd8279 (DEFAULT_TABLE table=default.Table_5e842888ea5549bb8f88b2bd74e1cfc8 [id=ca09f7f22bfe4a3eb8d69016753a48d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.858914 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 940dfd8781de444a9caa78dd363c7451 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:30.835655 751 tablet_service.cc:1505] Processing CreateTablet for tablet f91487dd7e0f4c8b8483c2b83945da3b (DEFAULT_TABLE table=default.Table_70ec351f0d724592bcfb6bf8b469d84d [id=508da7ce4977453d848dc14a4d8f6758]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.863523 748 tablet_service.cc:1505] Processing CreateTablet for tablet e59f8afe1834485d9fc64615e7bfe174 (DEFAULT_TABLE table=default.Table_bb1778853fbe4b7eaf0117deaf0877b9 [id=93b01cf646fd4f4dac01b7fecffa5f4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.863786 1574 raft_consensus.cc:515] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.866956 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9a3fdf17f274c15ac0bdfa9c174f649. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.863873 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 95866de91f9d437d80f40b7962253610 with delete_type TABLET_DATA_DELETED (replaced by tablet 51d12723f5194afb97d2c05fcbaf3776) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:30.876355 754 tablet_service.cc:1505] Processing CreateTablet for tablet 834b84a557b04e78b5d2c8c84520a644 (DEFAULT_TABLE table=default.Table_fd6097253c774917bd3fdf776d468873 [id=cc99459a5039424dbc192ff748b6708b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.879313 745 tablet_service.cc:1505] Processing CreateTablet for tablet 4fe5c534b572491c911aae01eaa79bae (DEFAULT_TABLE table=default.Table_e878c65986704075b805f91d6880bacb [id=14fa8debfe5443ce81587f02dfa15847]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.880336 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f91487dd7e0f4c8b8483c2b83945da3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.880558 1574 leader_election.cc:290] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:30.882719 847 leader_election.cc:343] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 5fc8d485cc234fd490c487a8bc881bf4
W20251211 20:52:30.882898 847 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 46263.8us
W20251211 20:52:30.883236 847 leader_election.cc:343] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 5fc8d485cc234fd490c487a8bc881bf4
I20251211 20:52:30.883348 847 leader_election.cc:304] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:30.884524 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b06cd366de5494780b5ac81abbf0702. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.885337 1612 raft_consensus.cc:2749] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:30.886145 1163 ts_tablet_manager.cc:1929] T bb5c5523e175406d9f6f9cd2736a2045 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:30.886286 1163 log.cc:1199] T bb5c5523e175406d9f6f9cd2736a2045 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/bb5c5523e175406d9f6f9cd2736a2045
I20251211 20:52:30.886502 742 tablet_service.cc:1505] Processing CreateTablet for tablet d53b074865f4490693152a247b0fffdc (DEFAULT_TABLE table=default.Table_c27589191ac249f4a9460f5bbae2611b [id=461d63ce3b4f4d12b5f395bb8c1cdbb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.886693 1163 ts_tablet_manager.cc:1950] T bb5c5523e175406d9f6f9cd2736a2045 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
W20251211 20:52:30.875761 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb already in progress: creating tablet
I20251211 20:52:30.887559 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f1840e4b5214ddb878935a79b214eae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.894137 1586 raft_consensus.cc:493] T dc5de12701714fcd81b9dba6973a28b5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:30.894304 1586 raft_consensus.cc:515] T dc5de12701714fcd81b9dba6973a28b5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.894987 1586 leader_election.cc:290] T dc5de12701714fcd81b9dba6973a28b5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:30.896179 889 tablet_service.cc:1505] Processing CreateTablet for tablet f9a3fdf17f274c15ac0bdfa9c174f649 (DEFAULT_TABLE table=default.Table_4aad4cf01eff415f86b8446c29977007 [id=01282adb7b834261a7750c350ad0b250]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.896546 879 tablet_service.cc:1505] Processing CreateTablet for tablet be876471f8bf453aa4467c91891cbbde (DEFAULT_TABLE table=default.Table_35338ef5e24d4a1c9972400654a56d83 [id=ad4931c1fa584e83bae93e064eca9621]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.896804 885 tablet_service.cc:1505] Processing CreateTablet for tablet 3f1840e4b5214ddb878935a79b214eae (DEFAULT_TABLE table=default.Table_494ee7d3e0dc4a47b47082643a3134ce [id=42422ac9805e49cbab71e262e73098b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.897173 875 tablet_service.cc:1552] Processing DeleteTablet for tablet bb0fc179a337469eae056c5bb3815e6b with delete_type TABLET_DATA_DELETED (replaced by tablet d53b074865f4490693152a247b0fffdc) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:30.897316 892 tablet_service.cc:1505] Processing CreateTablet for tablet 6a206abd6ea2420089cb07714e0d19c0 (DEFAULT_TABLE table=default.Table_e6252813479645a98ce2709dc7aba798 [id=db38368266194271a02ad265fd991439]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.898193 890 tablet_service.cc:1505] Processing CreateTablet for tablet 4b07f110f25645799ed0c89737cd8279 (DEFAULT_TABLE table=default.Table_5e842888ea5549bb8f88b2bd74e1cfc8 [id=ca09f7f22bfe4a3eb8d69016753a48d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.898511 881 tablet_service.cc:1505] Processing CreateTablet for tablet 32f8d104f6a948d1a97a83769ea82449 (DEFAULT_TABLE table=default.Table_20c88681d6824e06ad511ad2ccb67eb4 [id=4b8f683aba234b40b129ba35b19873ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.898814 882 tablet_service.cc:1505] Processing CreateTablet for tablet 969a8587019c4202b6e705d126f83d44 (DEFAULT_TABLE table=default.Table_aeca38849f1341c4b309630c08331c13 [id=22347b15537d451d9927a17e8e5a202a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.899106 886 tablet_service.cc:1505] Processing CreateTablet for tablet 7b06cd366de5494780b5ac81abbf0702 (DEFAULT_TABLE table=default.Table_b23d9e7bfa424f92a22e13faaeba571f [id=dcd95d2b036045be9b2eb93cdff84982]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.899430 878 tablet_service.cc:1505] Processing CreateTablet for tablet f2ae084982f44fdf8bfecdd2bc4c01b9 (DEFAULT_TABLE table=default.Table_162bb61ae9c74b42854b233dab7191ee [id=43a7e8312fcd4c9cb98abbde71164269]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.899766 891 tablet_service.cc:1505] Processing CreateTablet for tablet 7a0a20f95416425489af94408ac1e97c (DEFAULT_TABLE table=default.Table_903bfd8112e444bcbe3e677773384da5 [id=849873b9731344feb51f5fbefd42576d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.900373 887 tablet_service.cc:1505] Processing CreateTablet for tablet 962b80ae747b4470abd23643e711fad9 (DEFAULT_TABLE table=default.Table_464acf89dba243ae925fa3b824ceb333 [id=d69ab7378b1d4272b1c2f8df48b96e81]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.900379 876 tablet_service.cc:1505] Processing CreateTablet for tablet 5df87ea7d28340da84ef141c92efc710 (DEFAULT_TABLE table=default.Table_6bfa9e94e0b24faaa8240f4280e3d535 [id=4358b12fe36941e18e574120e6d892c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.901109 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dc5de12701714fcd81b9dba6973a28b5" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:30.901365 875 tablet_service.cc:1505] Processing CreateTablet for tablet e59f8afe1834485d9fc64615e7bfe174 (DEFAULT_TABLE table=default.Table_bb1778853fbe4b7eaf0117deaf0877b9 [id=93b01cf646fd4f4dac01b7fecffa5f4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.901556 884 tablet_service.cc:1505] Processing CreateTablet for tablet 04679dfddf564d6f8c16d1c257e29b41 (DEFAULT_TABLE table=default.Table_2cd4051c3df74bbc9b5d715b770ee926 [id=188ac848ba6b41e88df38554246be155]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.901710 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9a3fdf17f274c15ac0bdfa9c174f649. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:30.901759 983 leader_election.cc:343] T dc5de12701714fcd81b9dba6973a28b5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:30.901945 876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5df87ea7d28340da84ef141c92efc710. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.902295 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32f8d104f6a948d1a97a83769ea82449. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:30.902863 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 822abda3c85848728205b71847ef7029 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 822abda3c85848728205b71847ef7029 already in progress: creating tablet
I20251211 20:52:30.903290 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51d12723f5194afb97d2c05fcbaf3776. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.903846 1044 raft_consensus.cc:1275] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Refusing update from remote peer 6f8d784fd1cd492bbe3809403ec34919: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251211 20:52:30.904318 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a206abd6ea2420089cb07714e0d19c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:30.904547 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 940dfd8781de444a9caa78dd363c7451 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:30.905170 877 tablet_service.cc:1505] Processing CreateTablet for tablet 79d92736bb1c46d2bfa5678b2d396ec2 (DEFAULT_TABLE table=default.Table_00616be7e0f84573a75647eacd3a086d [id=41d6f5ec2735416db86dd259e8cb555c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.905200 880 tablet_service.cc:1505] Processing CreateTablet for tablet c081396b492049b996ac4661e1318388 (DEFAULT_TABLE table=default.Table_310ce452c2a54f069cf6c65f233c0603 [id=95c9ed1ec1834e2dbfd3198887b75369]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.905414 883 tablet_service.cc:1505] Processing CreateTablet for tablet 903c7f4b312d4677bb35be1aaa46ad6b (DEFAULT_TABLE table=default.Table_6d9aa092c758405fa974ff1675c60a8a [id=78f84aeb9f40475f91cdf41fd040b400]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.905527 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79d92736bb1c46d2bfa5678b2d396ec2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.905802 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 903c7f4b312d4677bb35be1aaa46ad6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.906088 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f1840e4b5214ddb878935a79b214eae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.910398 875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e59f8afe1834485d9fc64615e7bfe174. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.910689 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a206abd6ea2420089cb07714e0d19c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.911448 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 969a8587019c4202b6e705d126f83d44. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.912176 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04679dfddf564d6f8c16d1c257e29b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.912647 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b07f110f25645799ed0c89737cd8279. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.912853 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b07f110f25645799ed0c89737cd8279. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.913544 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a0a20f95416425489af94408ac1e97c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.914479 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c081396b492049b996ac4661e1318388. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.915082 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2ae084982f44fdf8bfecdd2bc4c01b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.915706 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b06cd366de5494780b5ac81abbf0702. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.915752 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dc5de12701714fcd81b9dba6973a28b5" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:30.915827 776 raft_consensus.cc:2468] T 91602f30b4464e35b16fa3dcac445b49 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 1.
I20251211 20:52:30.916280 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be876471f8bf453aa4467c91891cbbde. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:30.916405 982 leader_election.cc:343] T dc5de12701714fcd81b9dba6973a28b5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:30.916556 982 leader_election.cc:304] T dc5de12701714fcd81b9dba6973a28b5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:30.917034 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 962b80ae747b4470abd23643e711fad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.917028 1586 raft_consensus.cc:2749] T dc5de12701714fcd81b9dba6973a28b5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:30.917363 710 leader_election.cc:343] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: bd8fd6c977204c90b558827fc0b15f43
I20251211 20:52:30.917200 1160 tablet_bootstrap.cc:492] T 7474a08c51cf4346ae9a518963d56b7a P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:30.917956 1160 ts_tablet_manager.cc:1403] T 7474a08c51cf4346ae9a518963d56b7a P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.113s user 0.007s sys 0.013s
I20251211 20:52:30.918022 1512 consensus_queue.cc:1048] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:30.923025 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet bb5c5523e175406d9f6f9cd2736a2045 (table default.Table_349fa502a7c44fe79baed28f7da89ff9 [id=1bd6b7930f2447659da365b7c06b1640]) successfully deleted
W20251211 20:52:30.923312 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 940dfd8781de444a9caa78dd363c7451 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 940dfd8781de444a9caa78dd363c7451 already in progress: creating tablet
I20251211 20:52:30.923863 1586 raft_consensus.cc:493] T 49a33084c0fc460eb9592224b6dfc279 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:30.924023 1586 raft_consensus.cc:515] T 49a33084c0fc460eb9592224b6dfc279 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.924715 1586 leader_election.cc:290] T 49a33084c0fc460eb9592224b6dfc279 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:30.925261 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "49a33084c0fc460eb9592224b6dfc279" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:30.925383 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 834b84a557b04e78b5d2c8c84520a644. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:30.925822 983 leader_election.cc:343] T 49a33084c0fc460eb9592224b6dfc279 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:30.926976 888 tablet_service.cc:1505] Processing CreateTablet for tablet f91487dd7e0f4c8b8483c2b83945da3b (DEFAULT_TABLE table=default.Table_70ec351f0d724592bcfb6bf8b469d84d [id=508da7ce4977453d848dc14a4d8f6758]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.927796 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f91487dd7e0f4c8b8483c2b83945da3b. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:30.923527 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 95866de91f9d437d80f40b7962253610 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 95866de91f9d437d80f40b7962253610 already in progress: creating tablet
I20251211 20:52:30.927455 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_441ecee3b8204f9fa8d22146340e5303" } modify_external_catalogs: true
I20251211 20:52:30.927389 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_204dbc031ddb42f38079cbdc40beeff8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:30.928356 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_441ecee3b8204f9fa8d22146340e5303" } modify_external_catalogs: true
W20251211 20:52:30.928785 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_204dbc031ddb42f38079cbdc40beeff8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.929309 1166 tablet_bootstrap.cc:492] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:30.929458 1166 ts_tablet_manager.cc:1403] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.232s user 0.021s sys 0.025s
I20251211 20:52:30.932524 1166 raft_consensus.cc:359] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.932849 1166 raft_consensus.cc:385] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:30.933360 1166 raft_consensus.cc:740] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:30.933907 746 tablet_service.cc:1505] Processing CreateTablet for tablet c112714331bf4bd7a1ddcca17f338d4f (DEFAULT_TABLE table=default.Table_c14c424a6284406d968649c9991c7a92 [id=0396911aea9b4a5f860bce4c471e2df7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.934078 1166 consensus_queue.cc:260] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:30.934688 1166 ts_tablet_manager.cc:1434] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:30.935040 1166 tablet_bootstrap.cc:492] T dcdcff8235814b58b3683104a2ff0cbf P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:30.935323 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 47779034ad89466e9e1801987128b4dd
I20251211 20:52:30.935359 1168 tablet_bootstrap.cc:654] T f19a83b557e24f929b172d926996ae64 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:30.937186 1166 tablet_bootstrap.cc:654] T dcdcff8235814b58b3683104a2ff0cbf P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:30.938244 588 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_df56cd18522b410fade2a880c087eff4" } new_table_name: "default.Table_ea40e3f0df5c4db7b27261d32241d8d0" modify_external_catalogs: true
I20251211 20:52:30.938618 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 47779034ad89466e9e1801987128b4dd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:30.939311 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 47779034ad89466e9e1801987128b4dd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 47779034ad89466e9e1801987128b4dd already in progress: creating tablet
I20251211 20:52:30.941290 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_fac9e7762d1a492f87a0e5b18868f2ea" } modify_external_catalogs: true
I20251211 20:52:30.941483 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_fac9e7762d1a492f87a0e5b18868f2ea" } modify_external_catalogs: true
I20251211 20:52:30.942186 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fe5c534b572491c911aae01eaa79bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.942845 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_7e2c82ae1dbe44078774e5229684130d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.943472 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e2c82ae1dbe44078774e5229684130d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.943830 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e878c65986704075b805f91d6880bacb" } new_table_name: "default.Table_8ba21562eb054a09aa7b9e22a79433f2" modify_external_catalogs: true
I20251211 20:52:30.944085 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_2596194e7574455bb855e8ed55b588ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.944626 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2596194e7574455bb855e8ed55b588ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.943905 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "49a33084c0fc460eb9592224b6dfc279" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:30.944842 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_59b6bdb8884646d49dc69e9b618915c8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.945276 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_59b6bdb8884646d49dc69e9b618915c8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.945364 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_13a8009dbcc342d98426f71c053a964a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.945775 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_13a8009dbcc342d98426f71c053a964a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.946795 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_28744eba003244878ff0195d5d38643e" } new_table_name: "default.Table_f541a176465249d8aaba954bc23bcf01" modify_external_catalogs: true
W20251211 20:52:30.947989 982 leader_election.cc:343] T 49a33084c0fc460eb9592224b6dfc279 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:30.948146 982 leader_election.cc:304] T 49a33084c0fc460eb9592224b6dfc279 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:30.948609 1586 raft_consensus.cc:2749] T 49a33084c0fc460eb9592224b6dfc279 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:30.951328 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_290ed4f01870438ca11f93a98c4ce8e8" } modify_external_catalogs: true
I20251211 20:52:30.951551 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_290ed4f01870438ca11f93a98c4ce8e8" } modify_external_catalogs: true
I20251211 20:52:30.951522 757 tablet_service.cc:1505] Processing CreateTablet for tablet 51d12723f5194afb97d2c05fcbaf3776 (DEFAULT_TABLE table=default.Table_7c081945e59345a7830dedbfd3f8f3ee [id=ec0bb10933fa4529bef3580bee6409e7]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:30.954195 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 1e13965b6361488eaeb7dd13b467ca65 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1e13965b6361488eaeb7dd13b467ca65 already in progress: creating tablet
I20251211 20:52:30.955181 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_c6fbe03584554d35ae13e59b8d8f0e70"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.955827 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c6fbe03584554d35ae13e59b8d8f0e70 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:30.956313 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet bb0fc179a337469eae056c5bb3815e6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb0fc179a337469eae056c5bb3815e6b already in progress: creating tablet
W20251211 20:52:30.956557 582 catalog_manager.cc:4952] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 6a206abd6ea2420089cb07714e0d19c0 because the tablet was not found. No further retry: Not found: Tablet not found: 6a206abd6ea2420089cb07714e0d19c0
W20251211 20:52:30.956809 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 940dfd8781de444a9caa78dd363c7451 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 940dfd8781de444a9caa78dd363c7451 already in progress: creating tablet
W20251211 20:52:30.962903 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 98d575ac680d49d0990877666638f013 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 98d575ac680d49d0990877666638f013 already in progress: creating tablet
W20251211 20:52:30.971853 588 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d43210ec61af423ebde624397dea9f2e consensus state has no leader
I20251211 20:52:30.972123 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 71dc8d20d93243e289330d7b8c635511
I20251211 20:52:30.975142 1636 tablet.cc:1722] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:30.976169 893 tablet_service.cc:1505] Processing CreateTablet for tablet 4fe5c534b572491c911aae01eaa79bae (DEFAULT_TABLE table=default.Table_e878c65986704075b805f91d6880bacb [id=14fa8debfe5443ce81587f02dfa15847]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.977301 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_9756e2d986ee43839f4462f983927f6d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:30.977648 588 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 8308ad46551447439a248d20ed1e5922 (table default.Table_1c2d691fc52b40dcb94acbe811aefe97 [id=17b2359416c04980857565ab4df594bd]). Expected version 3 got 0
W20251211 20:52:30.978865 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9756e2d986ee43839f4462f983927f6d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.979018 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fe5c534b572491c911aae01eaa79bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:30.980854 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_06f4a9c6bfcd4fac94df56806a29ec03"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:30.980943 894 tablet_service.cc:1505] Processing CreateTablet for tablet 834b84a557b04e78b5d2c8c84520a644 (DEFAULT_TABLE table=default.Table_fd6097253c774917bd3fdf776d468873 [id=cc99459a5039424dbc192ff748b6708b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:30.983525 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 71dc8d20d93243e289330d7b8c635511 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:30.985303 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_06f4a9c6bfcd4fac94df56806a29ec03 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:30.985352 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_f516a45c0526407c96ee48db2945eb80"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.985981 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f516a45c0526407c96ee48db2945eb80 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:30.986493 592 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c4e048ca28824329b924845ccf5067cf consensus state has no leader
I20251211 20:52:30.986661 743 tablet_service.cc:1552] Processing DeleteTablet for tablet 71dc8d20d93243e289330d7b8c635511 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:30.986971 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_547385bc5ed1465ea82e0328b72f83c0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:30.987066 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 71dc8d20d93243e289330d7b8c635511 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 71dc8d20d93243e289330d7b8c635511 already in progress: creating tablet
W20251211 20:52:30.987494 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_547385bc5ed1465ea82e0328b72f83c0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:30.986789 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4fe5c534b572491c911aae01eaa79bae consensus state has no leader
I20251211 20:52:30.988044 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_47249158f89f4c16af084f0e1147ef9f" } new_table_name: "default.Table_3721b5725e8c4297a392d3986a081128" modify_external_catalogs: true
I20251211 20:52:30.988973 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_dd85571b35b14e7b8fa57c09d6bfa1b1" } new_table_name: "default.Table_3826c483ba654865be6be42ae333a65b" modify_external_catalogs: true
I20251211 20:52:30.993800 592 master_service.cc:954] ReplaceTablet: received request to replace tablet 4ce495233a5e4f3792faf44680df54ba from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:30.994293 592 master_service.cc:954] ReplaceTablet: received request to replace tablet bb112b6431af42a6a7b328dfdefbc2a2 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:30.998565 738 tablet_service.cc:1552] Processing DeleteTablet for tablet 47779034ad89466e9e1801987128b4dd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:30.998628 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d53b074865f4490693152a247b0fffdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.000300 1165 tablet_replica.cc:333] T 91602f30b4464e35b16fa3dcac445b49 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:31.001359 1628 tablet.cc:1722] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:31.001967 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e59f8afe1834485d9fc64615e7bfe174. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.015141 1165 raft_consensus.cc:2243] T 91602f30b4464e35b16fa3dcac445b49 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:31.018558 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51d12723f5194afb97d2c05fcbaf3776. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.022486 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 2e33606930774fe4aafc6e3819618af8
I20251211 20:52:31.023921 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 2e33606930774fe4aafc6e3819618af8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:31.021471 881 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fc8d485cc234fd490c487a8bc881bf4 with delete_type TABLET_DATA_DELETED (replaced by tablet 28ef13be3b84410d9c786c43502a3478) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:31.024977 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 2e33606930774fe4aafc6e3819618af8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2e33606930774fe4aafc6e3819618af8 already in progress: creating tablet
I20251211 20:52:31.026078 1164 tablet_replica.cc:333] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:31.037649 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c112714331bf4bd7a1ddcca17f338d4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.039341 1165 raft_consensus.cc:2272] T 91602f30b4464e35b16fa3dcac445b49 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:31.045992 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 91602f30b4464e35b16fa3dcac445b49 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.048861 1165 ts_tablet_manager.cc:1916] T 91602f30b4464e35b16fa3dcac445b49 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:31.050503 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 2e33606930774fe4aafc6e3819618af8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:31.056991 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bfb52935296443c58190d4d06e2e6e9c consensus state has no leader
W20251211 20:52:31.057420 591 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e9ead34065e240eeb1f946319537d32a consensus state has no leader
I20251211 20:52:31.061939 1586 raft_consensus.cc:493] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:31.062033 882 tablet_service.cc:1505] Processing CreateTablet for tablet 51d12723f5194afb97d2c05fcbaf3776 (DEFAULT_TABLE table=default.Table_7c081945e59345a7830dedbfd3f8f3ee [id=ec0bb10933fa4529bef3580bee6409e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.062098 1586 raft_consensus.cc:515] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.062762 1586 leader_election.cc:290] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:31.063652 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "605172f066d242f2abea133c08a145ca" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:31.064252 982 leader_election.cc:343] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:31.065970 1166 tablet_bootstrap.cc:492] T dcdcff8235814b58b3683104a2ff0cbf P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:31.069804 885 tablet_service.cc:1505] Processing CreateTablet for tablet c112714331bf4bd7a1ddcca17f338d4f (DEFAULT_TABLE table=default.Table_c14c424a6284406d968649c9991c7a92 [id=0396911aea9b4a5f860bce4c471e2df7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.070518 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 834b84a557b04e78b5d2c8c84520a644. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.070604 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "605172f066d242f2abea133c08a145ca" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:31.071151 983 leader_election.cc:343] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:31.071314 983 leader_election.cc:304] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:31.071743 1586 raft_consensus.cc:2749] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:31.072254 1166 ts_tablet_manager.cc:1403] T dcdcff8235814b58b3683104a2ff0cbf P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.137s user 0.009s sys 0.000s
I20251211 20:52:31.073047 1168 tablet_bootstrap.cc:492] T f19a83b557e24f929b172d926996ae64 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:31.073642 1168 ts_tablet_manager.cc:1403] T f19a83b557e24f929b172d926996ae64 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.385s user 0.009s sys 0.011s
I20251211 20:52:31.074069 1166 raft_consensus.cc:359] T dcdcff8235814b58b3683104a2ff0cbf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.074499 1166 raft_consensus.cc:385] T dcdcff8235814b58b3683104a2ff0cbf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.074651 1166 raft_consensus.cc:740] T dcdcff8235814b58b3683104a2ff0cbf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.075059 1166 consensus_queue.cc:260] T dcdcff8235814b58b3683104a2ff0cbf P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.076107 1166 ts_tablet_manager.cc:1434] T dcdcff8235814b58b3683104a2ff0cbf P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251211 20:52:31.076797 1166 tablet_bootstrap.cc:492] T d30376baf75b4761971f91da3493a142 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:31.082073 589 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_1c2d691fc52b40dcb94acbe811aefe97 [id=17b2359416c04980857565ab4df594bd] alter complete (version 3)
I20251211 20:52:31.083127 1166 tablet_bootstrap.cc:654] T d30376baf75b4761971f91da3493a142 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.087443 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 605172f066d242f2abea133c08a145ca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:31.087564 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet dcdcff8235814b58b3683104a2ff0cbf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:31.087787 1163 tablet_replica.cc:333] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:31.087993 1163 raft_consensus.cc:2243] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:31.088197 1163 raft_consensus.cc:2272] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:31.089258 1163 ts_tablet_manager.cc:1916] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:31.094142 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f574e04a625b4f00bd2e2e8693a7f496" } modify_external_catalogs: true
I20251211 20:52:31.094358 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f574e04a625b4f00bd2e2e8693a7f496" } modify_external_catalogs: true
I20251211 20:52:31.101287 1013 tablet_service.cc:1505] Processing CreateTablet for tablet 116b21a182474ca191c4a89e9474677d (DEFAULT_TABLE table=default.Table_204dbc031ddb42f38079cbdc40beeff8 [id=3c204174c01446ae80b93ddd0ea6979b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.101763 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 116b21a182474ca191c4a89e9474677d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.103817 1586 raft_consensus.cc:493] T cf1201bba4b6421f8d7ce593f4bd892b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:31.103966 1586 raft_consensus.cc:515] T cf1201bba4b6421f8d7ce593f4bd892b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.104611 1586 leader_election.cc:290] T cf1201bba4b6421f8d7ce593f4bd892b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:31.105425 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cf1201bba4b6421f8d7ce593f4bd892b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:31.106014 982 leader_election.cc:343] T cf1201bba4b6421f8d7ce593f4bd892b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:31.106693 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_37520ef8d4194d8aa3eadef57196a291" } modify_external_catalogs: true
I20251211 20:52:31.106937 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_37520ef8d4194d8aa3eadef57196a291" } modify_external_catalogs: true
I20251211 20:52:31.113022 754 tablet_service.cc:1505] Processing CreateTablet for tablet 116b21a182474ca191c4a89e9474677d (DEFAULT_TABLE table=default.Table_204dbc031ddb42f38079cbdc40beeff8 [id=3c204174c01446ae80b93ddd0ea6979b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.120885 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 116b21a182474ca191c4a89e9474677d. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:31.123135 581 catalog_manager.cc:5058] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): alter failed for tablet 33e1d8a576cd41b1a34778b1458df396 (table default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]),no further retry: Not found: Tablet not found: 33e1d8a576cd41b1a34778b1458df396
I20251211 20:52:31.126683 1164 raft_consensus.cc:2243] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:31.126911 1164 raft_consensus.cc:2272] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:31.129714 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 4df68d1b6a91458e9f212ae81aa25438
I20251211 20:52:31.130388 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 18b977f463d5420186c007b38110e07e
I20251211 20:52:31.130865 878 tablet_service.cc:1552] Processing DeleteTablet for tablet 47779034ad89466e9e1801987128b4dd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.131098 878 tablet_service.cc:1552] Processing DeleteTablet for tablet 71dc8d20d93243e289330d7b8c635511 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.131239 878 tablet_service.cc:1552] Processing DeleteTablet for tablet 2e33606930774fe4aafc6e3819618af8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.131372 888 tablet_service.cc:1505] Processing CreateTablet for tablet d53b074865f4490693152a247b0fffdc (DEFAULT_TABLE table=default.Table_c27589191ac249f4a9460f5bbae2611b [id=461d63ce3b4f4d12b5f395bb8c1cdbb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.134370 847 leader_election.cc:304] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters: e715a811a82446ce8d5bb4aafd5b81c7
W20251211 20:52:31.134732 847 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.1:42177, user_credentials={real_user=slave}} blocked reactor thread for 79558.6us
I20251211 20:52:31.137336 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bd8fd6c977204c90b558827fc0b15f43" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:31.137751 1612 raft_consensus.cc:2804] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:31.137889 1612 raft_consensus.cc:697] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 LEADER]: Becoming Leader. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Running, Role: LEADER
I20251211 20:52:31.138131 1612 consensus_queue.cc:237] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.140213 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 18b977f463d5420186c007b38110e07e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:31.140945 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 18b977f463d5420186c007b38110e07e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.141996 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 4df68d1b6a91458e9f212ae81aa25438 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:31.143172 913 raft_consensus.cc:3060] T 8308ad46551447439a248d20ed1e5922 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:31.144219 1168 raft_consensus.cc:359] T f19a83b557e24f929b172d926996ae64 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:31.144431 1168 raft_consensus.cc:385] T f19a83b557e24f929b172d926996ae64 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.144518 1168 raft_consensus.cc:740] T f19a83b557e24f929b172d926996ae64 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.144789 1168 consensus_queue.cc:260] T f19a83b557e24f929b172d926996ae64 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:31.145268 1168 ts_tablet_manager.cc:1434] T f19a83b557e24f929b172d926996ae64 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.072s user 0.000s sys 0.000s
I20251211 20:52:31.145589 1168 tablet_bootstrap.cc:492] T 0246f1df99684d0da43d97acbf1182a0 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:31.146076 913 raft_consensus.cc:1275] T 8308ad46551447439a248d20ed1e5922 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Refusing update from remote peer 6f8d784fd1cd492bbe3809403ec34919: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251211 20:52:31.146519 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51d12723f5194afb97d2c05fcbaf3776. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.147219 1168 tablet_bootstrap.cc:654] T 0246f1df99684d0da43d97acbf1182a0 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.148160 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c112714331bf4bd7a1ddcca17f338d4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.154712 1163 ts_tablet_manager.cc:1929] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:31.154889 1163 log.cc:1199] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/605172f066d242f2abea133c08a145ca
I20251211 20:52:31.155258 1163 ts_tablet_manager.cc:1950] T 605172f066d242f2abea133c08a145ca P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:31.162189 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d53b074865f4490693152a247b0fffdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.162178 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_05f224658edf4b369163c847787e14bf" } modify_external_catalogs: true
I20251211 20:52:31.162367 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_05f224658edf4b369163c847787e14bf" } modify_external_catalogs: true
I20251211 20:52:31.162981 1164 ts_tablet_manager.cc:1916] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:31.164201 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_c20ffa76ecfa4484a6285b6df74e1247"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:31.164960 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c20ffa76ecfa4484a6285b6df74e1247 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.167120 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 4df68d1b6a91458e9f212ae81aa25438 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.168356 1165 ts_tablet_manager.cc:1929] T 91602f30b4464e35b16fa3dcac445b49 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:31.168557 1160 raft_consensus.cc:359] T 7474a08c51cf4346ae9a518963d56b7a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.168882 1165 log.cc:1199] T 91602f30b4464e35b16fa3dcac445b49 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/91602f30b4464e35b16fa3dcac445b49
I20251211 20:52:31.169273 1165 ts_tablet_manager.cc:1950] T 91602f30b4464e35b16fa3dcac445b49 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:31.170334 757 tablet_service.cc:1552] Processing DeleteTablet for tablet f19a83b557e24f929b172d926996ae64 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.171671 881 tablet_service.cc:1552] Processing DeleteTablet for tablet 95866de91f9d437d80f40b7962253610 with delete_type TABLET_DATA_DELETED (replaced by tablet 51d12723f5194afb97d2c05fcbaf3776) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.172636 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 91602f30b4464e35b16fa3dcac445b49 (table default.Table_afc299e5a13c43878cf6c746698af917 [id=6534b4b4514f4763bcff70c04d93ac7c]) successfully deleted
W20251211 20:52:31.173695 711 leader_election.cc:343] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:31.173851 711 leader_election.cc:304] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:31.173854 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 605172f066d242f2abea133c08a145ca (table default.Table_40b5190e58a440e9a13d6c9221572914 [id=e2b03132f53845138e4fd9731c2d79bf]) successfully deleted
I20251211 20:52:31.174381 1696 raft_consensus.cc:2749] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:31.174603 1696 consensus_queue.cc:1048] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:31.174978 1163 tablet_replica.cc:333] T dcdcff8235814b58b3683104a2ff0cbf P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:31.176050 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9756e2d986ee43839f4462f983927f6d" } new_table_name: "default.Table_60c2303340ad4078a0bf2c796be9faf3" modify_external_catalogs: true
I20251211 20:52:31.176357 1166 tablet_bootstrap.cc:492] T d30376baf75b4761971f91da3493a142 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:31.176510 1166 ts_tablet_manager.cc:1403] T d30376baf75b4761971f91da3493a142 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.100s user 0.003s sys 0.001s
I20251211 20:52:31.177250 1166 raft_consensus.cc:359] T d30376baf75b4761971f91da3493a142 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.177413 1166 raft_consensus.cc:385] T d30376baf75b4761971f91da3493a142 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.177492 1166 raft_consensus.cc:740] T d30376baf75b4761971f91da3493a142 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.177532 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet f25bde14792248b188660036f07ae902
I20251211 20:52:31.177719 1166 consensus_queue.cc:260] T d30376baf75b4761971f91da3493a142 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.178158 1166 ts_tablet_manager.cc:1434] T d30376baf75b4761971f91da3493a142 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:31.178426 1166 tablet_bootstrap.cc:492] T c85624ad640041c29bb87416e1992a9e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:31.180318 1166 tablet_bootstrap.cc:654] T c85624ad640041c29bb87416e1992a9e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.181887 1160 raft_consensus.cc:385] T 7474a08c51cf4346ae9a518963d56b7a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.182443 757 tablet_service.cc:1552] Processing DeleteTablet for tablet f25bde14792248b188660036f07ae902 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.183192 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_fffa8ba66cb04202b2116b8fd4a741de"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:31.184120 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fffa8ba66cb04202b2116b8fd4a741de in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.184446 1165 tablet_replica.cc:333] T d824bc02b25842d18f28f711eef6669f P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:31.184940 1163 raft_consensus.cc:2243] T dcdcff8235814b58b3683104a2ff0cbf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:31.185312 1165 raft_consensus.cc:2243] T d824bc02b25842d18f28f711eef6669f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:31.185684 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_370d3069ceee4b8e8856bd74fe3a1815" } modify_external_catalogs: true
I20251211 20:52:31.185883 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_370d3069ceee4b8e8856bd74fe3a1815" } modify_external_catalogs: true
I20251211 20:52:31.186810 1163 raft_consensus.cc:2272] T dcdcff8235814b58b3683104a2ff0cbf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:31.187073 1165 raft_consensus.cc:2272] T d824bc02b25842d18f28f711eef6669f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:31.188019 1165 ts_tablet_manager.cc:1916] T d824bc02b25842d18f28f711eef6669f P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:31.188627 1163 ts_tablet_manager.cc:1916] T dcdcff8235814b58b3683104a2ff0cbf P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:31.188867 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_c4c8f005f6fe4b7ebcc05d11911718e5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:31.189498 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c4c8f005f6fe4b7ebcc05d11911718e5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.195686 1160 raft_consensus.cc:740] T 7474a08c51cf4346ae9a518963d56b7a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.196035 1160 consensus_queue.cc:260] T 7474a08c51cf4346ae9a518963d56b7a P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.196673 1160 ts_tablet_manager.cc:1434] T 7474a08c51cf4346ae9a518963d56b7a P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.278s user 0.002s sys 0.000s
I20251211 20:52:31.197422 881 tablet_service.cc:1505] Processing CreateTablet for tablet 116b21a182474ca191c4a89e9474677d (DEFAULT_TABLE table=default.Table_204dbc031ddb42f38079cbdc40beeff8 [id=3c204174c01446ae80b93ddd0ea6979b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.199381 1165 ts_tablet_manager.cc:1929] T d824bc02b25842d18f28f711eef6669f P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:31.200532 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet f25bde14792248b188660036f07ae902 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:31.203302 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c8214c97ed1640f5965aa5c4b999e2f1 has no consensus state
I20251211 20:52:31.205809 1163 ts_tablet_manager.cc:1929] T dcdcff8235814b58b3683104a2ff0cbf P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:31.206027 1163 log.cc:1199] T dcdcff8235814b58b3683104a2ff0cbf P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/dcdcff8235814b58b3683104a2ff0cbf
I20251211 20:52:31.206255 1165 log.cc:1199] T d824bc02b25842d18f28f711eef6669f P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/d824bc02b25842d18f28f711eef6669f
I20251211 20:52:31.206480 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 116b21a182474ca191c4a89e9474677d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.206683 1163 ts_tablet_manager.cc:1950] T dcdcff8235814b58b3683104a2ff0cbf P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:31.207674 1165 ts_tablet_manager.cc:1950] T d824bc02b25842d18f28f711eef6669f P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:31.208518 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet d30376baf75b4761971f91da3493a142 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:31.208827 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet d824bc02b25842d18f28f711eef6669f (table default.Table_984c6c545e3a4a2ca496d202198a177b [id=861123c9df6d4841a078e84897b4ce35]) successfully deleted
I20251211 20:52:31.209959 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_349fa502a7c44fe79baed28f7da89ff9" } new_table_name: "default.Table_32ebd46bd3cc460986098c79c9b67ca7" modify_external_catalogs: true
I20251211 20:52:31.210069 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet dcdcff8235814b58b3683104a2ff0cbf (table default.Table_0209a7c0449b42e9ac582543b779f5c7 [id=3aa2f2349ee9431fb9bfb0634e5768b7]) successfully deleted
W20251211 20:52:31.210465 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 18b977f463d5420186c007b38110e07e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 18b977f463d5420186c007b38110e07e already in progress: creating tablet
I20251211 20:52:31.206126 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cf1201bba4b6421f8d7ce593f4bd892b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:31.210899 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_36ad93a3bb49488b87ef36eb462443b4" } new_table_name: "default.Table_99917cb4c70840e19004ad381746d77e" modify_external_catalogs: true
W20251211 20:52:31.210922 983 leader_election.cc:343] T cf1201bba4b6421f8d7ce593f4bd892b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:31.211198 983 leader_election.cc:304] T cf1201bba4b6421f8d7ce593f4bd892b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:31.211412 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_574610b44ec44cbfa5a7071d1956f2f6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:31.211658 1586 raft_consensus.cc:2749] T cf1201bba4b6421f8d7ce593f4bd892b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:31.211958 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_574610b44ec44cbfa5a7071d1956f2f6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.213069 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_1e17ae4fb7e84e41aea563972b80af32"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:31.213636 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e17ae4fb7e84e41aea563972b80af32 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.214787 1165 tablet_replica.cc:333] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:31.215466 1165 raft_consensus.cc:2243] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
W20251211 20:52:31.215907 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 4df68d1b6a91458e9f212ae81aa25438 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4df68d1b6a91458e9f212ae81aa25438 already in progress: creating tablet
I20251211 20:52:31.216065 1165 raft_consensus.cc:2272] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:31.216281 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet f25bde14792248b188660036f07ae902 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f25bde14792248b188660036f07ae902 already in progress: creating tablet
I20251211 20:52:31.216600 1165 ts_tablet_manager.cc:1916] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:31.219882 1163 tablet_replica.cc:333] T d30376baf75b4761971f91da3493a142 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:31.220160 1163 raft_consensus.cc:2243] T d30376baf75b4761971f91da3493a142 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:31.220409 1163 raft_consensus.cc:2272] T d30376baf75b4761971f91da3493a142 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:31.221185 1160 tablet_bootstrap.cc:492] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:31.221201 1163 ts_tablet_manager.cc:1916] T d30376baf75b4761971f91da3493a142 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:31.224957 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cce44b80c97948e1af7ac68e77cccfd6 consensus state has no leader
I20251211 20:52:31.228549 1165 ts_tablet_manager.cc:1929] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20251211 20:52:31.232203 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1843dc4548134f29969d46fbc760f5d7 consensus state has no leader
I20251211 20:52:31.232856 1163 ts_tablet_manager.cc:1929] T d30376baf75b4761971f91da3493a142 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:31.232859 593 master_service.cc:954] ReplaceTablet: received request to replace tablet 8139acb0ae324d0f973ce877edc1cd04 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:31.233090 1165 log.cc:1199] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/bd8fd6c977204c90b558827fc0b15f43
I20251211 20:52:31.233314 1163 log.cc:1199] T d30376baf75b4761971f91da3493a142 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/d30376baf75b4761971f91da3493a142
I20251211 20:52:31.233400 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 18b977f463d5420186c007b38110e07e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.233719 1163 ts_tablet_manager.cc:1950] T d30376baf75b4761971f91da3493a142 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:31.234118 1165 ts_tablet_manager.cc:1950] T bd8fd6c977204c90b558827fc0b15f43 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:31.234715 890 tablet_service.cc:1552] Processing DeleteTablet for tablet f25bde14792248b188660036f07ae902 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.234400 1168 tablet_bootstrap.cc:492] T 0246f1df99684d0da43d97acbf1182a0 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:31.235077 1168 ts_tablet_manager.cc:1403] T 0246f1df99684d0da43d97acbf1182a0 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.090s user 0.006s sys 0.001s
I20251211 20:52:31.235878 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet bd8fd6c977204c90b558827fc0b15f43 (table default.Table_5b8aa62e6fdf46089c4c8ae541856fbe [id=e5caa22bce84422290707816a93bd3f5]) successfully deleted
I20251211 20:52:31.235980 1168 raft_consensus.cc:359] T 0246f1df99684d0da43d97acbf1182a0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.236241 1168 raft_consensus.cc:385] T 0246f1df99684d0da43d97acbf1182a0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.236363 1168 raft_consensus.cc:740] T 0246f1df99684d0da43d97acbf1182a0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.236471 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet d30376baf75b4761971f91da3493a142 (table default.Table_cec7f350a9d04bd590db8a29fb8ceb6b [id=62d2078174bb4b5f952896c6ada6a414]) successfully deleted
I20251211 20:52:31.236619 1168 consensus_queue.cc:260] T 0246f1df99684d0da43d97acbf1182a0 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.237167 1168 ts_tablet_manager.cc:1434] T 0246f1df99684d0da43d97acbf1182a0 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20251211 20:52:31.237231 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 6a206abd6ea2420089cb07714e0d19c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a206abd6ea2420089cb07714e0d19c0 already in progress: creating tablet
I20251211 20:52:31.237548 1168 tablet_bootstrap.cc:492] T 6a206b37e43247b5babe3113eab2c9fb P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:31.239253 1168 tablet_bootstrap.cc:654] T 6a206b37e43247b5babe3113eab2c9fb P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.245103 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 4df68d1b6a91458e9f212ae81aa25438 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:31.247077 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet bb0fc179a337469eae056c5bb3815e6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb0fc179a337469eae056c5bb3815e6b already in progress: creating tablet
W20251211 20:52:31.247772 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 940dfd8781de444a9caa78dd363c7451 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 940dfd8781de444a9caa78dd363c7451 already in progress: creating tablet
I20251211 20:52:31.249053 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 0246f1df99684d0da43d97acbf1182a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.250247 1165 tablet_replica.cc:333] T f19a83b557e24f929b172d926996ae64 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
W20251211 20:52:31.250277 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 95866de91f9d437d80f40b7962253610 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 95866de91f9d437d80f40b7962253610 already in progress: creating tablet
I20251211 20:52:31.250504 1165 raft_consensus.cc:2243] T f19a83b557e24f929b172d926996ae64 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
W20251211 20:52:31.250626 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 71dc8d20d93243e289330d7b8c635511 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 71dc8d20d93243e289330d7b8c635511 already in progress: creating tablet
I20251211 20:52:31.250775 1165 raft_consensus.cc:2272] T f19a83b557e24f929b172d926996ae64 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:31.250836 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 47779034ad89466e9e1801987128b4dd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 47779034ad89466e9e1801987128b4dd already in progress: creating tablet
W20251211 20:52:31.251017 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 2e33606930774fe4aafc6e3819618af8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2e33606930774fe4aafc6e3819618af8 already in progress: creating tablet
W20251211 20:52:31.251207 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 18b977f463d5420186c007b38110e07e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 18b977f463d5420186c007b38110e07e already in progress: creating tablet
W20251211 20:52:31.251408 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 4df68d1b6a91458e9f212ae81aa25438 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4df68d1b6a91458e9f212ae81aa25438 already in progress: creating tablet
I20251211 20:52:31.251832 1165 ts_tablet_manager.cc:1916] T f19a83b557e24f929b172d926996ae64 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:31.257218 1160 tablet_bootstrap.cc:654] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.258956 1165 ts_tablet_manager.cc:1929] T f19a83b557e24f929b172d926996ae64 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:31.259166 1165 log.cc:1199] T f19a83b557e24f929b172d926996ae64 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/f19a83b557e24f929b172d926996ae64
I20251211 20:52:31.259608 1165 ts_tablet_manager.cc:1950] T f19a83b557e24f929b172d926996ae64 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:31.260936 1165 tablet_replica.cc:333] T 0246f1df99684d0da43d97acbf1182a0 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:31.261174 1165 raft_consensus.cc:2243] T 0246f1df99684d0da43d97acbf1182a0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:31.261853 1165 raft_consensus.cc:2272] T 0246f1df99684d0da43d97acbf1182a0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:31.262663 1165 ts_tablet_manager.cc:1916] T 0246f1df99684d0da43d97acbf1182a0 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:31.265874 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet f19a83b557e24f929b172d926996ae64 (table default.Table_4a3a3fe6b0294c939ebc90871cbcb4de [id=a06a0b2452ea4100b09772b53320b537]) successfully deleted
W20251211 20:52:31.266153 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet f25bde14792248b188660036f07ae902 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f25bde14792248b188660036f07ae902 already in progress: creating tablet
I20251211 20:52:31.268836 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_d7d5d2408f644d09b54b10d82a401116"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:31.269017 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_47b57806fd3149c8ab84567d76177fce" } new_table_name: "default.Table_7325cceb834541c895db5f104ae375aa" modify_external_catalogs: true
I20251211 20:52:31.269289 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_78bd2a084ad542c2968d1338f1ac71b5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:31.269490 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d7d5d2408f644d09b54b10d82a401116 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:31.269826 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_78bd2a084ad542c2968d1338f1ac71b5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.270749 1165 ts_tablet_manager.cc:1929] T 0246f1df99684d0da43d97acbf1182a0 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:31.270731 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_e250eb3a69ef4e6da3893852edc00469"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:31.271015 1165 log.cc:1199] T 0246f1df99684d0da43d97acbf1182a0 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/0246f1df99684d0da43d97acbf1182a0
W20251211 20:52:31.271188 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e250eb3a69ef4e6da3893852edc00469 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.271485 1165 ts_tablet_manager.cc:1950] T 0246f1df99684d0da43d97acbf1182a0 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:31.272264 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_2596194e7574455bb855e8ed55b588ac" } modify_external_catalogs: true
I20251211 20:52:31.272408 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_2596194e7574455bb855e8ed55b588ac" } modify_external_catalogs: true
I20251211 20:52:31.272753 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 0246f1df99684d0da43d97acbf1182a0 (table default.Table_03fdaef4fcc144f485f238db0e76fab4 [id=4264180c89854054b14e98d5a49984fa]) successfully deleted
I20251211 20:52:31.272884 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4de1e61d3b1c4ddebf22cdebca28beac" } new_table_name: "default.Table_0042a65523294a70b215f6e796d70cd5" modify_external_catalogs: true
I20251211 20:52:31.273756 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_e891143fbbcd41509b6372a316eaac2c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:31.274125 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e891143fbbcd41509b6372a316eaac2c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.278645 1164 ts_tablet_manager.cc:1929] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:31.278795 1164 log.cc:1199] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/5fc8d485cc234fd490c487a8bc881bf4
I20251211 20:52:31.279122 1164 ts_tablet_manager.cc:1950] T 5fc8d485cc234fd490c487a8bc881bf4 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
W20251211 20:52:31.280051 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c26c3d0170f64a41ae8944c6668cc5f3 consensus state has no leader
I20251211 20:52:31.281194 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 5fc8d485cc234fd490c487a8bc881bf4 (table default.Table_88dcd36b4447481487bc918a05511862 [id=d9453cea23a8430c98a3317856b3cec3]) successfully deleted
I20251211 20:52:31.285904 1164 tablet_replica.cc:333] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:31.287303 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_e7b2a8479479460fa5a4f3c23c6eeee6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:31.288425 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e7b2a8479479460fa5a4f3c23c6eeee6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:31.293218 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 49a33084c0fc460eb9592224b6dfc279 consensus state has no leader
I20251211 20:52:31.300473 1160 tablet_bootstrap.cc:492] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:31.300653 1160 ts_tablet_manager.cc:1403] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.080s user 0.001s sys 0.009s
I20251211 20:52:31.301393 1160 raft_consensus.cc:359] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.301614 1160 raft_consensus.cc:385] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.301684 1160 raft_consensus.cc:740] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.301896 1164 raft_consensus.cc:2243] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:31.308048 1164 pending_rounds.cc:70] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41: Trying to abort 1 pending ops.
I20251211 20:52:31.308147 1164 pending_rounds.cc:77] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432094263787520 op_type: NO_OP noop_request { }
I20251211 20:52:31.308276 1164 raft_consensus.cc:2889] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:31.308374 1164 raft_consensus.cc:2272] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:31.309465 1164 ts_tablet_manager.cc:1916] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:31.312580 1166 tablet_bootstrap.cc:492] T c85624ad640041c29bb87416e1992a9e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:31.312786 1166 ts_tablet_manager.cc:1403] T c85624ad640041c29bb87416e1992a9e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.134s user 0.003s sys 0.003s
I20251211 20:52:31.313612 1166 raft_consensus.cc:359] T c85624ad640041c29bb87416e1992a9e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.313745 1704 mvcc.cc:204] Tried to move back new op lower bound from 7231432093048107008 to 7231432089404981248. Current Snapshot: MvccSnapshot[applied={T|T < 7231432093048107008}]
I20251211 20:52:31.313810 1166 raft_consensus.cc:385] T c85624ad640041c29bb87416e1992a9e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.313874 1166 raft_consensus.cc:740] T c85624ad640041c29bb87416e1992a9e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.314095 1166 consensus_queue.cc:260] T c85624ad640041c29bb87416e1992a9e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.314567 1166 ts_tablet_manager.cc:1434] T c85624ad640041c29bb87416e1992a9e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:31.315268 1166 tablet_bootstrap.cc:492] T 1843dc4548134f29969d46fbc760f5d7 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:31.315800 1160 consensus_queue.cc:260] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.316848 1166 tablet_bootstrap.cc:654] T 1843dc4548134f29969d46fbc760f5d7 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.319939 1708 tablet.cc:1722] T 8308ad46551447439a248d20ed1e5922 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:31.322000 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet c85624ad640041c29bb87416e1992a9e with delete_type TABLET_DATA_DELETED (replaced by tablet fa79b67b9eee45e2b41c4a90967606cc) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:31.322270 1163 tablet_replica.cc:333] T c85624ad640041c29bb87416e1992a9e P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:31.322474 1163 raft_consensus.cc:2243] T c85624ad640041c29bb87416e1992a9e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:31.322649 1163 raft_consensus.cc:2272] T c85624ad640041c29bb87416e1992a9e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:31.326476 1695 consensus_queue.cc:1048] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:31.330330 1160 ts_tablet_manager.cc:1434] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.030s user 0.001s sys 0.002s
I20251211 20:52:31.331194 1160 tablet_bootstrap.cc:492] T ed9ee283045a44978422cd080aa2cda5 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:31.333968 1160 tablet_bootstrap.cc:654] T ed9ee283045a44978422cd080aa2cda5 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.335700 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 28ef13be3b84410d9c786c43502a3478 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:31.336483 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b23d9e7bfa424f92a22e13faaeba571f" } modify_external_catalogs: true
I20251211 20:52:31.336588 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_59b6bdb8884646d49dc69e9b618915c8" } modify_external_catalogs: true
I20251211 20:52:31.336678 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b23d9e7bfa424f92a22e13faaeba571f" } modify_external_catalogs: true
I20251211 20:52:31.336727 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_59b6bdb8884646d49dc69e9b618915c8" } modify_external_catalogs: true
I20251211 20:52:31.337960 1695 consensus_queue.cc:1048] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:31.344612 1168 tablet_bootstrap.cc:492] T 6a206b37e43247b5babe3113eab2c9fb P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:31.345007 1168 ts_tablet_manager.cc:1403] T 6a206b37e43247b5babe3113eab2c9fb P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.108s user 0.004s sys 0.011s
I20251211 20:52:31.346199 1168 raft_consensus.cc:359] T 6a206b37e43247b5babe3113eab2c9fb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.347813 1168 raft_consensus.cc:385] T 6a206b37e43247b5babe3113eab2c9fb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.348258 1168 raft_consensus.cc:740] T 6a206b37e43247b5babe3113eab2c9fb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.348296 885 tablet_service.cc:1505] Processing CreateTablet for tablet c8214c97ed1640f5965aa5c4b999e2f1 (DEFAULT_TABLE table=default.Table_60c2303340ad4078a0bf2c796be9faf3 [id=b3fb50681b1b47d3b2c158fee448724b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.349156 1168 consensus_queue.cc:260] T 6a206b37e43247b5babe3113eab2c9fb P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.351392 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 993616e226274c229a5924fbe69911af
I20251211 20:52:31.352241 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 5fd2a6cf80a04f20aef8ff9c84bc88db
I20251211 20:52:31.353127 1168 ts_tablet_manager.cc:1434] T 6a206b37e43247b5babe3113eab2c9fb P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20251211 20:52:31.354157 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 7b06cd366de5494780b5ac81abbf0702
I20251211 20:52:31.354760 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 29463a8f71f943a0927942235af3c2da
I20251211 20:52:31.354060 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 993616e226274c229a5924fbe69911af with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.355715 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 993616e226274c229a5924fbe69911af with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:31.355840 1168 tablet_bootstrap.cc:492] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:31.357566 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fd2a6cf80a04f20aef8ff9c84bc88db with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.357671 1168 tablet_bootstrap.cc:654] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.358559 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 29463a8f71f943a0927942235af3c2da with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.358819 881 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fd2a6cf80a04f20aef8ff9c84bc88db with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:31.359229 581 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 49a33084c0fc460eb9592224b6dfc279 (table default.Table_0042a65523294a70b215f6e796d70cd5 [id=9c24845e4efe475fa792a51ce8a66be6], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 49a33084c0fc460eb9592224b6dfc279 consensus state has no leader
I20251211 20:52:31.361574 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fd2a6cf80a04f20aef8ff9c84bc88db with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:31.361922 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 7b06cd366de5494780b5ac81abbf0702 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.362211 882 tablet_service.cc:1552] Processing DeleteTablet for tablet 993616e226274c229a5924fbe69911af with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.362208 595 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 8308ad46551447439a248d20ed1e5922 (table default.Table_1c2d691fc52b40dcb94acbe811aefe97 [id=17b2359416c04980857565ab4df594bd]). Expected version 3 got 0
I20251211 20:52:31.363356 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8214c97ed1640f5965aa5c4b999e2f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.367681 882 tablet_service.cc:1505] Processing CreateTablet for tablet 062f980b65504d3baf99ae400e521775 (DEFAULT_TABLE table=default.Table_c6fbe03584554d35ae13e59b8d8f0e70 [id=30620a01c27f456bac05fa7c02a2cb4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.368124 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 062f980b65504d3baf99ae400e521775. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.372135 888 tablet_service.cc:1505] Processing CreateTablet for tablet 29463a8f71f943a0927942235af3c2da (DEFAULT_TABLE table=default.Table_59b6bdb8884646d49dc69e9b618915c8 [id=b6b3af92db7d47fa8baa1c1f6b68288e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.372545 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29463a8f71f943a0927942235af3c2da. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.372964 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 3c578d81e8dd41aaa0f707c4e3fed225 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:31.375902 1586 raft_consensus.cc:493] T ae1abef61473443b9922eb13f2ae92fa P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:31.376070 1586 raft_consensus.cc:515] T ae1abef61473443b9922eb13f2ae92fa P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.376720 1586 leader_election.cc:290] T ae1abef61473443b9922eb13f2ae92fa P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:31.377876 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ae1abef61473443b9922eb13f2ae92fa" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:31.379890 982 leader_election.cc:343] T ae1abef61473443b9922eb13f2ae92fa P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:31.381134 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 7b06cd366de5494780b5ac81abbf0702 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.382256 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 3c578d81e8dd41aaa0f707c4e3fed225
I20251211 20:52:31.382727 595 catalog_manager.cc:6724] ReplaceTablet: tablet 3c578d81e8dd41aaa0f707c4e3fed225 deleted and replaced by tablet b115d737d7fc483d9f5a1f040b511bae
I20251211 20:52:31.383514 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 7b06cd366de5494780b5ac81abbf0702 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:31.383886 711 consensus_peers.cc:597] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:31.385030 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_10e55a82b5d1413b8b68da9df264da06" } modify_external_catalogs: true
I20251211 20:52:31.385010 893 tablet_service.cc:1505] Processing CreateTablet for tablet 683d0f2068fb4a6ebc9c169ad7b455ce (DEFAULT_TABLE table=default.Table_7e2c82ae1dbe44078774e5229684130d [id=a046f539a58a4c279d461cf9510c97d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.385200 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_10e55a82b5d1413b8b68da9df264da06" } modify_external_catalogs: true
I20251211 20:52:31.385430 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 683d0f2068fb4a6ebc9c169ad7b455ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.387871 881 tablet_service.cc:1552] Processing DeleteTablet for tablet 3c578d81e8dd41aaa0f707c4e3fed225 with delete_type TABLET_DATA_DELETED (replaced by tablet b115d737d7fc483d9f5a1f040b511bae) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.389199 1160 tablet_bootstrap.cc:492] T ed9ee283045a44978422cd080aa2cda5 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:31.389408 1160 ts_tablet_manager.cc:1403] T ed9ee283045a44978422cd080aa2cda5 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.058s user 0.002s sys 0.006s
I20251211 20:52:31.390271 1160 raft_consensus.cc:359] T ed9ee283045a44978422cd080aa2cda5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.390501 1160 raft_consensus.cc:385] T ed9ee283045a44978422cd080aa2cda5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.390604 1160 raft_consensus.cc:740] T ed9ee283045a44978422cd080aa2cda5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.390868 1160 consensus_queue.cc:260] T ed9ee283045a44978422cd080aa2cda5 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.393035 1160 ts_tablet_manager.cc:1434] T ed9ee283045a44978422cd080aa2cda5 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:31.394078 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 29463a8f71f943a0927942235af3c2da with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.394804 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 260bdeb73439437cbf234d20f7264ab3
I20251211 20:52:31.394831 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 29463a8f71f943a0927942235af3c2da with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:31.395259 581 catalog_manager.cc:4952] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 993616e226274c229a5924fbe69911af because the tablet was not found. No further retry: Not found: Tablet not found: 993616e226274c229a5924fbe69911af
I20251211 20:52:31.396015 881 tablet_service.cc:1552] Processing DeleteTablet for tablet 260bdeb73439437cbf234d20f7264ab3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.397729 881 tablet_service.cc:1505] Processing CreateTablet for tablet 4388893262b54ad8be1bdf82516031a5 (DEFAULT_TABLE table=default.Table_13a8009dbcc342d98426f71c053a964a [id=eb64e2c8b06a44afbb4aa2c4df0cac69]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.397895 1168 tablet_bootstrap.cc:492] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:31.398080 1168 ts_tablet_manager.cc:1403] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.042s user 0.004s sys 0.001s
I20251211 20:52:31.398906 1168 raft_consensus.cc:359] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.399514 1168 raft_consensus.cc:385] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.399652 1168 raft_consensus.cc:740] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.400043 1160 tablet_bootstrap.cc:492] T f19a83b557e24f929b172d926996ae64 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:31.402292 1160 tablet_bootstrap.cc:654] T f19a83b557e24f929b172d926996ae64 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.403388 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4388893262b54ad8be1bdf82516031a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.404062 879 tablet_service.cc:1505] Processing CreateTablet for tablet c549c2bfebd64b87a4fc8ce8210e14f6 (DEFAULT_TABLE table=default.Table_06f4a9c6bfcd4fac94df56806a29ec03 [id=bd10f854725745eebefbb6269a5d9450]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.404429 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c549c2bfebd64b87a4fc8ce8210e14f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.408948 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_f14b182e182d40afa389182bafa08912"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:31.409515 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_06bc050f989a4d97a4f76d09b835932d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:31.410323 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f14b182e182d40afa389182bafa08912 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.410482 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_6310a6161f1f4def90affd0ea6fc67d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:31.411005 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6310a6161f1f4def90affd0ea6fc67d2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:31.412142 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 5fd2a6cf80a04f20aef8ff9c84bc88db because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5fd2a6cf80a04f20aef8ff9c84bc88db already in progress: creating tablet
W20251211 20:52:31.412236 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_06bc050f989a4d97a4f76d09b835932d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.412747 595 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_547385bc5ed1465ea82e0328b72f83c0" } new_table_name: "default.Table_f84cd3de863b446894133bb40aaf1a39" modify_external_catalogs: true
I20251211 20:52:31.414093 1168 consensus_queue.cc:260] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.415099 1168 ts_tablet_manager.cc:1434] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.017s user 0.001s sys 0.001s
I20251211 20:52:31.416395 1168 tablet_bootstrap.cc:492] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:31.421087 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 7b06cd366de5494780b5ac81abbf0702 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7b06cd366de5494780b5ac81abbf0702 already in progress: creating tablet
W20251211 20:52:31.421604 581 catalog_manager.cc:4952] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 29463a8f71f943a0927942235af3c2da because the tablet was not found. No further retry: Not found: Tablet not found: 29463a8f71f943a0927942235af3c2da
I20251211 20:52:31.422003 1612 raft_consensus.cc:493] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:31.422178 1612 raft_consensus.cc:515] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:31.423738 847 leader_election.cc:343] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: d824bc02b25842d18f28f711eef6669f
I20251211 20:52:31.423878 1168 tablet_bootstrap.cc:654] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.424616 1612 leader_election.cc:290] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:31.425007 847 leader_election.cc:343] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: d824bc02b25842d18f28f711eef6669f
I20251211 20:52:31.425138 847 leader_election.cc:304] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:31.425395 1616 raft_consensus.cc:2749] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:31.426311 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ae1abef61473443b9922eb13f2ae92fa" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:31.428443 1168 tablet_bootstrap.cc:492] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:31.428661 1168 ts_tablet_manager.cc:1403] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.002s
I20251211 20:52:31.429482 1168 raft_consensus.cc:359] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.429713 1168 raft_consensus.cc:385] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.429816 1168 raft_consensus.cc:740] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
W20251211 20:52:31.430583 983 leader_election.cc:343] T ae1abef61473443b9922eb13f2ae92fa P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:31.430563 1168 consensus_queue.cc:260] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.430748 983 leader_election.cc:304] T ae1abef61473443b9922eb13f2ae92fa P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:31.431082 1168 ts_tablet_manager.cc:1434] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:31.431290 1586 raft_consensus.cc:2749] T ae1abef61473443b9922eb13f2ae92fa P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:31.433193 754 tablet_service.cc:1505] Processing CreateTablet for tablet c8214c97ed1640f5965aa5c4b999e2f1 (DEFAULT_TABLE table=default.Table_60c2303340ad4078a0bf2c796be9faf3 [id=b3fb50681b1b47d3b2c158fee448724b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.433637 757 tablet_service.cc:1505] Processing CreateTablet for tablet 062f980b65504d3baf99ae400e521775 (DEFAULT_TABLE table=default.Table_c6fbe03584554d35ae13e59b8d8f0e70 [id=30620a01c27f456bac05fa7c02a2cb4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.434023 746 tablet_service.cc:1505] Processing CreateTablet for tablet 29463a8f71f943a0927942235af3c2da (DEFAULT_TABLE table=default.Table_59b6bdb8884646d49dc69e9b618915c8 [id=b6b3af92db7d47fa8baa1c1f6b68288e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.434881 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 062f980b65504d3baf99ae400e521775 (DEFAULT_TABLE table=default.Table_c6fbe03584554d35ae13e59b8d8f0e70 [id=30620a01c27f456bac05fa7c02a2cb4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.435004 1028 tablet_service.cc:1505] Processing CreateTablet for tablet 29463a8f71f943a0927942235af3c2da (DEFAULT_TABLE table=default.Table_59b6bdb8884646d49dc69e9b618915c8 [id=b6b3af92db7d47fa8baa1c1f6b68288e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.435308 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 062f980b65504d3baf99ae400e521775. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.435353 748 tablet_service.cc:1552] Processing DeleteTablet for tablet 3c578d81e8dd41aaa0f707c4e3fed225 with delete_type TABLET_DATA_DELETED (replaced by tablet b115d737d7fc483d9f5a1f040b511bae) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.435616 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29463a8f71f943a0927942235af3c2da. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.439016 740 tablet_service.cc:1505] Processing CreateTablet for tablet 683d0f2068fb4a6ebc9c169ad7b455ce (DEFAULT_TABLE table=default.Table_7e2c82ae1dbe44078774e5229684130d [id=a046f539a58a4c279d461cf9510c97d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.439743 745 tablet_service.cc:1505] Processing CreateTablet for tablet 4388893262b54ad8be1bdf82516031a5 (DEFAULT_TABLE table=default.Table_13a8009dbcc342d98426f71c053a964a [id=eb64e2c8b06a44afbb4aa2c4df0cac69]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.440026 748 tablet_service.cc:1552] Processing DeleteTablet for tablet 260bdeb73439437cbf234d20f7264ab3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.440800 1013 tablet_service.cc:1505] Processing CreateTablet for tablet c8214c97ed1640f5965aa5c4b999e2f1 (DEFAULT_TABLE table=default.Table_60c2303340ad4078a0bf2c796be9faf3 [id=b3fb50681b1b47d3b2c158fee448724b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.441186 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8214c97ed1640f5965aa5c4b999e2f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.441383 1168 tablet_bootstrap.cc:492] T c4e048ca28824329b924845ccf5067cf P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:31.441699 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_02295a0d7c9c43bab5f8b6db77d27dd1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:31.442267 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_02295a0d7c9c43bab5f8b6db77d27dd1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.444092 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_468ef643730a494baa1b40f6e81b0a11" } new_table_name: "default.Table_b40488a7667a4d79a917e877f21cee86" modify_external_catalogs: true
I20251211 20:52:31.444706 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29463a8f71f943a0927942235af3c2da. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.447543 1168 tablet_bootstrap.cc:654] T c4e048ca28824329b924845ccf5067cf P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.447701 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_734c2082e7ad4fd4a10082f89e767abe" } new_table_name: "default.Table_dffb2ba445c0475e96e343895b21cb51" modify_external_catalogs: true
W20251211 20:52:31.447850 595 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 59e955aa07c14fbbb347b88eca59a74e consensus state has no leader
I20251211 20:52:31.449146 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 062f980b65504d3baf99ae400e521775. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.452014 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4388893262b54ad8be1bdf82516031a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.454573 748 tablet_service.cc:1505] Processing CreateTablet for tablet c549c2bfebd64b87a4fc8ce8210e14f6 (DEFAULT_TABLE table=default.Table_06f4a9c6bfcd4fac94df56806a29ec03 [id=bd10f854725745eebefbb6269a5d9450]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.454972 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c549c2bfebd64b87a4fc8ce8210e14f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.455832 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 683d0f2068fb4a6ebc9c169ad7b455ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.466068 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8214c97ed1640f5965aa5c4b999e2f1. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:31.469671 591 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8229b704e7464b30b7d2902a938ce33b consensus state has no leader
I20251211 20:52:31.471205 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_24737d22fc454a3da12979ae8966c9b3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:31.471987 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_24737d22fc454a3da12979ae8966c9b3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.483124 881 tablet_service.cc:1505] Processing CreateTablet for tablet 66cdc6f297b24cfba3a3a0ae9f45d12d (DEFAULT_TABLE table=default.Table_f516a45c0526407c96ee48db2945eb80 [id=c4ab744662c44bd088a61c3eed381f56]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.484655 877 tablet_service.cc:1505] Processing CreateTablet for tablet 59e955aa07c14fbbb347b88eca59a74e (DEFAULT_TABLE table=default.Table_f84cd3de863b446894133bb40aaf1a39 [id=f4d608cc41d0408486d3f6b0a09383a4]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:31.487836 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0f046ea2637e4a5583b7969e19a605d5 consensus state has no leader
I20251211 20:52:31.488933 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_fbe890176f3d42768b00d57c928f38c4" } new_table_name: "default.Table_d571120dedcb48cd9d4b75314be3a654" modify_external_catalogs: true
I20251211 20:52:31.491101 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_c2f02a2463774499a7858b0a53ee965d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:31.492029 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_e887667aa38d47e9814a79f7b0e6990d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:31.492516 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e887667aa38d47e9814a79f7b0e6990d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:31.493605 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c2f02a2463774499a7858b0a53ee965d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.495455 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_740dd6a3832643c49f5f2bf34e924c41" } new_table_name: "default.Table_c504f8f3796f43b697488bd604268543" modify_external_catalogs: true
I20251211 20:52:31.495569 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_8488ced67fe54a25918bbd9c8c9934e8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:31.496099 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8488ced67fe54a25918bbd9c8c9934e8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.502391 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_4e49f78c60234aea972f8f5068e89b9f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:31.503086 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4e49f78c60234aea972f8f5068e89b9f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:31.508522 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3eac21a07ba34e1fa275e25ee9dcabb1 consensus state has no leader
I20251211 20:52:31.512243 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_8aecd4725c9d43a2b561e3e917e463c6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:31.512948 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8aecd4725c9d43a2b561e3e917e463c6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.515974 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66cdc6f297b24cfba3a3a0ae9f45d12d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.522631 1160 tablet_bootstrap.cc:492] T f19a83b557e24f929b172d926996ae64 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:31.522994 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59e955aa07c14fbbb347b88eca59a74e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.524222 1160 ts_tablet_manager.cc:1403] T f19a83b557e24f929b172d926996ae64 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.123s user 0.003s sys 0.004s
I20251211 20:52:31.525439 889 tablet_service.cc:1505] Processing CreateTablet for tablet 993616e226274c229a5924fbe69911af (DEFAULT_TABLE table=default.Table_2596194e7574455bb855e8ed55b588ac [id=ed55e87252c44bdb82bb23523326a893]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.526595 1160 raft_consensus.cc:359] T f19a83b557e24f929b172d926996ae64 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:31.527462 1160 raft_consensus.cc:385] T f19a83b557e24f929b172d926996ae64 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.527565 1160 raft_consensus.cc:740] T f19a83b557e24f929b172d926996ae64 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.527860 1160 consensus_queue.cc:260] T f19a83b557e24f929b172d926996ae64 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:31.528376 1160 ts_tablet_manager.cc:1434] T f19a83b557e24f929b172d926996ae64 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20251211 20:52:31.528702 1160 tablet_bootstrap.cc:492] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:31.534509 1164 ts_tablet_manager.cc:1929] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:31.534693 1164 log.cc:1199] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/91602f30b4464e35b16fa3dcac445b49
I20251211 20:52:31.534967 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a206b37e43247b5babe3113eab2c9fb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.535133 1164 ts_tablet_manager.cc:1950] T 91602f30b4464e35b16fa3dcac445b49 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:31.535290 1165 tablet_replica.cc:333] T 6a206b37e43247b5babe3113eab2c9fb P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:31.535562 1165 raft_consensus.cc:2243] T 6a206b37e43247b5babe3113eab2c9fb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:31.535558 588 master_service.cc:954] ReplaceTablet: received request to replace tablet 1578887b2f5748cd83a6fbdec2f3fc99 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:31.535773 1165 raft_consensus.cc:2272] T 6a206b37e43247b5babe3113eab2c9fb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:31.538398 1165 ts_tablet_manager.cc:1916] T 6a206b37e43247b5babe3113eab2c9fb P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:31.538421 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 993616e226274c229a5924fbe69911af. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.549443 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 91602f30b4464e35b16fa3dcac445b49 (table default.Table_afc299e5a13c43878cf6c746698af917 [id=6534b4b4514f4763bcff70c04d93ac7c]) successfully deleted
W20251211 20:52:31.549979 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 47779034ad89466e9e1801987128b4dd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 47779034ad89466e9e1801987128b4dd already in progress: creating tablet
I20251211 20:52:31.550130 1512 raft_consensus.cc:493] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:31.550302 1512 raft_consensus.cc:515] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:31.550508 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 71dc8d20d93243e289330d7b8c635511 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 71dc8d20d93243e289330d7b8c635511 already in progress: creating tablet
W20251211 20:52:31.550864 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 2e33606930774fe4aafc6e3819618af8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2e33606930774fe4aafc6e3819618af8 already in progress: creating tablet
I20251211 20:52:31.551003 1512 leader_election.cc:290] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:31.551249 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 95866de91f9d437d80f40b7962253610 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 95866de91f9d437d80f40b7962253610 already in progress: creating tablet
W20251211 20:52:31.551607 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 18b977f463d5420186c007b38110e07e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 18b977f463d5420186c007b38110e07e already in progress: creating tablet
W20251211 20:52:31.551968 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet f25bde14792248b188660036f07ae902 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f25bde14792248b188660036f07ae902 already in progress: creating tablet
W20251211 20:52:31.552321 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 4df68d1b6a91458e9f212ae81aa25438 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4df68d1b6a91458e9f212ae81aa25438 already in progress: creating tablet
W20251211 20:52:31.552423 710 leader_election.cc:343] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 78922f26cedb4bb989aa1c1cb40bfa3a
W20251211 20:52:31.552711 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 29463a8f71f943a0927942235af3c2da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 29463a8f71f943a0927942235af3c2da already in progress: creating tablet
I20251211 20:52:31.552770 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "78922f26cedb4bb989aa1c1cb40bfa3a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:31.553061 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 5fd2a6cf80a04f20aef8ff9c84bc88db because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5fd2a6cf80a04f20aef8ff9c84bc88db already in progress: creating tablet
W20251211 20:52:31.553373 711 leader_election.cc:343] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:31.553445 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 7b06cd366de5494780b5ac81abbf0702 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7b06cd366de5494780b5ac81abbf0702 already in progress: creating tablet
I20251211 20:52:31.553514 711 leader_election.cc:304] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:31.553805 582 catalog_manager.cc:4952] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 993616e226274c229a5924fbe69911af because the tablet was not found. No further retry: Not found: Tablet not found: 993616e226274c229a5924fbe69911af
I20251211 20:52:31.553920 1512 raft_consensus.cc:2749] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:31.554745 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 3c578d81e8dd41aaa0f707c4e3fed225 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3c578d81e8dd41aaa0f707c4e3fed225 already in progress: creating tablet
W20251211 20:52:31.555191 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 260bdeb73439437cbf234d20f7264ab3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 260bdeb73439437cbf234d20f7264ab3 already in progress: creating tablet
I20251211 20:52:31.558624 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4813c06395dd41c78eceebb97804cdad" } modify_external_catalogs: true
I20251211 20:52:31.556775 754 tablet_service.cc:1505] Processing CreateTablet for tablet 66cdc6f297b24cfba3a3a0ae9f45d12d (DEFAULT_TABLE table=default.Table_f516a45c0526407c96ee48db2945eb80 [id=c4ab744662c44bd088a61c3eed381f56]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.559427 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_5e842888ea5549bb8f88b2bd74e1cfc8" } modify_external_catalogs: true
I20251211 20:52:31.559767 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66cdc6f297b24cfba3a3a0ae9f45d12d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.560617 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_99d08ab1dfe14c2c9be51f4f6d6498db" } modify_external_catalogs: true
I20251211 20:52:31.558542 748 tablet_service.cc:1505] Processing CreateTablet for tablet 59e955aa07c14fbbb347b88eca59a74e (DEFAULT_TABLE table=default.Table_f84cd3de863b446894133bb40aaf1a39 [id=f4d608cc41d0408486d3f6b0a09383a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.563673 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59e955aa07c14fbbb347b88eca59a74e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.559279 740 tablet_service.cc:1505] Processing CreateTablet for tablet 993616e226274c229a5924fbe69911af (DEFAULT_TABLE table=default.Table_2596194e7574455bb855e8ed55b588ac [id=ed55e87252c44bdb82bb23523326a893]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:31.565286 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 3c578d81e8dd41aaa0f707c4e3fed225 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3c578d81e8dd41aaa0f707c4e3fed225 already in progress: creating tablet
W20251211 20:52:31.565670 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 260bdeb73439437cbf234d20f7264ab3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 260bdeb73439437cbf234d20f7264ab3 already in progress: creating tablet
I20251211 20:52:31.568289 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 993616e226274c229a5924fbe69911af. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.575419 1163 ts_tablet_manager.cc:1916] T c85624ad640041c29bb87416e1992a9e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:31.577697 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 3c578d81e8dd41aaa0f707c4e3fed225 with delete_type TABLET_DATA_DELETED (replaced by tablet b115d737d7fc483d9f5a1f040b511bae) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:31.581985 1160 tablet_bootstrap.cc:654] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.582496 590 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet dd425601d89148eb95525f796a3abf1e (table default.Table_d4afd6ed615a41ac8f6714c8aac27857 [id=c842792b01cb4c689fcfd3d796151508]). Expected version 1 got 0
I20251211 20:52:31.583667 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_99d08ab1dfe14c2c9be51f4f6d6498db" } modify_external_catalogs: true
I20251211 20:52:31.584870 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_5e842888ea5549bb8f88b2bd74e1cfc8" } modify_external_catalogs: true
I20251211 20:52:31.586340 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_204dbc031ddb42f38079cbdc40beeff8" } modify_external_catalogs: true
I20251211 20:52:31.586484 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_204dbc031ddb42f38079cbdc40beeff8" } modify_external_catalogs: true
I20251211 20:52:31.587476 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4813c06395dd41c78eceebb97804cdad" } modify_external_catalogs: true
I20251211 20:52:31.590998 590 master_service.cc:954] ReplaceTablet: received request to replace tablet fa79b67b9eee45e2b41c4a90967606cc from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:31.591629 888 tablet_service.cc:1552] Processing DeleteTablet for tablet d824bc02b25842d18f28f711eef6669f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.595788 1164 tablet_replica.cc:333] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:31.596016 1164 raft_consensus.cc:2243] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:31.596226 1164 raft_consensus.cc:2272] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:31.597281 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c20ffa76ecfa4484a6285b6df74e1247" } modify_external_catalogs: true
I20251211 20:52:31.597433 1164 ts_tablet_manager.cc:1916] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:31.597484 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c20ffa76ecfa4484a6285b6df74e1247" } modify_external_catalogs: true
I20251211 20:52:31.597951 893 tablet_service.cc:1552] Processing DeleteTablet for tablet ed9ee283045a44978422cd080aa2cda5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.603724 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 260bdeb73439437cbf234d20f7264ab3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:31.605178 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 683d0f2068fb4a6ebc9c169ad7b455ce (DEFAULT_TABLE table=default.Table_7e2c82ae1dbe44078774e5229684130d [id=a046f539a58a4c279d461cf9510c97d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.605664 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 683d0f2068fb4a6ebc9c169ad7b455ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.589408 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 4388893262b54ad8be1bdf82516031a5 (DEFAULT_TABLE table=default.Table_13a8009dbcc342d98426f71c053a964a [id=eb64e2c8b06a44afbb4aa2c4df0cac69]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.609859 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4388893262b54ad8be1bdf82516031a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.613760 1018 tablet_service.cc:1505] Processing CreateTablet for tablet 59e955aa07c14fbbb347b88eca59a74e (DEFAULT_TABLE table=default.Table_f84cd3de863b446894133bb40aaf1a39 [id=f4d608cc41d0408486d3f6b0a09383a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.614393 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 66cdc6f297b24cfba3a3a0ae9f45d12d (DEFAULT_TABLE table=default.Table_f516a45c0526407c96ee48db2945eb80 [id=c4ab744662c44bd088a61c3eed381f56]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.617107 1026 tablet_service.cc:1505] Processing CreateTablet for tablet c549c2bfebd64b87a4fc8ce8210e14f6 (DEFAULT_TABLE table=default.Table_06f4a9c6bfcd4fac94df56806a29ec03 [id=bd10f854725745eebefbb6269a5d9450]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.618214 597 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet dd425601d89148eb95525f796a3abf1e (table default.Table_d4afd6ed615a41ac8f6714c8aac27857 [id=c842792b01cb4c689fcfd3d796151508]). Expected version 1 got 0
I20251211 20:52:31.619891 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 993616e226274c229a5924fbe69911af (DEFAULT_TABLE table=default.Table_2596194e7574455bb855e8ed55b588ac [id=ed55e87252c44bdb82bb23523326a893]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.621356 597 master_service.cc:954] ReplaceTablet: received request to replace tablet 3f1840e4b5214ddb878935a79b214eae from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:31.623399 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59e955aa07c14fbbb347b88eca59a74e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.623713 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c549c2bfebd64b87a4fc8ce8210e14f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.624437 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66cdc6f297b24cfba3a3a0ae9f45d12d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.625106 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 993616e226274c229a5924fbe69911af. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.630689 1164 ts_tablet_manager.cc:1929] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:31.630896 1164 log.cc:1199] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/d824bc02b25842d18f28f711eef6669f
I20251211 20:52:31.631887 1164 ts_tablet_manager.cc:1950] T d824bc02b25842d18f28f711eef6669f P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
W20251211 20:52:31.646166 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2d849c62f7434099a16f62c2a2fc5e85 consensus state has no leader
I20251211 20:52:31.651322 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_4e2726de74fd437796c8e0e6dd8b3af0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:31.651911 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4e2726de74fd437796c8e0e6dd8b3af0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.655282 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_fde35da450a941b38489c3954c1a3839"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:31.656031 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fde35da450a941b38489c3954c1a3839 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.581497 1165 ts_tablet_manager.cc:1929] T 6a206b37e43247b5babe3113eab2c9fb P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:31.668759 1165 log.cc:1199] T 6a206b37e43247b5babe3113eab2c9fb P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/6a206b37e43247b5babe3113eab2c9fb
I20251211 20:52:31.669122 1165 ts_tablet_manager.cc:1950] T 6a206b37e43247b5babe3113eab2c9fb P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:31.670322 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 6a206b37e43247b5babe3113eab2c9fb (table default.Table_028df988934346debc0f8c62b4740309 [id=e1949419f3924474bf195ac17a794497]) successfully deleted
I20251211 20:52:31.653388 889 tablet_service.cc:1552] Processing DeleteTablet for tablet f19a83b557e24f929b172d926996ae64 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.658737 912 raft_consensus.cc:3060] T 7474a08c51cf4346ae9a518963d56b7a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:31.674682 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet d824bc02b25842d18f28f711eef6669f (table default.Table_984c6c545e3a4a2ca496d202198a177b [id=861123c9df6d4841a078e84897b4ce35]) successfully deleted
I20251211 20:52:31.674988 1164 tablet_replica.cc:333] T ed9ee283045a44978422cd080aa2cda5 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:31.675220 1164 raft_consensus.cc:2243] T ed9ee283045a44978422cd080aa2cda5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:31.675420 1164 raft_consensus.cc:2272] T ed9ee283045a44978422cd080aa2cda5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:31.676240 1164 ts_tablet_manager.cc:1916] T ed9ee283045a44978422cd080aa2cda5 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:31.677763 1696 consensus_queue.cc:1048] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:31.681457 1163 ts_tablet_manager.cc:1929] T c85624ad640041c29bb87416e1992a9e P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:31.681680 1163 log.cc:1199] T c85624ad640041c29bb87416e1992a9e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/c85624ad640041c29bb87416e1992a9e
I20251211 20:52:31.633046 1168 tablet_bootstrap.cc:492] T c4e048ca28824329b924845ccf5067cf P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:31.683111 1168 ts_tablet_manager.cc:1403] T c4e048ca28824329b924845ccf5067cf P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.242s user 0.005s sys 0.023s
I20251211 20:52:31.683863 1163 ts_tablet_manager.cc:1950] T c85624ad640041c29bb87416e1992a9e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:31.684172 1168 raft_consensus.cc:359] T c4e048ca28824329b924845ccf5067cf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.684362 1168 raft_consensus.cc:385] T c4e048ca28824329b924845ccf5067cf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.684445 1168 raft_consensus.cc:740] T c4e048ca28824329b924845ccf5067cf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.684677 1168 consensus_queue.cc:260] T c4e048ca28824329b924845ccf5067cf P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.685124 1163 tablet_replica.cc:333] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:31.685191 1168 ts_tablet_manager.cc:1434] T c4e048ca28824329b924845ccf5067cf P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:31.685514 1168 tablet_bootstrap.cc:492] T b8828385cec34d70b717ae7f3813b9c3 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:31.686414 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet c85624ad640041c29bb87416e1992a9e (table default.Table_8f7e347b9ac64793b1073dfa955ebe21 [id=a6ab64c3e0d94462b716f2d153bc3314]) successfully deleted
W20251211 20:52:31.686671 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 993616e226274c229a5924fbe69911af because the tablet was not found. No further retry: Not found: Tablet not found: 993616e226274c229a5924fbe69911af
I20251211 20:52:31.687299 1168 tablet_bootstrap.cc:654] T b8828385cec34d70b717ae7f3813b9c3 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.642215 746 tablet_service.cc:1552] Processing DeleteTablet for tablet c38c1a4c0e574eb0957de127d41c26ad with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.589547 1574 raft_consensus.cc:493] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:31.688930 1574 raft_consensus.cc:515] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.689546 1574 leader_election.cc:290] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:31.690033 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c38c1a4c0e574eb0957de127d41c26ad" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:31.690747 711 leader_election.cc:343] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:31.691097 710 leader_election.cc:343] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: c38c1a4c0e574eb0957de127d41c26ad
I20251211 20:52:31.691215 710 leader_election.cc:304] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:31.691514 1163 raft_consensus.cc:2243] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:31.691596 1574 raft_consensus.cc:2749] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:31.642670 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 78922f26cedb4bb989aa1c1cb40bfa3a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.691874 1163 raft_consensus.cc:2272] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:31.692641 1165 tablet_replica.cc:333] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:31.692895 1165 raft_consensus.cc:2243] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:31.693116 1165 raft_consensus.cc:2272] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:31.693895 1165 ts_tablet_manager.cc:1916] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:31.701169 1168 tablet_bootstrap.cc:492] T b8828385cec34d70b717ae7f3813b9c3 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:31.701187 1160 tablet_bootstrap.cc:492] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:31.701340 1168 ts_tablet_manager.cc:1403] T b8828385cec34d70b717ae7f3813b9c3 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.003s
I20251211 20:52:31.701462 1160 ts_tablet_manager.cc:1403] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.173s user 0.004s sys 0.029s
I20251211 20:52:31.702195 1160 raft_consensus.cc:359] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:31.702437 1160 raft_consensus.cc:385] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.702571 1160 raft_consensus.cc:740] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.702831 1160 consensus_queue.cc:260] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:31.703526 1160 ts_tablet_manager.cc:1434] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:31.703867 1166 tablet_bootstrap.cc:492] T 1843dc4548134f29969d46fbc760f5d7 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:31.703876 1160 tablet_bootstrap.cc:492] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:31.704018 1166 ts_tablet_manager.cc:1403] T 1843dc4548134f29969d46fbc760f5d7 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.389s user 0.014s sys 0.004s
I20251211 20:52:31.704751 1166 raft_consensus.cc:359] T 1843dc4548134f29969d46fbc760f5d7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.704916 1166 raft_consensus.cc:385] T 1843dc4548134f29969d46fbc760f5d7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.704988 1166 raft_consensus.cc:740] T 1843dc4548134f29969d46fbc760f5d7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.705186 1166 consensus_queue.cc:260] T 1843dc4548134f29969d46fbc760f5d7 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.705648 1160 tablet_bootstrap.cc:654] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.705680 1166 ts_tablet_manager.cc:1434] T 1843dc4548134f29969d46fbc760f5d7 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:31.707103 1166 tablet_bootstrap.cc:492] T 804a11c8f7414eb1a6bcff2952be9ae2 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:31.707054 1168 raft_consensus.cc:359] T b8828385cec34d70b717ae7f3813b9c3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.707278 1168 raft_consensus.cc:385] T b8828385cec34d70b717ae7f3813b9c3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.707360 1168 raft_consensus.cc:740] T b8828385cec34d70b717ae7f3813b9c3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.707659 1168 consensus_queue.cc:260] T b8828385cec34d70b717ae7f3813b9c3 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.708290 1168 ts_tablet_manager.cc:1434] T b8828385cec34d70b717ae7f3813b9c3 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20251211 20:52:31.708639 1168 tablet_bootstrap.cc:492] T 4b736a1b347d44b398980654caa21332 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:31.711378 1166 tablet_bootstrap.cc:654] T 804a11c8f7414eb1a6bcff2952be9ae2 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.711834 1168 tablet_bootstrap.cc:654] T 4b736a1b347d44b398980654caa21332 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.700304 1165 ts_tablet_manager.cc:1929] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:31.714529 1165 log.cc:1199] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/78922f26cedb4bb989aa1c1cb40bfa3a
W20251211 20:52:31.714638 582 consensus_peers.cc:597] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 -> Peer 11f5f94732f049d0be4c97cebfffcb05 (127.0.104.61:40749): Couldn't send request to peer 11f5f94732f049d0be4c97cebfffcb05. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.61:40749: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20251211 20:52:31.715075 1165 ts_tablet_manager.cc:1950] T 78922f26cedb4bb989aa1c1cb40bfa3a P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:31.716169 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 78922f26cedb4bb989aa1c1cb40bfa3a (table default.Table_736d6c2eb4dd4eb9a72dca6b9ed2073f [id=289e09e5a445461a9d0949c41ddcbd18]) successfully deleted
I20251211 20:52:31.718205 1164 ts_tablet_manager.cc:1929] T ed9ee283045a44978422cd080aa2cda5 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:31.718505 1165 tablet_replica.cc:333] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:31.719067 1165 raft_consensus.cc:2243] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:31.719295 1165 raft_consensus.cc:2272] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:31.720139 1165 ts_tablet_manager.cc:1916] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:31.722913 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 1578887b2f5748cd83a6fbdec2f3fc99
I20251211 20:52:31.723450 588 catalog_manager.cc:6724] ReplaceTablet: tablet 1578887b2f5748cd83a6fbdec2f3fc99 deleted and replaced by tablet bc4e7362e6d045da81bf39f4135f500e
I20251211 20:52:31.723704 1164 log.cc:1199] T ed9ee283045a44978422cd080aa2cda5 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/ed9ee283045a44978422cd080aa2cda5
I20251211 20:52:31.724042 1164 ts_tablet_manager.cc:1950] T ed9ee283045a44978422cd080aa2cda5 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:31.725031 1164 tablet_replica.cc:333] T f19a83b557e24f929b172d926996ae64 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:31.725199 1164 raft_consensus.cc:2243] T f19a83b557e24f929b172d926996ae64 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:31.725347 1164 raft_consensus.cc:2272] T f19a83b557e24f929b172d926996ae64 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:31.727095 588 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 1843dc4548134f29969d46fbc760f5d7 (table default.Table_99917cb4c70840e19004ad381746d77e [id=6abb187bfd214e91b2e11267065a2841]). Expected version 1 got 0
I20251211 20:52:31.727543 1164 ts_tablet_manager.cc:1916] T f19a83b557e24f929b172d926996ae64 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:31.732136 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet ed9ee283045a44978422cd080aa2cda5 (table default.Table_786e1caf9f674d48a6aad3ef7edc5b01 [id=33c5cbdd3d004439ab01b4ae69db56d1]) successfully deleted
I20251211 20:52:31.732535 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 1578887b2f5748cd83a6fbdec2f3fc99 with delete_type TABLET_DATA_DELETED (replaced by tablet bc4e7362e6d045da81bf39f4135f500e) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.733377 1160 tablet_bootstrap.cc:492] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:31.733636 1160 ts_tablet_manager.cc:1403] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.030s user 0.000s sys 0.003s
I20251211 20:52:31.735049 1160 raft_consensus.cc:359] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.735347 1160 raft_consensus.cc:385] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.735452 1160 raft_consensus.cc:740] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.735813 1160 consensus_queue.cc:260] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:31.737708 1667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:52:31.738996 1667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20251211 20:52:31.739236 1160 ts_tablet_manager.cc:1434] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
W20251211 20:52:31.739408 1667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20251211 20:52:31.740581 1168 tablet_bootstrap.cc:492] T 4b736a1b347d44b398980654caa21332 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:31.740801 1168 ts_tablet_manager.cc:1403] T 4b736a1b347d44b398980654caa21332 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.032s user 0.003s sys 0.009s
I20251211 20:52:31.741698 1168 raft_consensus.cc:359] T 4b736a1b347d44b398980654caa21332 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.741941 1168 raft_consensus.cc:385] T 4b736a1b347d44b398980654caa21332 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.742079 1168 raft_consensus.cc:740] T 4b736a1b347d44b398980654caa21332 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.742686 1160 tablet_bootstrap.cc:492] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:31.742386 1168 consensus_queue.cc:260] T 4b736a1b347d44b398980654caa21332 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.744905 1160 tablet_bootstrap.cc:654] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.745895 1168 ts_tablet_manager.cc:1434] T 4b736a1b347d44b398980654caa21332 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251211 20:52:31.746831 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 79cc81e91e1846038621af7a0a9f7a2e
I20251211 20:52:31.747232 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_aab9c3f649474051bcf63611d32aa48a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:31.747632 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_7eabd17b66f44d40aebe078e0404f66f" } new_table_name: "default.Table_0455012740604013898f53488fd5c371" modify_external_catalogs: true
W20251211 20:52:31.747865 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aab9c3f649474051bcf63611d32aa48a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.747929 1168 tablet_bootstrap.cc:492] T dc40b9826ea84728910b1fd61ce62a25 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:31.748939 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 4b07f110f25645799ed0c89737cd8279
I20251211 20:52:31.749500 594 master_service.cc:954] ReplaceTablet: received request to replace tablet d5f0ba3ab16240f9822be69f941f5da2 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:31.750073 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8aecd4725c9d43a2b561e3e917e463c6" } new_table_name: "default.Table_8db9acb9d35149fdabcb9a1d24061b46" modify_external_catalogs: true
I20251211 20:52:31.751255 1165 ts_tablet_manager.cc:1929] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:31.751343 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 116b21a182474ca191c4a89e9474677d
I20251211 20:52:31.751439 1165 log.cc:1199] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/c38c1a4c0e574eb0957de127d41c26ad
I20251211 20:52:31.751808 1165 ts_tablet_manager.cc:1950] T c38c1a4c0e574eb0957de127d41c26ad P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:31.752029 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b377fea34b4145719629f5ca57b25a41"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:31.752576 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b377fea34b4145719629f5ca57b25a41 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.753751 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 4a39870c7eea4a379b284ea2e5ddf760
I20251211 20:52:31.754350 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_00616be7e0f84573a75647eacd3a086d" } new_table_name: "default.Table_f9c6b985657d4e44a74358541fb67452" modify_external_catalogs: true
I20251211 20:52:31.755401 1164 ts_tablet_manager.cc:1929] T f19a83b557e24f929b172d926996ae64 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:31.755530 1164 log.cc:1199] T f19a83b557e24f929b172d926996ae64 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/f19a83b557e24f929b172d926996ae64
I20251211 20:52:31.755538 1168 tablet_bootstrap.cc:654] T dc40b9826ea84728910b1fd61ce62a25 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.755719 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet fa79b67b9eee45e2b41c4a90967606cc
I20251211 20:52:31.755901 1164 ts_tablet_manager.cc:1950] T f19a83b557e24f929b172d926996ae64 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:31.756168 590 catalog_manager.cc:6724] ReplaceTablet: tablet fa79b67b9eee45e2b41c4a90967606cc deleted and replaced by tablet 83067b139f504bc191a515feed0ec771
I20251211 20:52:31.756469 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_7dc91b2b86fc4c95b162a34345fc8443"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:31.756996 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7dc91b2b86fc4c95b162a34345fc8443 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.758109 596 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 4464859cdf5d494a87c348bb582463f6
I20251211 20:52:31.758406 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_737b2e537604423ab32fd1cc85baab78" } new_table_name: "default.Table_3482fbfc5b144a7198a7d01aa003cb6b" modify_external_catalogs: true
I20251211 20:52:31.758934 1160 tablet_bootstrap.cc:492] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:31.759068 1160 ts_tablet_manager.cc:1403] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.003s
I20251211 20:52:31.759773 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 3f1840e4b5214ddb878935a79b214eae
I20251211 20:52:31.760197 597 catalog_manager.cc:6724] ReplaceTablet: tablet 3f1840e4b5214ddb878935a79b214eae deleted and replaced by tablet 4fc56df5ccb94198b5b0e516eeae6941
I20251211 20:52:31.760382 597 master_service.cc:954] ReplaceTablet: received request to replace tablet 4a39870c7eea4a379b284ea2e5ddf760 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:31.760425 1160 raft_consensus.cc:359] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.760599 1160 raft_consensus.cc:385] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.760674 1160 raft_consensus.cc:740] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.760906 1160 consensus_queue.cc:260] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.764334 1160 ts_tablet_manager.cc:1434] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20251211 20:52:31.765678 1168 tablet_bootstrap.cc:492] T dc40b9826ea84728910b1fd61ce62a25 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:31.765930 1168 ts_tablet_manager.cc:1403] T dc40b9826ea84728910b1fd61ce62a25 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.003s
I20251211 20:52:31.766738 1168 raft_consensus.cc:359] T dc40b9826ea84728910b1fd61ce62a25 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.767244 1160 tablet_bootstrap.cc:492] T dd425601d89148eb95525f796a3abf1e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:31.767004 1168 raft_consensus.cc:385] T dc40b9826ea84728910b1fd61ce62a25 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.768061 1168 raft_consensus.cc:740] T dc40b9826ea84728910b1fd61ce62a25 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.768395 1168 consensus_queue.cc:260] T dc40b9826ea84728910b1fd61ce62a25 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.769521 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet f19a83b557e24f929b172d926996ae64 (table default.Table_4a3a3fe6b0294c939ebc90871cbcb4de [id=a06a0b2452ea4100b09772b53320b537]) successfully deleted
I20251211 20:52:31.769685 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 116b21a182474ca191c4a89e9474677d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:31.769790 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 1578887b2f5748cd83a6fbdec2f3fc99 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1578887b2f5748cd83a6fbdec2f3fc99 already in progress: creating tablet
I20251211 20:52:31.769939 888 tablet_service.cc:1552] Processing DeleteTablet for tablet fa79b67b9eee45e2b41c4a90967606cc with delete_type TABLET_DATA_DELETED (replaced by tablet 83067b139f504bc191a515feed0ec771) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.770011 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 79cc81e91e1846038621af7a0a9f7a2e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.770232 881 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b07f110f25645799ed0c89737cd8279 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.770417 877 tablet_service.cc:1552] Processing DeleteTablet for tablet 4a39870c7eea4a379b284ea2e5ddf760 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.770746 1168 ts_tablet_manager.cc:1434] T dc40b9826ea84728910b1fd61ce62a25 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251211 20:52:31.771004 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 3f1840e4b5214ddb878935a79b214eae with delete_type TABLET_DATA_DELETED (replaced by tablet 4fc56df5ccb94198b5b0e516eeae6941) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.771613 1168 tablet_bootstrap.cc:492] T 8b89ae724b7f41c98741a41d75592da7 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:31.773269 1168 tablet_bootstrap.cc:654] T 8b89ae724b7f41c98741a41d75592da7 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:31.776012 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 116b21a182474ca191c4a89e9474677d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 116b21a182474ca191c4a89e9474677d already in progress: creating tablet
W20251211 20:52:31.776357 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet fa79b67b9eee45e2b41c4a90967606cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa79b67b9eee45e2b41c4a90967606cc already in progress: creating tablet
I20251211 20:52:31.777704 1160 tablet_bootstrap.cc:654] T dd425601d89148eb95525f796a3abf1e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:31.777827 1667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251211 20:52:31.777951 1667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:52:31.778067 1667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251211 20:52:31.778167 1667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20251211 20:52:31.781980 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 79cc81e91e1846038621af7a0a9f7a2e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79cc81e91e1846038621af7a0a9f7a2e already in progress: creating tablet
W20251211 20:52:31.782284 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 4b07f110f25645799ed0c89737cd8279 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4b07f110f25645799ed0c89737cd8279 already in progress: creating tablet
W20251211 20:52:31.782464 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 4a39870c7eea4a379b284ea2e5ddf760 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a39870c7eea4a379b284ea2e5ddf760 already in progress: creating tablet
W20251211 20:52:31.782712 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 3f1840e4b5214ddb878935a79b214eae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3f1840e4b5214ddb878935a79b214eae already in progress: creating tablet
I20251211 20:52:31.789674 1168 tablet_bootstrap.cc:492] T 8b89ae724b7f41c98741a41d75592da7 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:31.789952 1168 ts_tablet_manager.cc:1403] T 8b89ae724b7f41c98741a41d75592da7 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.005s
I20251211 20:52:31.790859 1168 raft_consensus.cc:359] T 8b89ae724b7f41c98741a41d75592da7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.791148 1168 raft_consensus.cc:385] T 8b89ae724b7f41c98741a41d75592da7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.791277 1168 raft_consensus.cc:740] T 8b89ae724b7f41c98741a41d75592da7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.791642 1168 consensus_queue.cc:260] T 8b89ae724b7f41c98741a41d75592da7 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.792407 1168 ts_tablet_manager.cc:1434] T 8b89ae724b7f41c98741a41d75592da7 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:31.793470 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_bdb9aa9507d94416bbc3c8fe9cb11fa7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:31.794018 1168 tablet_bootstrap.cc:492] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:31.794258 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bdb9aa9507d94416bbc3c8fe9cb11fa7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.795344 1160 tablet_bootstrap.cc:492] T dd425601d89148eb95525f796a3abf1e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:31.795537 1160 ts_tablet_manager.cc:1403] T dd425601d89148eb95525f796a3abf1e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.028s user 0.006s sys 0.010s
I20251211 20:52:31.795924 1168 tablet_bootstrap.cc:654] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.796434 1160 raft_consensus.cc:359] T dd425601d89148eb95525f796a3abf1e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.796417 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_9d7fd7d87cf049c28685144bb909d0c3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:31.796643 1160 raft_consensus.cc:385] T dd425601d89148eb95525f796a3abf1e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.796712 1160 raft_consensus.cc:740] T dd425601d89148eb95525f796a3abf1e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
W20251211 20:52:31.796990 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9d7fd7d87cf049c28685144bb909d0c3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.796994 1160 consensus_queue.cc:260] T dd425601d89148eb95525f796a3abf1e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.797542 1160 ts_tablet_manager.cc:1434] T dd425601d89148eb95525f796a3abf1e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:31.800590 1160 tablet_bootstrap.cc:492] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:31.802256 1160 tablet_bootstrap.cc:654] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.803663 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 1578887b2f5748cd83a6fbdec2f3fc99 with delete_type TABLET_DATA_DELETED (replaced by tablet bc4e7362e6d045da81bf39f4135f500e) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.804473 1168 tablet_bootstrap.cc:492] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:31.804649 1168 ts_tablet_manager.cc:1403] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.001s
I20251211 20:52:31.804944 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 1578887b2f5748cd83a6fbdec2f3fc99 with delete_type TABLET_DATA_DELETED (replaced by tablet bc4e7362e6d045da81bf39f4135f500e) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:31.805423 1168 raft_consensus.cc:359] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.805650 1168 raft_consensus.cc:385] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.805743 1168 raft_consensus.cc:740] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.805990 1168 consensus_queue.cc:260] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.806447 1168 ts_tablet_manager.cc:1434] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:31.808365 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 79cc81e91e1846038621af7a0a9f7a2e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.808634 1168 tablet_bootstrap.cc:492] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:31.809329 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 79cc81e91e1846038621af7a0a9f7a2e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:31.809883 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b07f110f25645799ed0c89737cd8279 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.810323 1168 tablet_bootstrap.cc:654] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.810829 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b07f110f25645799ed0c89737cd8279 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:31.811342 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 116b21a182474ca191c4a89e9474677d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.811817 1667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.20:41333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.62:41123,127.0.104.61:40749,127.0.104.60:42087
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.61:40749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.104.61
--webserver_port=42143
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:52:31.812175 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 116b21a182474ca191c4a89e9474677d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:31.812613 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 4a39870c7eea4a379b284ea2e5ddf760 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.813376 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 4a39870c7eea4a379b284ea2e5ddf760 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:31.813879 746 tablet_service.cc:1552] Processing DeleteTablet for tablet fa79b67b9eee45e2b41c4a90967606cc with delete_type TABLET_DATA_DELETED (replaced by tablet 83067b139f504bc191a515feed0ec771) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.814628 1667 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:52:31.814627 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet fa79b67b9eee45e2b41c4a90967606cc with delete_type TABLET_DATA_DELETED (replaced by tablet 83067b139f504bc191a515feed0ec771) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:31.815086 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 3f1840e4b5214ddb878935a79b214eae with delete_type TABLET_DATA_DELETED (replaced by tablet 4fc56df5ccb94198b5b0e516eeae6941) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.815927 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 3f1840e4b5214ddb878935a79b214eae with delete_type TABLET_DATA_DELETED (replaced by tablet 4fc56df5ccb94198b5b0e516eeae6941) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:31.816478 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet c38c1a4c0e574eb0957de127d41c26ad (table default.Table_88975c1ad9604f50b7f69fbbdacabce3 [id=52f34b88d367454286943a27138b20f2]) successfully deleted
W20251211 20:52:31.816813 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 1578887b2f5748cd83a6fbdec2f3fc99 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1578887b2f5748cd83a6fbdec2f3fc99 already in progress: creating tablet
I20251211 20:52:31.817277 1667 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:52:31.817744 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 79cc81e91e1846038621af7a0a9f7a2e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79cc81e91e1846038621af7a0a9f7a2e already in progress: creating tablet
W20251211 20:52:31.818689 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 4b07f110f25645799ed0c89737cd8279 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4b07f110f25645799ed0c89737cd8279 already in progress: creating tablet
W20251211 20:52:31.818933 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 116b21a182474ca191c4a89e9474677d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 116b21a182474ca191c4a89e9474677d already in progress: creating tablet
W20251211 20:52:31.819141 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 4a39870c7eea4a379b284ea2e5ddf760 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a39870c7eea4a379b284ea2e5ddf760 already in progress: creating tablet
W20251211 20:52:31.819365 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet fa79b67b9eee45e2b41c4a90967606cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa79b67b9eee45e2b41c4a90967606cc already in progress: creating tablet
W20251211 20:52:31.819597 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 3f1840e4b5214ddb878935a79b214eae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3f1840e4b5214ddb878935a79b214eae already in progress: creating tablet
I20251211 20:52:31.820375 1163 ts_tablet_manager.cc:1916] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:31.833078 1160 tablet_bootstrap.cc:492] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:31.833343 1160 ts_tablet_manager.cc:1403] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.001s
I20251211 20:52:31.833763 1168 tablet_bootstrap.cc:492] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:31.833947 1168 ts_tablet_manager.cc:1403] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.026s user 0.001s sys 0.009s
I20251211 20:52:31.834412 1160 raft_consensus.cc:359] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.834695 1160 raft_consensus.cc:385] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.834832 1160 raft_consensus.cc:740] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.835230 1160 consensus_queue.cc:260] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.837072 1160 ts_tablet_manager.cc:1434] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251211 20:52:31.837100 1168 raft_consensus.cc:359] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.838485 1160 tablet_bootstrap.cc:492] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:31.837306 1168 raft_consensus.cc:385] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.838814 1168 raft_consensus.cc:740] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.839618 1168 consensus_queue.cc:260] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.840183 1168 ts_tablet_manager.cc:1434] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20251211 20:52:31.840523 1168 tablet_bootstrap.cc:492] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:31.842310 1612 raft_consensus.cc:493] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:31.842554 1612 raft_consensus.cc:515] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:31.842849 1737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:52:31.843189 1612 leader_election.cc:290] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:31.844347 847 leader_election.cc:343] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 27466459d8e6411ab87ac474fb617735
W20251211 20:52:31.844594 847 leader_election.cc:343] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 27466459d8e6411ab87ac474fb617735
I20251211 20:52:31.844696 847 leader_election.cc:304] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:31.845351 1160 tablet_bootstrap.cc:654] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.846388 1612 raft_consensus.cc:2749] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:31.849979 1738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:52:31.850589 1166 tablet_bootstrap.cc:492] T 804a11c8f7414eb1a6bcff2952be9ae2 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:31.850873 1166 ts_tablet_manager.cc:1403] T 804a11c8f7414eb1a6bcff2952be9ae2 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.144s user 0.057s sys 0.048s
I20251211 20:52:31.851378 1163 ts_tablet_manager.cc:1929] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:31.851536 1163 log.cc:1199] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/5fd2a6cf80a04f20aef8ff9c84bc88db
I20251211 20:52:31.851991 1163 ts_tablet_manager.cc:1950] T 5fd2a6cf80a04f20aef8ff9c84bc88db P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:31.852196 1166 raft_consensus.cc:359] T 804a11c8f7414eb1a6bcff2952be9ae2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.852548 1166 raft_consensus.cc:385] T 804a11c8f7414eb1a6bcff2952be9ae2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.852747 1166 raft_consensus.cc:740] T 804a11c8f7414eb1a6bcff2952be9ae2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.853049 1166 consensus_queue.cc:260] T 804a11c8f7414eb1a6bcff2952be9ae2 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.853209 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 5fd2a6cf80a04f20aef8ff9c84bc88db (table default.Table_370d3069ceee4b8e8856bd74fe3a1815 [id=10fd0368198c447c95b692b1f3e6a023]) successfully deleted
I20251211 20:52:31.856829 1168 tablet_bootstrap.cc:654] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.860461 1166 ts_tablet_manager.cc:1434] T 804a11c8f7414eb1a6bcff2952be9ae2 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20251211 20:52:31.861948 1586 raft_consensus.cc:493] T 1843dc4548134f29969d46fbc760f5d7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:31.862098 1586 raft_consensus.cc:515] T 1843dc4548134f29969d46fbc760f5d7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.862699 1586 leader_election.cc:290] T 1843dc4548134f29969d46fbc760f5d7 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:31.863358 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1843dc4548134f29969d46fbc760f5d7" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:31.864154 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1843dc4548134f29969d46fbc760f5d7" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:31.864825 982 leader_election.cc:343] T 1843dc4548134f29969d46fbc760f5d7 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:31.865238 983 leader_election.cc:343] T 1843dc4548134f29969d46fbc760f5d7 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:31.865375 983 leader_election.cc:304] T 1843dc4548134f29969d46fbc760f5d7 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:31.866348 1586 raft_consensus.cc:2749] T 1843dc4548134f29969d46fbc760f5d7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:31.867357 1166 tablet_bootstrap.cc:492] T f93006227cc6489e91f3593e11789b54 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:31.869247 1166 tablet_bootstrap.cc:654] T f93006227cc6489e91f3593e11789b54 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.872586 1160 tablet_bootstrap.cc:492] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:31.872779 1160 ts_tablet_manager.cc:1403] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.034s user 0.001s sys 0.002s
I20251211 20:52:31.873605 1160 raft_consensus.cc:359] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.873826 1160 raft_consensus.cc:385] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.873890 1160 raft_consensus.cc:740] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.874131 1160 consensus_queue.cc:260] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:31.875041 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 7b06cd366de5494780b5ac81abbf0702 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7b06cd366de5494780b5ac81abbf0702 already in progress: creating tablet
W20251211 20:52:31.875324 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 29463a8f71f943a0927942235af3c2da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 29463a8f71f943a0927942235af3c2da already in progress: creating tablet
W20251211 20:52:31.876152 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 3c578d81e8dd41aaa0f707c4e3fed225 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3c578d81e8dd41aaa0f707c4e3fed225 already in progress: creating tablet
W20251211 20:52:31.876379 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 260bdeb73439437cbf234d20f7264ab3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 260bdeb73439437cbf234d20f7264ab3 already in progress: creating tablet
W20251211 20:52:31.876556 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 1578887b2f5748cd83a6fbdec2f3fc99 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1578887b2f5748cd83a6fbdec2f3fc99 already in progress: creating tablet
W20251211 20:52:31.876746 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 79cc81e91e1846038621af7a0a9f7a2e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79cc81e91e1846038621af7a0a9f7a2e already in progress: creating tablet
W20251211 20:52:31.876919 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 4b07f110f25645799ed0c89737cd8279 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4b07f110f25645799ed0c89737cd8279 already in progress: creating tablet
W20251211 20:52:31.877131 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 116b21a182474ca191c4a89e9474677d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 116b21a182474ca191c4a89e9474677d already in progress: creating tablet
W20251211 20:52:31.877319 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 4a39870c7eea4a379b284ea2e5ddf760 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a39870c7eea4a379b284ea2e5ddf760 already in progress: creating tablet
W20251211 20:52:31.877533 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet fa79b67b9eee45e2b41c4a90967606cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa79b67b9eee45e2b41c4a90967606cc already in progress: creating tablet
W20251211 20:52:31.877722 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 3f1840e4b5214ddb878935a79b214eae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3f1840e4b5214ddb878935a79b214eae already in progress: creating tablet
I20251211 20:52:31.878559 592 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet dd425601d89148eb95525f796a3abf1e (table default.Table_d4afd6ed615a41ac8f6714c8aac27857 [id=c842792b01cb4c689fcfd3d796151508]). Expected version 1 got 0
W20251211 20:52:31.881342 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dfd194cf00f549c8937cb6ec40987240 has no consensus state
I20251211 20:52:31.881696 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_54b89a8f30cf414590cfd7a5773bf326"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:31.882083 1166 tablet_bootstrap.cc:492] T f93006227cc6489e91f3593e11789b54 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:31.882236 1166 ts_tablet_manager.cc:1403] T f93006227cc6489e91f3593e11789b54 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.000s
W20251211 20:52:31.882268 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_54b89a8f30cf414590cfd7a5773bf326 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.883075 1166 raft_consensus.cc:359] T f93006227cc6489e91f3593e11789b54 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.883246 1166 raft_consensus.cc:385] T f93006227cc6489e91f3593e11789b54 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.883329 1166 raft_consensus.cc:740] T f93006227cc6489e91f3593e11789b54 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
W20251211 20:52:31.883522 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 79d92736bb1c46d2bfa5678b2d396ec2 consensus state has no leader
I20251211 20:52:31.883615 1166 consensus_queue.cc:260] T f93006227cc6489e91f3593e11789b54 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.884057 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_5453a73a163149baa88d85b027bb42ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:31.884182 1160 ts_tablet_manager.cc:1434] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20251211 20:52:31.884564 1160 tablet_bootstrap.cc:492] T c4e048ca28824329b924845ccf5067cf P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:31.884569 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5453a73a163149baa88d85b027bb42ee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.885871 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_68c2d3a1dc614ca4bcf28180d6e4faf8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:31.886349 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_68c2d3a1dc614ca4bcf28180d6e4faf8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.886983 1160 tablet_bootstrap.cc:654] T c4e048ca28824329b924845ccf5067cf P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:31.887428 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 46cb1d16a0e94b3a833f32c635213fd2 consensus state has no leader
I20251211 20:52:31.887785 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b426c9f38ff44d76af3480168e4003cf" } new_table_name: "default.Table_b26cc06e1b0745e08181124e1cbce934" modify_external_catalogs: true
I20251211 20:52:31.890420 1168 tablet_bootstrap.cc:492] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:31.890594 1168 ts_tablet_manager.cc:1403] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.050s user 0.002s sys 0.002s
I20251211 20:52:31.891405 1168 raft_consensus.cc:359] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.891618 1168 raft_consensus.cc:385] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.891695 1168 raft_consensus.cc:740] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.893225 595 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 1843dc4548134f29969d46fbc760f5d7 (table default.Table_99917cb4c70840e19004ad381746d77e [id=6abb187bfd214e91b2e11267065a2841]). Expected version 1 got 0
I20251211 20:52:31.893255 1168 consensus_queue.cc:260] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.894079 1168 ts_tablet_manager.cc:1434] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:31.894361 1168 tablet_bootstrap.cc:492] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:31.894394 877 tablet_service.cc:1505] Processing CreateTablet for tablet a471a9accbfc4875bd1d7802f5620808 (DEFAULT_TABLE table=default.Table_6310a6161f1f4def90affd0ea6fc67d2 [id=9f579df085864c668e817e5bcb975502]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.894874 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a471a9accbfc4875bd1d7802f5620808. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.896729 893 tablet_service.cc:1505] Processing CreateTablet for tablet f26ba8f4bd3a4a1f8fa2ba6e80d543f8 (DEFAULT_TABLE table=default.Table_02295a0d7c9c43bab5f8b6db77d27dd1 [id=b966d0c0b49d4023894950c5dc06f678]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.896886 1168 tablet_bootstrap.cc:654] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.897190 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f26ba8f4bd3a4a1f8fa2ba6e80d543f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.899936 1026 tablet_service.cc:1505] Processing CreateTablet for tablet f26ba8f4bd3a4a1f8fa2ba6e80d543f8 (DEFAULT_TABLE table=default.Table_02295a0d7c9c43bab5f8b6db77d27dd1 [id=b966d0c0b49d4023894950c5dc06f678]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.900334 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f26ba8f4bd3a4a1f8fa2ba6e80d543f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.901223 1166 ts_tablet_manager.cc:1434] T f93006227cc6489e91f3593e11789b54 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.019s user 0.000s sys 0.002s
I20251211 20:52:31.903688 593 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet c4e048ca28824329b924845ccf5067cf (table default.Table_f541a176465249d8aaba954bc23bcf01 [id=6b55ad645bec4a7daefc996978420295]). Expected version 4 got 0
I20251211 20:52:31.903731 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 804a11c8f7414eb1a6bcff2952be9ae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:31.904026 1163 tablet_replica.cc:333] T 804a11c8f7414eb1a6bcff2952be9ae2 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:31.904217 1163 raft_consensus.cc:2243] T 804a11c8f7414eb1a6bcff2952be9ae2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:31.904408 1163 raft_consensus.cc:2272] T 804a11c8f7414eb1a6bcff2952be9ae2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:31.904914 1166 tablet_bootstrap.cc:492] T 6a2442a4ebc940a281f9ea1a394f6621 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:31.905089 1612 raft_consensus.cc:493] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:31.905267 1612 raft_consensus.cc:515] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.905962 1612 leader_election.cc:290] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:31.907109 847 leader_election.cc:343] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: bd8fd6c977204c90b558827fc0b15f43
I20251211 20:52:31.907332 746 tablet_service.cc:1505] Processing CreateTablet for tablet f26ba8f4bd3a4a1f8fa2ba6e80d543f8 (DEFAULT_TABLE table=default.Table_02295a0d7c9c43bab5f8b6db77d27dd1 [id=b966d0c0b49d4023894950c5dc06f678]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.908624 1166 tablet_bootstrap.cc:654] T 6a2442a4ebc940a281f9ea1a394f6621 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:31.909050 1163 ts_tablet_manager.cc:1916] T 804a11c8f7414eb1a6bcff2952be9ae2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:31.912838 1020 tablet_service.cc:1505] Processing CreateTablet for tablet e95b0d8c507a4779a087c8933d464228 (DEFAULT_TABLE table=default.Table_e7b2a8479479460fa5a4f3c23c6eeee6 [id=06f467911b6f45849e99f0b6fe615163]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.912871 881 tablet_service.cc:1505] Processing CreateTablet for tablet e95b0d8c507a4779a087c8933d464228 (DEFAULT_TABLE table=default.Table_e7b2a8479479460fa5a4f3c23c6eeee6 [id=06f467911b6f45849e99f0b6fe615163]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.913278 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e95b0d8c507a4779a087c8933d464228. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.913376 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e95b0d8c507a4779a087c8933d464228. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.915912 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 10746162da4640e6a2a891467690b770 (DEFAULT_TABLE table=default.Table_d7d5d2408f644d09b54b10d82a401116 [id=7bc8a2b91fe54b91b6594b371fea7bd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.915948 889 tablet_service.cc:1505] Processing CreateTablet for tablet 10746162da4640e6a2a891467690b770 (DEFAULT_TABLE table=default.Table_d7d5d2408f644d09b54b10d82a401116 [id=7bc8a2b91fe54b91b6594b371fea7bd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.916251 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10746162da4640e6a2a891467690b770. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.918524 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 27466459d8e6411ab87ac474fb617735 with delete_type TABLET_DATA_DELETED (replaced by tablet 8feee01fc0aa4b32be926ff2b617b894) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.918813 1164 tablet_replica.cc:333] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:31.919673 885 tablet_service.cc:1552] Processing DeleteTablet for tablet bd8fd6c977204c90b558827fc0b15f43 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.919878 879 tablet_service.cc:1552] Processing DeleteTablet for tablet 2260850f32b541aab1128818f144b5a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:31.919878 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 12aaf618d6474f599458026308d7afdd (DEFAULT_TABLE table=default.Table_1e17ae4fb7e84e41aea563972b80af32 [id=bf980f11122944cab06195b0522a2500]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:31.919993 847 leader_election.cc:343] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: bd8fd6c977204c90b558827fc0b15f43
I20251211 20:52:31.920128 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10746162da4640e6a2a891467690b770. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.920200 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12aaf618d6474f599458026308d7afdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.921904 882 tablet_service.cc:1505] Processing CreateTablet for tablet 200b6db115964f3c8882b514c794698d (DEFAULT_TABLE table=default.Table_fffa8ba66cb04202b2116b8fd4a741de [id=4d2c08402d5845e4ae592cc28ce8d9b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.922284 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 200b6db115964f3c8882b514c794698d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.923686 1029 tablet_service.cc:1505] Processing CreateTablet for tablet a471a9accbfc4875bd1d7802f5620808 (DEFAULT_TABLE table=default.Table_6310a6161f1f4def90affd0ea6fc67d2 [id=9f579df085864c668e817e5bcb975502]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.924026 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a471a9accbfc4875bd1d7802f5620808. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.927703 1164 raft_consensus.cc:2243] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:31.927902 1164 raft_consensus.cc:2272] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:31.928018 888 tablet_service.cc:1505] Processing CreateTablet for tablet 12aaf618d6474f599458026308d7afdd (DEFAULT_TABLE table=default.Table_1e17ae4fb7e84e41aea563972b80af32 [id=bf980f11122944cab06195b0522a2500]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.931805 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12aaf618d6474f599458026308d7afdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.932049 1164 ts_tablet_manager.cc:1916] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:31.932243 1018 tablet_service.cc:1505] Processing CreateTablet for tablet 200b6db115964f3c8882b514c794698d (DEFAULT_TABLE table=default.Table_fffa8ba66cb04202b2116b8fd4a741de [id=4d2c08402d5845e4ae592cc28ce8d9b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.932621 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 200b6db115964f3c8882b514c794698d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.935407 1013 tablet_service.cc:1505] Processing CreateTablet for tablet 6663f13691a24fb991b48f543d9de76b (DEFAULT_TABLE table=default.Table_c4c8f005f6fe4b7ebcc05d11911718e5 [id=bb14d230726a40f1a147c6c268d15b11]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.935791 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6663f13691a24fb991b48f543d9de76b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.935544 894 tablet_service.cc:1505] Processing CreateTablet for tablet b115d737d7fc483d9f5a1f040b511bae (DEFAULT_TABLE table=default.Table_6d94053661f142b79292e19752df02df [id=c5be59a2c3934b3789641ceeb2d4d6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.939927 1028 tablet_service.cc:1505] Processing CreateTablet for tablet 69085cd41db846e7a6e03bcfee4377f6 (DEFAULT_TABLE table=default.Table_24737d22fc454a3da12979ae8966c9b3 [id=9a943e65becc4d47828265f814118328]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.940303 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69085cd41db846e7a6e03bcfee4377f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.944823 847 leader_election.cc:304] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:31.946314 879 tablet_service.cc:1505] Processing CreateTablet for tablet 6663f13691a24fb991b48f543d9de76b (DEFAULT_TABLE table=default.Table_c4c8f005f6fe4b7ebcc05d11911718e5 [id=bb14d230726a40f1a147c6c268d15b11]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.947000 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet f93006227cc6489e91f3593e11789b54 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:31.947397 885 tablet_service.cc:1505] Processing CreateTablet for tablet 69085cd41db846e7a6e03bcfee4377f6 (DEFAULT_TABLE table=default.Table_24737d22fc454a3da12979ae8966c9b3 [id=9a943e65becc4d47828265f814118328]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.947876 1016 tablet_service.cc:1505] Processing CreateTablet for tablet b115d737d7fc483d9f5a1f040b511bae (DEFAULT_TABLE table=default.Table_6d94053661f142b79292e19752df02df [id=c5be59a2c3934b3789641ceeb2d4d6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.948912 1016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b115d737d7fc483d9f5a1f040b511bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.950054 748 tablet_service.cc:1552] Processing DeleteTablet for tablet dc40b9826ea84728910b1fd61ce62a25 with delete_type TABLET_DATA_DELETED (replaced by tablet bc32a778e5424850b23ccaf74bb9df07) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:31.950220 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d37a7d0d662e45079b0cfc02765d2e01 consensus state has no leader
I20251211 20:52:31.950416 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b736a1b347d44b398980654caa21332 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.950698 745 tablet_service.cc:1552] Processing DeleteTablet for tablet b8828385cec34d70b717ae7f3813b9c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.951738 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b89ae724b7f41c98741a41d75592da7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.951890 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b115d737d7fc483d9f5a1f040b511bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.953778 1165 tablet_replica.cc:333] T dc40b9826ea84728910b1fd61ce62a25 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:31.953986 1165 raft_consensus.cc:2243] T dc40b9826ea84728910b1fd61ce62a25 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:31.954403 1165 raft_consensus.cc:2272] T dc40b9826ea84728910b1fd61ce62a25 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:31.955313 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f26ba8f4bd3a4a1f8fa2ba6e80d543f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.955742 745 tablet_service.cc:1505] Processing CreateTablet for tablet a471a9accbfc4875bd1d7802f5620808 (DEFAULT_TABLE table=default.Table_6310a6161f1f4def90affd0ea6fc67d2 [id=9f579df085864c668e817e5bcb975502]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.956166 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a471a9accbfc4875bd1d7802f5620808. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:31.957535 1741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:52:31.959192 1667 server_base.cc:1047] running on GCE node
I20251211 20:52:31.960402 1667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:52:31.963836 757 tablet_service.cc:1505] Processing CreateTablet for tablet 200b6db115964f3c8882b514c794698d (DEFAULT_TABLE table=default.Table_fffa8ba66cb04202b2116b8fd4a741de [id=4d2c08402d5845e4ae592cc28ce8d9b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.964295 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 200b6db115964f3c8882b514c794698d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.965168 747 tablet_service.cc:1505] Processing CreateTablet for tablet 6663f13691a24fb991b48f543d9de76b (DEFAULT_TABLE table=default.Table_c4c8f005f6fe4b7ebcc05d11911718e5 [id=bb14d230726a40f1a147c6c268d15b11]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.965541 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6663f13691a24fb991b48f543d9de76b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.967255 741 tablet_service.cc:1505] Processing CreateTablet for tablet 69085cd41db846e7a6e03bcfee4377f6 (DEFAULT_TABLE table=default.Table_24737d22fc454a3da12979ae8966c9b3 [id=9a943e65becc4d47828265f814118328]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.967856 739 tablet_service.cc:1505] Processing CreateTablet for tablet b115d737d7fc483d9f5a1f040b511bae (DEFAULT_TABLE table=default.Table_6d94053661f142b79292e19752df02df [id=c5be59a2c3934b3789641ceeb2d4d6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.968238 750 tablet_service.cc:1552] Processing DeleteTablet for tablet c910246b3d5d4e7e8a71d9f18e94d24c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.968443 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 19d0aa259ac94f36b7f6a24d11a3d420 with delete_type TABLET_DATA_DELETED (replaced by tablet b9e5ad8030a44e03acfe73124b156ac8) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.968761 754 tablet_service.cc:1505] Processing CreateTablet for tablet 10746162da4640e6a2a891467690b770 (DEFAULT_TABLE table=default.Table_d7d5d2408f644d09b54b10d82a401116 [id=7bc8a2b91fe54b91b6594b371fea7bd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.969213 740 tablet_service.cc:1505] Processing CreateTablet for tablet e95b0d8c507a4779a087c8933d464228 (DEFAULT_TABLE table=default.Table_e7b2a8479479460fa5a4f3c23c6eeee6 [id=06f467911b6f45849e99f0b6fe615163]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.969656 748 tablet_service.cc:1505] Processing CreateTablet for tablet 12aaf618d6474f599458026308d7afdd (DEFAULT_TABLE table=default.Table_1e17ae4fb7e84e41aea563972b80af32 [id=bf980f11122944cab06195b0522a2500]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.970404 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_5c5941c768984fc49f485d17cb1a485d" } modify_external_catalogs: true
I20251211 20:52:31.970616 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_5c5941c768984fc49f485d17cb1a485d" } modify_external_catalogs: true
I20251211 20:52:31.971171 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_a9542664049c4b818c73d1958c2f2030"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:31.971726 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a9542664049c4b818c73d1958c2f2030 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.973047 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_320e1edc81a842849ac04e448aa2e2d4" } modify_external_catalogs: true
I20251211 20:52:31.973232 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_320e1edc81a842849ac04e448aa2e2d4" } modify_external_catalogs: true
I20251211 20:52:31.973623 1163 ts_tablet_manager.cc:1929] T 804a11c8f7414eb1a6bcff2952be9ae2 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:31.973755 1163 log.cc:1199] T 804a11c8f7414eb1a6bcff2952be9ae2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/804a11c8f7414eb1a6bcff2952be9ae2
I20251211 20:52:31.973716 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_fc80da3fac43460dabeea7e43bf098df" } modify_external_catalogs: true
I20251211 20:52:31.973845 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_fc80da3fac43460dabeea7e43bf098df" } modify_external_catalogs: true
I20251211 20:52:31.974308 1163 ts_tablet_manager.cc:1950] T 804a11c8f7414eb1a6bcff2952be9ae2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:31.975183 1163 tablet_replica.cc:333] T f93006227cc6489e91f3593e11789b54 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:31.975368 1163 raft_consensus.cc:2243] T f93006227cc6489e91f3593e11789b54 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:31.975518 1163 raft_consensus.cc:2272] T f93006227cc6489e91f3593e11789b54 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:31.976275 1163 ts_tablet_manager.cc:1916] T f93006227cc6489e91f3593e11789b54 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:31.977712 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69085cd41db846e7a6e03bcfee4377f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.978490 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b115d737d7fc483d9f5a1f040b511bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.978998 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10746162da4640e6a2a891467690b770. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.979665 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e95b0d8c507a4779a087c8933d464228. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.980041 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_60175b4e1c9744e0b6aefa1dece53aa0" } modify_external_catalogs: true
I20251211 20:52:31.980139 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12aaf618d6474f599458026308d7afdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.980206 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_60175b4e1c9744e0b6aefa1dece53aa0" } modify_external_catalogs: true
I20251211 20:52:31.980731 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4aad4cf01eff415f86b8446c29977007" } modify_external_catalogs: true
I20251211 20:52:31.980901 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4aad4cf01eff415f86b8446c29977007" } modify_external_catalogs: true
I20251211 20:52:31.982209 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_bd0dcfadc59f4ffa81d0004ae868ed3e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:31.982762 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd0dcfadc59f4ffa81d0004ae868ed3e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:31.984113 1165 ts_tablet_manager.cc:1916] T dc40b9826ea84728910b1fd61ce62a25 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:31.984434 1166 tablet_bootstrap.cc:492] T 6a2442a4ebc940a281f9ea1a394f6621 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:31.984596 1166 ts_tablet_manager.cc:1403] T 6a2442a4ebc940a281f9ea1a394f6621 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.080s user 0.001s sys 0.005s
I20251211 20:52:31.985786 1166 raft_consensus.cc:359] T 6a2442a4ebc940a281f9ea1a394f6621 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.986052 1166 raft_consensus.cc:385] T 6a2442a4ebc940a281f9ea1a394f6621 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:31.986122 1166 raft_consensus.cc:740] T 6a2442a4ebc940a281f9ea1a394f6621 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:31.986559 1166 consensus_queue.cc:260] T 6a2442a4ebc940a281f9ea1a394f6621 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:31.989748 750 tablet_service.cc:1505] Processing CreateTablet for tablet 1b8acfdf1a26427aa9863280dc39b818 (DEFAULT_TABLE table=default.Table_78bd2a084ad542c2968d1338f1ac71b5 [id=9649589ef6e94eebb27bb2b518105d2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:31.991037 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b8acfdf1a26427aa9863280dc39b818. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:31.993325 1166 ts_tablet_manager.cc:1434] T 6a2442a4ebc940a281f9ea1a394f6621 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20251211 20:52:31.995199 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet f9a3fdf17f274c15ac0bdfa9c174f649
I20251211 20:52:31.996177 1166 tablet_bootstrap.cc:492] T d5f0ba3ab16240f9822be69f941f5da2 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:31.996265 756 tablet_service.cc:1552] Processing DeleteTablet for tablet c4b76a7e84d344268567c551236ce772 with delete_type TABLET_DATA_DELETED (replaced by tablet 68be21a25c6743f5b86d518d98fbc35e) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:31.998975 1163 ts_tablet_manager.cc:1929] T f93006227cc6489e91f3593e11789b54 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:31.999120 1163 log.cc:1199] T f93006227cc6489e91f3593e11789b54 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/f93006227cc6489e91f3593e11789b54
I20251211 20:52:31.999469 1163 ts_tablet_manager.cc:1950] T f93006227cc6489e91f3593e11789b54 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:32.000101 1166 tablet_bootstrap.cc:654] T d5f0ba3ab16240f9822be69f941f5da2 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:32.005790 742 tablet_service.cc:1505] Processing CreateTablet for tablet 13ceabb190494e718a52892faaa53986 (DEFAULT_TABLE table=default.Table_f14b182e182d40afa389182bafa08912 [id=52b4d4e401ff405db42878f24247f818]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.005865 752 tablet_service.cc:1505] Processing CreateTablet for tablet a56f1a7bd5464d289bfd018c826ee574 (DEFAULT_TABLE table=default.Table_e250eb3a69ef4e6da3893852edc00469 [id=ba41048fe47d4749b5e084582989eb38]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.006775 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13ceabb190494e718a52892faaa53986. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.007300 756 tablet_service.cc:1505] Processing CreateTablet for tablet 8e09b172980c4e40b1d3f695d8e54c37 (DEFAULT_TABLE table=default.Table_574610b44ec44cbfa5a7071d1956f2f6 [id=1a595087add94970af8aca08c75bd834]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.008191 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a56f1a7bd5464d289bfd018c826ee574. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.008641 1166 tablet_bootstrap.cc:492] T d5f0ba3ab16240f9822be69f941f5da2 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:32.008867 1166 ts_tablet_manager.cc:1403] T d5f0ba3ab16240f9822be69f941f5da2 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.005s
I20251211 20:52:32.010088 1166 raft_consensus.cc:359] T d5f0ba3ab16240f9822be69f941f5da2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.010423 1166 raft_consensus.cc:385] T d5f0ba3ab16240f9822be69f941f5da2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:32.010599 1166 raft_consensus.cc:740] T d5f0ba3ab16240f9822be69f941f5da2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:32.010905 1166 consensus_queue.cc:260] T d5f0ba3ab16240f9822be69f941f5da2 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.015686 753 tablet_service.cc:1552] Processing DeleteTablet for tablet f9a3fdf17f274c15ac0bdfa9c174f649 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:32.016031 751 tablet_service.cc:1505] Processing CreateTablet for tablet 1c4b2c22bd4c456eb77af45576de466f (DEFAULT_TABLE table=default.Table_e891143fbbcd41509b6372a316eaac2c [id=3e7d6cefcb7d412191c6540e9ad77c2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.016852 1166 ts_tablet_manager.cc:1434] T d5f0ba3ab16240f9822be69f941f5da2 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20251211 20:52:32.018436 744 tablet_service.cc:1505] Processing CreateTablet for tablet aeff1fbabf7845dc9863f59f286ddcec (DEFAULT_TABLE table=default.Table_06bc050f989a4d97a4f76d09b835932d [id=22001d1e17ee44af9bc7bb5cd1540cf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.021253 1166 tablet_bootstrap.cc:492] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:32.022229 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c4b2c22bd4c456eb77af45576de466f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.023905 1166 tablet_bootstrap.cc:654] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:32.023923 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeff1fbabf7845dc9863f59f286ddcec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.024819 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e09b172980c4e40b1d3f695d8e54c37. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.030766 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 01104975fd8b470aa026578b97ff6b8d
I20251211 20:52:32.032377 1166 tablet_bootstrap.cc:492] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:32.032557 1166 ts_tablet_manager.cc:1403] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.005s
I20251211 20:52:32.033397 1166 raft_consensus.cc:359] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.033613 1166 raft_consensus.cc:385] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:32.033687 1166 raft_consensus.cc:740] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:32.033895 1166 consensus_queue.cc:260] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.034253 753 tablet_service.cc:1552] Processing DeleteTablet for tablet 01104975fd8b470aa026578b97ff6b8d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:32.038400 1013 tablet_service.cc:1505] Processing CreateTablet for tablet 1c4b2c22bd4c456eb77af45576de466f (DEFAULT_TABLE table=default.Table_e891143fbbcd41509b6372a316eaac2c [id=3e7d6cefcb7d412191c6540e9ad77c2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.038811 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c4b2c22bd4c456eb77af45576de466f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.039068 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 13ceabb190494e718a52892faaa53986 (DEFAULT_TABLE table=default.Table_f14b182e182d40afa389182bafa08912 [id=52b4d4e401ff405db42878f24247f818]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.039544 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13ceabb190494e718a52892faaa53986. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.041301 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 14e2650f8ec1482ea9511b989fdc963c
I20251211 20:52:32.043931 1018 tablet_service.cc:1505] Processing CreateTablet for tablet aeff1fbabf7845dc9863f59f286ddcec (DEFAULT_TABLE table=default.Table_06bc050f989a4d97a4f76d09b835932d [id=22001d1e17ee44af9bc7bb5cd1540cf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.044354 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeff1fbabf7845dc9863f59f286ddcec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.045727 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a2442a4ebc940a281f9ea1a394f6621 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:32.047891 1029 tablet_service.cc:1505] Processing CreateTablet for tablet a56f1a7bd5464d289bfd018c826ee574 (DEFAULT_TABLE table=default.Table_e250eb3a69ef4e6da3893852edc00469 [id=ba41048fe47d4749b5e084582989eb38]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.048238 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a56f1a7bd5464d289bfd018c826ee574. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.050165 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 3acfef6f082641a199b80eefcfc772ac
I20251211 20:52:32.051813 1019 tablet_service.cc:1552] Processing DeleteTablet for tablet d5f0ba3ab16240f9822be69f941f5da2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:32.052011 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet 01104975fd8b470aa026578b97ff6b8d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:32.052268 1163 tablet_replica.cc:333] T 6a2442a4ebc940a281f9ea1a394f6621 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:32.052464 1163 raft_consensus.cc:2243] T 6a2442a4ebc940a281f9ea1a394f6621 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:32.052664 1163 raft_consensus.cc:2272] T 6a2442a4ebc940a281f9ea1a394f6621 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:32.054253 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 804a11c8f7414eb1a6bcff2952be9ae2 (table default.Table_415c09e71243460aaa9acfd57742bcd7 [id=7f9092a9ef3945a7851d0d9f762e161a]) successfully deleted
I20251211 20:52:32.054484 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet f93006227cc6489e91f3593e11789b54 (table default.Table_4cd9b395cfa94302b298a3df4971d631 [id=42b8ff20f93f4d3fa10fe7ff92dc598d]) successfully deleted
I20251211 20:52:32.054759 753 tablet_service.cc:1552] Processing DeleteTablet for tablet 14e2650f8ec1482ea9511b989fdc963c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:32.055142 753 tablet_service.cc:1552] Processing DeleteTablet for tablet 3acfef6f082641a199b80eefcfc772ac with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:32.057266 1016 tablet_service.cc:1505] Processing CreateTablet for tablet 8e09b172980c4e40b1d3f695d8e54c37 (DEFAULT_TABLE table=default.Table_574610b44ec44cbfa5a7071d1956f2f6 [id=1a595087add94970af8aca08c75bd834]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.057659 1016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e09b172980c4e40b1d3f695d8e54c37. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.059880 1028 tablet_service.cc:1505] Processing CreateTablet for tablet 1b8acfdf1a26427aa9863280dc39b818 (DEFAULT_TABLE table=default.Table_78bd2a084ad542c2968d1338f1ac71b5 [id=9649589ef6e94eebb27bb2b518105d2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.060235 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b8acfdf1a26427aa9863280dc39b818. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.061540 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 648f43e95e034f2188a20e1b837f2818
I20251211 20:52:32.061993 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 14e2650f8ec1482ea9511b989fdc963c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:32.062270 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet 3acfef6f082641a199b80eefcfc772ac with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:32.051663 1011 tablet_service.cc:1552] Processing DeleteTablet for tablet f9a3fdf17f274c15ac0bdfa9c174f649 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:32.063103 1163 ts_tablet_manager.cc:1916] T 6a2442a4ebc940a281f9ea1a394f6621 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:32.068463 1011 tablet_service.cc:1552] Processing DeleteTablet for tablet f7f52d2547534ef99255c5d3f59410c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:32.069200 1011 tablet_service.cc:1505] Processing CreateTablet for tablet fcf5afcfd7f142d29ff91272913e5fcb (DEFAULT_TABLE table=default.Table_5453a73a163149baa88d85b027bb42ee [id=b6d9975565684e9784951ede7dbd7988]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.069608 1011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcf5afcfd7f142d29ff91272913e5fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.072082 1024 tablet_service.cc:1505] Processing CreateTablet for tablet 47428f41973642dbbd42ba051ea3d3b0 (DEFAULT_TABLE table=default.Table_68c2d3a1dc614ca4bcf28180d6e4faf8 [id=7e7414fd3d1e4ec2b7372f5783297766]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.072460 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47428f41973642dbbd42ba051ea3d3b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.075822 1667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:52:32.077562 1667 hybrid_clock.cc:648] HybridClock initialized: now 1765486352077427 us; error 127 us; skew 500 ppm
I20251211 20:52:32.078090 1667 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:52:32.082868 1010 tablet_service.cc:1505] Processing CreateTablet for tablet fbc2d4aea084492c8270ad1d87096e58 (DEFAULT_TABLE table=default.Table_9d7fd7d87cf049c28685144bb909d0c3 [id=669be6263d1744068a8decf4550fbf45]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.083294 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbc2d4aea084492c8270ad1d87096e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.085716 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 07e4336171714da8adc21977ce33ce50 (DEFAULT_TABLE table=default.Table_7dc91b2b86fc4c95b162a34345fc8443 [id=fe2e8383ba2d46c1a3e0b1ba3d3d4aa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.086102 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07e4336171714da8adc21977ce33ce50. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.094758 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_5fbbb8bea9fb4160a6ef7f69d736549d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:32.095395 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5fbbb8bea9fb4160a6ef7f69d736549d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.096424 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_5a5a43caab0f4ed3a42c60ab127ca9bd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:32.096774 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_813136dbd41e4469b377741cec73e21d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:32.096932 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a5a43caab0f4ed3a42c60ab127ca9bd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:32.097254 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_813136dbd41e4469b377741cec73e21d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.098673 595 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_1d00ad67ef20403b94de5d39425be56d" } new_table_name: "default.Table_d18401707c88493abdf90d2c713432ae" modify_external_catalogs: true
I20251211 20:52:32.098701 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_0e678448f89e48f0ab54479b58c50c84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:32.099189 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0e678448f89e48f0ab54479b58c50c84 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.099227 1612 raft_consensus.cc:2749] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:32.099480 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 78922f26cedb4bb989aa1c1cb40bfa3a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:32.099893 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e5def7ad31744669b691d5517da9c02b" } new_table_name: "default.Table_31cc05174eeb48948593096a3613f70c" modify_external_catalogs: true
I20251211 20:52:32.101006 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_018c974d7bdf4d5abae02a0ca19b05d9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:32.101418 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_018c974d7bdf4d5abae02a0ca19b05d9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.094741 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_58da4dae54de4c8e9d471de1711d3d6d" } new_table_name: "default.Table_4a209d1f3ab4458bb6197e926297e60a" modify_external_catalogs: true
I20251211 20:52:32.110597 588 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_d7d5d2408f644d09b54b10d82a401116" } new_table_name: "default.Table_d89334f1d2b04caf9c31917b346399b5" modify_external_catalogs: true
I20251211 20:52:32.122031 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69085cd41db846e7a6e03bcfee4377f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.123809 890 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b736a1b347d44b398980654caa21332 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:32.126513 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6663f13691a24fb991b48f543d9de76b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.127900 1612 raft_consensus.cc:493] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251211 20:52:32.128867 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 645b08b33676420c96e880de3d460829 consensus state has no leader
W20251211 20:52:32.133533 595 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a3b1aadf17ea4bf0a896d23cd543fca7 consensus state has no leader
W20251211 20:52:32.133978 588 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 10746162da4640e6a2a891467690b770 consensus state has no leader
I20251211 20:52:32.137847 1612 raft_consensus.cc:515] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:32.138175 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e1aeae8c18b6499e9dc1accabdc3f633 consensus state has no leader
I20251211 20:52:32.140571 1160 tablet_bootstrap.cc:492] T c4e048ca28824329b924845ccf5067cf P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:32.140760 1160 ts_tablet_manager.cc:1403] T c4e048ca28824329b924845ccf5067cf P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.256s user 0.040s sys 0.083s
I20251211 20:52:32.142637 1612 leader_election.cc:290] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:32.143704 1612 raft_consensus.cc:493] T dd425601d89148eb95525f796a3abf1e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:32.143859 1612 raft_consensus.cc:515] T dd425601d89148eb95525f796a3abf1e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:32.143935 847 leader_election.cc:343] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 78922f26cedb4bb989aa1c1cb40bfa3a
I20251211 20:52:32.150714 1019 tablet_service.cc:1505] Processing CreateTablet for tablet dfd194cf00f549c8937cb6ec40987240 (DEFAULT_TABLE table=default.Table_8db9acb9d35149fdabcb9a1d24061b46 [id=d919c6132b3c466e8a1404f981b003b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.151106 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 648f43e95e034f2188a20e1b837f2818 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:32.151374 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfd194cf00f549c8937cb6ec40987240. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.153249 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_015f1fc2445947f38320ea7699cc7388"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:32.153844 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_015f1fc2445947f38320ea7699cc7388 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.155103 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_cce138cd7e6f4a91aeb7632af1b61ade"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:32.155645 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cce138cd7e6f4a91aeb7632af1b61ade in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.157174 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b3b56f469dc54741924e5b55045fa626"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:32.157675 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b3b56f469dc54741924e5b55045fa626 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.158957 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_aeca38849f1341c4b309630c08331c13" } new_table_name: "default.Table_6460313002684bba93b30ad8631f0f49" modify_external_catalogs: true
I20251211 20:52:32.157796 1160 raft_consensus.cc:359] T c4e048ca28824329b924845ccf5067cf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.159353 1160 raft_consensus.cc:385] T c4e048ca28824329b924845ccf5067cf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:32.159759 1165 ts_tablet_manager.cc:1929] T dc40b9826ea84728910b1fd61ce62a25 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:32.160701 1696 raft_consensus.cc:493] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:32.161775 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dd425601d89148eb95525f796a3abf1e" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:32.162433 1160 raft_consensus.cc:740] T c4e048ca28824329b924845ccf5067cf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:32.162851 1612 leader_election.cc:290] T dd425601d89148eb95525f796a3abf1e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:32.163673 1696 raft_consensus.cc:515] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.166939 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dd425601d89148eb95525f796a3abf1e" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:32.164750 1165 log.cc:1199] T dc40b9826ea84728910b1fd61ce62a25 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/dc40b9826ea84728910b1fd61ce62a25
I20251211 20:52:32.168356 1165 ts_tablet_manager.cc:1950] T dc40b9826ea84728910b1fd61ce62a25 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:32.169265 1165 tablet_replica.cc:333] T 4b736a1b347d44b398980654caa21332 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:32.169983 1165 raft_consensus.cc:2243] T 4b736a1b347d44b398980654caa21332 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:32.170146 1165 raft_consensus.cc:2272] T 4b736a1b347d44b398980654caa21332 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:32.170432 1696 leader_election.cc:290] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:32.172036 1165 ts_tablet_manager.cc:1916] T 4b736a1b347d44b398980654caa21332 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:32.172508 1168 tablet_bootstrap.cc:492] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:32.172650 1168 ts_tablet_manager.cc:1403] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.278s user 0.019s sys 0.042s
I20251211 20:52:32.173458 1168 raft_consensus.cc:359] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.173663 1168 raft_consensus.cc:385] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:32.173756 1168 raft_consensus.cc:740] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:32.173992 1168 consensus_queue.cc:260] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.177312 1168 ts_tablet_manager.cc:1434] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251211 20:52:32.177827 1168 tablet_bootstrap.cc:492] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:32.179783 1168 tablet_bootstrap.cc:654] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:32.179693 1160 consensus_queue.cc:260] T c4e048ca28824329b924845ccf5067cf P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:32.186721 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 969a8587019c4202b6e705d126f83d44 consensus state has no leader
I20251211 20:52:32.187336 1696 raft_consensus.cc:493] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:32.187510 1696 raft_consensus.cc:515] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.188999 1010 tablet_service.cc:1505] Processing CreateTablet for tablet a5b2f4079315477bb5862629b7896f80 (DEFAULT_TABLE table=default.Table_e887667aa38d47e9814a79f7b0e6990d [id=9d71d585608f49f48128aa54ecd0b724]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.193857 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_4b43e2845ad847e39b035931723fe49f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:32.194391 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4b43e2845ad847e39b035931723fe49f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.195561 1696 leader_election.cc:290] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:32.196101 847 leader_election.cc:343] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 78922f26cedb4bb989aa1c1cb40bfa3a
W20251211 20:52:32.196336 710 leader_election.cc:343] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: c4b76a7e84d344268567c551236ce772
W20251211 20:52:32.196565 710 leader_election.cc:343] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 19d0aa259ac94f36b7f6a24d11a3d420
I20251211 20:52:32.196962 756 tablet_service.cc:1505] Processing CreateTablet for tablet fcf5afcfd7f142d29ff91272913e5fcb (DEFAULT_TABLE table=default.Table_5453a73a163149baa88d85b027bb42ee [id=b6d9975565684e9784951ede7dbd7988]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.197850 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c4b76a7e84d344268567c551236ce772" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:32.198123 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcf5afcfd7f142d29ff91272913e5fcb. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:32.201073 711 leader_election.cc:343] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:32.201205 711 leader_election.cc:304] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:32.202075 754 tablet_service.cc:1505] Processing CreateTablet for tablet 47428f41973642dbbd42ba051ea3d3b0 (DEFAULT_TABLE table=default.Table_68c2d3a1dc614ca4bcf28180d6e4faf8 [id=7e7414fd3d1e4ec2b7372f5783297766]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.202775 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47428f41973642dbbd42ba051ea3d3b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.204957 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_01868fec1884449da230c4236ab2d506" } modify_external_catalogs: true
I20251211 20:52:32.205138 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_01868fec1884449da230c4236ab2d506" } modify_external_catalogs: true
I20251211 20:52:32.206393 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_c496f43b462445bab2d5da82a2f69156"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:32.206959 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c496f43b462445bab2d5da82a2f69156 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.208321 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 648f43e95e034f2188a20e1b837f2818 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:32.209748 847 leader_election.cc:304] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:32.212460 1696 raft_consensus.cc:2749] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:32.213692 746 tablet_service.cc:1505] Processing CreateTablet for tablet 07e4336171714da8adc21977ce33ce50 (DEFAULT_TABLE table=default.Table_7dc91b2b86fc4c95b162a34345fc8443 [id=fe2e8383ba2d46c1a3e0b1ba3d3d4aa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.214612 741 tablet_service.cc:1505] Processing CreateTablet for tablet a4ccf9a4ecbc4ca6978a189cf3a15893 (DEFAULT_TABLE table=default.Table_aab9c3f649474051bcf63611d32aa48a [id=bafad2cf86e6404aaaba777365ad4838]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.214871 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 9adaee28edd74df68b44b25d505fdfcb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:32.215181 757 tablet_service.cc:1505] Processing CreateTablet for tablet dfd194cf00f549c8937cb6ec40987240 (DEFAULT_TABLE table=default.Table_8db9acb9d35149fdabcb9a1d24061b46 [id=d919c6132b3c466e8a1404f981b003b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.217566 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet dc40b9826ea84728910b1fd61ce62a25 (table default.Table_e6252813479645a98ce2709dc7aba798 [id=db38368266194271a02ad265fd991439]) successfully deleted
I20251211 20:52:32.218597 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5b2f4079315477bb5862629b7896f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.220959 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07e4336171714da8adc21977ce33ce50. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.221587 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 3621735fcb294e3a8c0180ba9ef86442
I20251211 20:52:32.222095 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4ccf9a4ecbc4ca6978a189cf3a15893. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.222998 754 tablet_service.cc:1505] Processing CreateTablet for tablet fbc2d4aea084492c8270ad1d87096e58 (DEFAULT_TABLE table=default.Table_9d7fd7d87cf049c28685144bb909d0c3 [id=669be6263d1744068a8decf4550fbf45]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.223340 739 tablet_service.cc:1505] Processing CreateTablet for tablet 664c6a3afe5b4a4caf87dede87f199ba (DEFAULT_TABLE table=default.Table_c2f02a2463774499a7858b0a53ee965d [id=c342962bfd9242c6811897d1c980dfe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.223739 742 tablet_service.cc:1505] Processing CreateTablet for tablet a5b2f4079315477bb5862629b7896f80 (DEFAULT_TABLE table=default.Table_e887667aa38d47e9814a79f7b0e6990d [id=9d71d585608f49f48128aa54ecd0b724]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.222820 740 tablet_service.cc:1505] Processing CreateTablet for tablet ab4e7a9062134e489d6f9abe6ff26fb4 (DEFAULT_TABLE table=default.Table_8488ced67fe54a25918bbd9c8c9934e8 [id=b958532650374d609199cab9c07bb64b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.224179 747 tablet_service.cc:1505] Processing CreateTablet for tablet e8db5a11a69e4554bc689c38ab57b28a (DEFAULT_TABLE table=default.Table_4e49f78c60234aea972f8f5068e89b9f [id=6cf77cf74a6f48bcb7113538bfa28ffd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.224541 1160 ts_tablet_manager.cc:1434] T c4e048ca28824329b924845ccf5067cf P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.064s user 0.003s sys 0.000s
I20251211 20:52:32.224588 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfd194cf00f549c8937cb6ec40987240. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.224948 1160 tablet_bootstrap.cc:492] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:32.226855 1160 tablet_bootstrap.cc:654] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:32.235188 847 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.1:42177, user_credentials={real_user=slave}} blocked reactor thread for 49776.2us
I20251211 20:52:32.235970 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "19d0aa259ac94f36b7f6a24d11a3d420" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:32.237308 711 leader_election.cc:343] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:32.237483 711 leader_election.cc:304] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:32.239135 1612 raft_consensus.cc:2749] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:32.245060 1013 tablet_service.cc:1505] Processing CreateTablet for tablet 43044ba5660345eaa2b4e6562cc8d448 (DEFAULT_TABLE table=default.Table_54b89a8f30cf414590cfd7a5773bf326 [id=c3d36e0944a74f6eb98b9bb510d8352a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.245430 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43044ba5660345eaa2b4e6562cc8d448. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.246904 1165 ts_tablet_manager.cc:1929] T 4b736a1b347d44b398980654caa21332 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:32.247093 1165 log.cc:1199] T 4b736a1b347d44b398980654caa21332 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/4b736a1b347d44b398980654caa21332
I20251211 20:52:32.247514 1165 ts_tablet_manager.cc:1950] T 4b736a1b347d44b398980654caa21332 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:32.247551 847 leader_election.cc:304] T dd425601d89148eb95525f796a3abf1e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:32.248145 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 664c6a3afe5b4a4caf87dede87f199ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.249516 1020 tablet_service.cc:1505] Processing CreateTablet for tablet cc43ee61e78249bfa00b11f3a1fa21a3 (DEFAULT_TABLE table=default.Table_fde35da450a941b38489c3954c1a3839 [id=25f259d8c2e34d63b7857492d92d8c1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.249951 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc43ee61e78249bfa00b11f3a1fa21a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.223989 1024 tablet_service.cc:1505] Processing CreateTablet for tablet ab4e7a9062134e489d6f9abe6ff26fb4 (DEFAULT_TABLE table=default.Table_8488ced67fe54a25918bbd9c8c9934e8 [id=b958532650374d609199cab9c07bb64b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.224473 1011 tablet_service.cc:1505] Processing CreateTablet for tablet 664c6a3afe5b4a4caf87dede87f199ba (DEFAULT_TABLE table=default.Table_c2f02a2463774499a7858b0a53ee965d [id=c342962bfd9242c6811897d1c980dfe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.251675 1011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 664c6a3afe5b4a4caf87dede87f199ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.254722 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbc2d4aea084492c8270ad1d87096e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.257817 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab4e7a9062134e489d6f9abe6ff26fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.258466 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab4e7a9062134e489d6f9abe6ff26fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.260931 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 83067b139f504bc191a515feed0ec771 (DEFAULT_TABLE table=default.Table_8f7e347b9ac64793b1073dfa955ebe21 [id=a6ab64c3e0d94462b716f2d153bc3314]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.261314 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83067b139f504bc191a515feed0ec771. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.224694 1016 tablet_service.cc:1505] Processing CreateTablet for tablet a4ccf9a4ecbc4ca6978a189cf3a15893 (DEFAULT_TABLE table=default.Table_aab9c3f649474051bcf63611d32aa48a [id=bafad2cf86e6404aaaba777365ad4838]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.263868 1016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4ccf9a4ecbc4ca6978a189cf3a15893. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.263932 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5b2f4079315477bb5862629b7896f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.267822 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8db5a11a69e4554bc689c38ab57b28a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.268870 1574 raft_consensus.cc:2749] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:32.269488 882 tablet_service.cc:1505] Processing CreateTablet for tablet 1b8acfdf1a26427aa9863280dc39b818 (DEFAULT_TABLE table=default.Table_78bd2a084ad542c2968d1338f1ac71b5 [id=9649589ef6e94eebb27bb2b518105d2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.274269 877 tablet_service.cc:1505] Processing CreateTablet for tablet 8e09b172980c4e40b1d3f695d8e54c37 (DEFAULT_TABLE table=default.Table_574610b44ec44cbfa5a7071d1956f2f6 [id=1a595087add94970af8aca08c75bd834]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.278857 1165 tablet_replica.cc:333] T b8828385cec34d70b717ae7f3813b9c3 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:32.280247 1612 raft_consensus.cc:493] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:32.227223 1163 ts_tablet_manager.cc:1929] T 6a2442a4ebc940a281f9ea1a394f6621 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:32.239110 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 88970b575ba943129729eceef5636519 (DEFAULT_TABLE table=default.Table_b377fea34b4145719629f5ca57b25a41 [id=df63f62424bf4da28edb775ecc32ad0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.282116 1165 raft_consensus.cc:2243] T b8828385cec34d70b717ae7f3813b9c3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:32.280289 756 tablet_service.cc:1505] Processing CreateTablet for tablet cc43ee61e78249bfa00b11f3a1fa21a3 (DEFAULT_TABLE table=default.Table_fde35da450a941b38489c3954c1a3839 [id=25f259d8c2e34d63b7857492d92d8c1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.280647 740 tablet_service.cc:1505] Processing CreateTablet for tablet 06ba98a726424dd89450053c5c5793f4 (DEFAULT_TABLE table=default.Table_bdb9aa9507d94416bbc3c8fe9cb11fa7 [id=367894d4c46a4cc6bde351a099a635b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.280678 741 tablet_service.cc:1505] Processing CreateTablet for tablet 83067b139f504bc191a515feed0ec771 (DEFAULT_TABLE table=default.Table_8f7e347b9ac64793b1073dfa955ebe21 [id=a6ab64c3e0d94462b716f2d153bc3314]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.280972 746 tablet_service.cc:1505] Processing CreateTablet for tablet bc4e7362e6d045da81bf39f4135f500e (DEFAULT_TABLE table=default.Table_c0ed4563b1d44eafa92065d3ec8adb93 [id=9e7c08f833e14a90b17a9b9f60b702de]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.281064 739 tablet_service.cc:1505] Processing CreateTablet for tablet 845adb6038fc4c4dbc94418d8ebe3fec (DEFAULT_TABLE table=default.Table_4e2726de74fd437796c8e0e6dd8b3af0 [id=6bca2b1cbc244d34b21efa75f651d35f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.281451 757 tablet_service.cc:1505] Processing CreateTablet for tablet 43044ba5660345eaa2b4e6562cc8d448 (DEFAULT_TABLE table=default.Table_54b89a8f30cf414590cfd7a5773bf326 [id=c3d36e0944a74f6eb98b9bb510d8352a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.281801 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 3621735fcb294e3a8c0180ba9ef86442 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:32.279886 747 tablet_service.cc:1505] Processing CreateTablet for tablet 88970b575ba943129729eceef5636519 (DEFAULT_TABLE table=default.Table_b377fea34b4145719629f5ca57b25a41 [id=df63f62424bf4da28edb775ecc32ad0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.283169 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83067b139f504bc191a515feed0ec771. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.284580 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc43ee61e78249bfa00b11f3a1fa21a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.285962 889 tablet_service.cc:1505] Processing CreateTablet for tablet 13ceabb190494e718a52892faaa53986 (DEFAULT_TABLE table=default.Table_f14b182e182d40afa389182bafa08912 [id=52b4d4e401ff405db42878f24247f818]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.286522 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b8acfdf1a26427aa9863280dc39b818. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.286806 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 4b736a1b347d44b398980654caa21332 (table default.Table_f9014306b36e47b98c8e4c6070ea4a37 [id=4e64e54fb4fc42d48679371e4bc1efe5]) successfully deleted
I20251211 20:52:32.287492 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06ba98a726424dd89450053c5c5793f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.290390 1752 raft_consensus.cc:2749] T dd425601d89148eb95525f796a3abf1e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:32.291728 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 845adb6038fc4c4dbc94418d8ebe3fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.292054 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13ceabb190494e718a52892faaa53986. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.293839 1586 raft_consensus.cc:493] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:32.294030 1586 raft_consensus.cc:515] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.294052 1612 raft_consensus.cc:515] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } }
I20251211 20:52:32.295213 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet 3621735fcb294e3a8c0180ba9ef86442 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:32.295558 1586 leader_election.cc:290] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:32.296018 879 tablet_service.cc:1505] Processing CreateTablet for tablet aeff1fbabf7845dc9863f59f286ddcec (DEFAULT_TABLE table=default.Table_06bc050f989a4d97a4f76d09b835932d [id=22001d1e17ee44af9bc7bb5cd1540cf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.296509 1163 log.cc:1199] T 6a2442a4ebc940a281f9ea1a394f6621 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/6a2442a4ebc940a281f9ea1a394f6621
I20251211 20:52:32.296500 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c4e048ca28824329b924845ccf5067cf" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:32.296638 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43044ba5660345eaa2b4e6562cc8d448. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.297219 745 tablet_service.cc:1505] Processing CreateTablet for tablet 4fc56df5ccb94198b5b0e516eeae6941 (DEFAULT_TABLE table=default.Table_494ee7d3e0dc4a47b47082643a3134ce [id=42422ac9805e49cbab71e262e73098b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.297382 913 raft_consensus.cc:2468] T c4e048ca28824329b924845ccf5067cf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 0.
I20251211 20:52:32.297580 1165 raft_consensus.cc:2272] T b8828385cec34d70b717ae7f3813b9c3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:32.297780 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c4e048ca28824329b924845ccf5067cf" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:32.297993 776 raft_consensus.cc:2468] T c4e048ca28824329b924845ccf5067cf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 0.
I20251211 20:52:32.298154 1165 ts_tablet_manager.cc:1916] T b8828385cec34d70b717ae7f3813b9c3 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:32.298599 983 leader_election.cc:304] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41; no voters:
W20251211 20:52:32.299700 847 leader_election.cc:343] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 2260850f32b541aab1128818f144b5a7
W20251211 20:52:32.300942 847 leader_election.cc:343] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 2260850f32b541aab1128818f144b5a7
I20251211 20:52:32.301249 847 leader_election.cc:304] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:32.303326 888 tablet_service.cc:1505] Processing CreateTablet for tablet 1c4b2c22bd4c456eb77af45576de466f (DEFAULT_TABLE table=default.Table_e891143fbbcd41509b6372a316eaac2c [id=3e7d6cefcb7d412191c6540e9ad77c2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.303499 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 3acfef6f082641a199b80eefcfc772ac with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:32.303603 892 tablet_service.cc:1552] Processing DeleteTablet for tablet 648f43e95e034f2188a20e1b837f2818 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:32.303779 881 tablet_service.cc:1552] Processing DeleteTablet for tablet f9a3fdf17f274c15ac0bdfa9c174f649 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:32.303978 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 01104975fd8b470aa026578b97ff6b8d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:32.304126 890 tablet_service.cc:1552] Processing DeleteTablet for tablet 14e2650f8ec1482ea9511b989fdc963c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:32.304214 881 tablet_service.cc:1505] Processing CreateTablet for tablet fcf5afcfd7f142d29ff91272913e5fcb (DEFAULT_TABLE table=default.Table_5453a73a163149baa88d85b027bb42ee [id=b6d9975565684e9784951ede7dbd7988]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.304747 1166 ts_tablet_manager.cc:1434] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.264s user 0.001s sys 0.001s
I20251211 20:52:32.305881 892 tablet_service.cc:1505] Processing CreateTablet for tablet fbc2d4aea084492c8270ad1d87096e58 (DEFAULT_TABLE table=default.Table_9d7fd7d87cf049c28685144bb909d0c3 [id=669be6263d1744068a8decf4550fbf45]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.306955 894 tablet_service.cc:1505] Processing CreateTablet for tablet a56f1a7bd5464d289bfd018c826ee574 (DEFAULT_TABLE table=default.Table_e250eb3a69ef4e6da3893852edc00469 [id=ba41048fe47d4749b5e084582989eb38]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.309628 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88970b575ba943129729eceef5636519. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.305162 890 tablet_service.cc:1505] Processing CreateTablet for tablet 47428f41973642dbbd42ba051ea3d3b0 (DEFAULT_TABLE table=default.Table_68c2d3a1dc614ca4bcf28180d6e4faf8 [id=7e7414fd3d1e4ec2b7372f5783297766]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.314702 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88970b575ba943129729eceef5636519. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.315640 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc4e7362e6d045da81bf39f4135f500e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.314306 1586 raft_consensus.cc:2804] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:32.316270 1586 raft_consensus.cc:493] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:32.316403 1586 raft_consensus.cc:3060] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:32.317302 1018 tablet_service.cc:1505] Processing CreateTablet for tablet e8db5a11a69e4554bc689c38ab57b28a (DEFAULT_TABLE table=default.Table_4e49f78c60234aea972f8f5068e89b9f [id=6cf77cf74a6f48bcb7113538bfa28ffd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.317759 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8db5a11a69e4554bc689c38ab57b28a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.319139 1612 leader_election.cc:290] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:32.319890 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fc56df5ccb94198b5b0e516eeae6941. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.320665 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 845adb6038fc4c4dbc94418d8ebe3fec (DEFAULT_TABLE table=default.Table_4e2726de74fd437796c8e0e6dd8b3af0 [id=6bca2b1cbc244d34b21efa75f651d35f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.321389 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeff1fbabf7845dc9863f59f286ddcec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.321648 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbc2d4aea084492c8270ad1d87096e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.322249 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c4b2c22bd4c456eb77af45576de466f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.323503 1586 raft_consensus.cc:515] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.325143 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcf5afcfd7f142d29ff91272913e5fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.325791 1574 raft_consensus.cc:493] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:32.325950 1574 raft_consensus.cc:515] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.326573 1574 leader_election.cc:290] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:32.326764 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e09b172980c4e40b1d3f695d8e54c37. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.324859 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 845adb6038fc4c4dbc94418d8ebe3fec. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:32.327644 710 leader_election.cc:343] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: c910246b3d5d4e7e8a71d9f18e94d24c
I20251211 20:52:32.325053 1011 tablet_service.cc:1505] Processing CreateTablet for tablet 4fc56df5ccb94198b5b0e516eeae6941 (DEFAULT_TABLE table=default.Table_494ee7d3e0dc4a47b47082643a3134ce [id=42422ac9805e49cbab71e262e73098b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.328779 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47428f41973642dbbd42ba051ea3d3b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.329246 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c4e048ca28824329b924845ccf5067cf" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:32.329350 1163 ts_tablet_manager.cc:1950] T 6a2442a4ebc940a281f9ea1a394f6621 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:32.329442 776 raft_consensus.cc:3060] T c4e048ca28824329b924845ccf5067cf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:32.330416 1163 tablet_replica.cc:333] T d5f0ba3ab16240f9822be69f941f5da2 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:32.330822 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 6a2442a4ebc940a281f9ea1a394f6621 (table default.Table_cbe68d84efef4745822f6aef5a723f11 [id=f9c1230a7d71435cac705d4675a9908d]) successfully deleted
I20251211 20:52:32.331199 1163 raft_consensus.cc:2243] T d5f0ba3ab16240f9822be69f941f5da2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:32.331414 1163 raft_consensus.cc:2272] T d5f0ba3ab16240f9822be69f941f5da2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:32.332166 776 raft_consensus.cc:2468] T c4e048ca28824329b924845ccf5067cf P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 1.
I20251211 20:52:32.333171 1011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fc56df5ccb94198b5b0e516eeae6941. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.335171 982 leader_election.cc:304] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:32.336450 912 raft_consensus.cc:3060] T dd425601d89148eb95525f796a3abf1e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:32.340193 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a56f1a7bd5464d289bfd018c826ee574. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.341384 1163 ts_tablet_manager.cc:1916] T d5f0ba3ab16240f9822be69f941f5da2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:32.342361 1752 raft_consensus.cc:2749] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:32.342919 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c4e048ca28824329b924845ccf5067cf" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:32.343119 914 raft_consensus.cc:3060] T c4e048ca28824329b924845ccf5067cf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:32.343545 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c910246b3d5d4e7e8a71d9f18e94d24c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:32.344193 711 leader_election.cc:343] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:32.344324 711 leader_election.cc:304] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:32.344586 1574 raft_consensus.cc:2749] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:32.242177 1028 tablet_service.cc:1505] Processing CreateTablet for tablet bc4e7362e6d045da81bf39f4135f500e (DEFAULT_TABLE table=default.Table_c0ed4563b1d44eafa92065d3ec8adb93 [id=9e7c08f833e14a90b17a9b9f60b702de]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.241837 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 06ba98a726424dd89450053c5c5793f4 (DEFAULT_TABLE table=default.Table_bdb9aa9507d94416bbc3c8fe9cb11fa7 [id=367894d4c46a4cc6bde351a099a635b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.334939 1586 leader_election.cc:290] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Requested vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:32.351934 1586 raft_consensus.cc:2804] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:32.352102 1586 raft_consensus.cc:697] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 LEADER]: Becoming Leader. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Running, Role: LEADER
I20251211 20:52:32.352509 1586 consensus_queue.cc:237] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.353900 914 raft_consensus.cc:2468] T c4e048ca28824329b924845ccf5067cf P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 1.
I20251211 20:52:32.355427 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc4e7362e6d045da81bf39f4135f500e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.362753 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06ba98a726424dd89450053c5c5793f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.363955 746 tablet_service.cc:1505] Processing CreateTablet for tablet ba0e3a6b2699493ba9b9627b7d944b74 (DEFAULT_TABLE table=default.Table_cce138cd7e6f4a91aeb7632af1b61ade [id=7f521f77d3234e5a9b9612598e00fc94]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.364509 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba0e3a6b2699493ba9b9627b7d944b74. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.369783 745 tablet_service.cc:1505] Processing CreateTablet for tablet 36d91772201d4c5b928d9dee3f583eb4 (DEFAULT_TABLE table=default.Table_b3b56f469dc54741924e5b55045fa626 [id=471a1295a9534492aff0ae89a5362a1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.370862 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36d91772201d4c5b928d9dee3f583eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.372432 1667 webserver.cc:492] Webserver started at http://127.0.104.61:42143/ using document root <none> and password file <none>
I20251211 20:52:32.375118 1667 fs_manager.cc:362] Metadata directory not provided
W20251211 20:52:32.375681 582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.61:40749, user_credentials={real_user=slave}} blocked reactor thread for 102017us
W20251211 20:52:32.376008 582 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 645b08b33676420c96e880de3d460829 (table default.Table_31cc05174eeb48948593096a3613f70c [id=6d42f260246e420d8818b3841926302b], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 645b08b33676420c96e880de3d460829 consensus state has no leader
I20251211 20:52:32.376720 1165 ts_tablet_manager.cc:1929] T b8828385cec34d70b717ae7f3813b9c3 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:32.377239 1165 log.cc:1199] T b8828385cec34d70b717ae7f3813b9c3 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/b8828385cec34d70b717ae7f3813b9c3
I20251211 20:52:32.377786 739 tablet_service.cc:1505] Processing CreateTablet for tablet 3c5819ae1abb48fe8a83af9067726d02 (DEFAULT_TABLE table=default.Table_015f1fc2445947f38320ea7699cc7388 [id=34516c0e51a64f3d88a7d5744160367c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.378232 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b377fea34b4145719629f5ca57b25a41" } modify_external_catalogs: true
I20251211 20:52:32.378461 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b377fea34b4145719629f5ca57b25a41" } modify_external_catalogs: true
I20251211 20:52:32.378502 1165 ts_tablet_manager.cc:1950] T b8828385cec34d70b717ae7f3813b9c3 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:32.379192 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_1e17ae4fb7e84e41aea563972b80af32" } modify_external_catalogs: true
I20251211 20:52:32.379375 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_1e17ae4fb7e84e41aea563972b80af32" } modify_external_catalogs: true
I20251211 20:52:32.380298 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c4c8f005f6fe4b7ebcc05d11911718e5" } modify_external_catalogs: true
I20251211 20:52:32.380473 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c4c8f005f6fe4b7ebcc05d11911718e5" } modify_external_catalogs: true
I20251211 20:52:32.383127 1014 tablet_service.cc:1505] Processing CreateTablet for tablet 36d91772201d4c5b928d9dee3f583eb4 (DEFAULT_TABLE table=default.Table_b3b56f469dc54741924e5b55045fa626 [id=471a1295a9534492aff0ae89a5362a1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.383610 1013 tablet_service.cc:1505] Processing CreateTablet for tablet ba0e3a6b2699493ba9b9627b7d944b74 (DEFAULT_TABLE table=default.Table_cce138cd7e6f4a91aeb7632af1b61ade [id=7f521f77d3234e5a9b9612598e00fc94]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.383913 891 tablet_service.cc:1505] Processing CreateTablet for tablet 07e4336171714da8adc21977ce33ce50 (DEFAULT_TABLE table=default.Table_7dc91b2b86fc4c95b162a34345fc8443 [id=fe2e8383ba2d46c1a3e0b1ba3d3d4aa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.384266 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07e4336171714da8adc21977ce33ce50. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.384784 1014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36d91772201d4c5b928d9dee3f583eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.386670 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba0e3a6b2699493ba9b9627b7d944b74. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.388036 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 12aaf618d6474f599458026308d7afdd
I20251211 20:52:32.389014 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 88970b575ba943129729eceef5636519
I20251211 20:52:32.389168 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 12aaf618d6474f599458026308d7afdd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:32.390427 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 6663f13691a24fb991b48f543d9de76b
I20251211 20:52:32.391137 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_4721ef8d42714525ba28461e11e30dec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:32.391381 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_018c974d7bdf4d5abae02a0ca19b05d9" } modify_external_catalogs: true
I20251211 20:52:32.391551 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_018c974d7bdf4d5abae02a0ca19b05d9" } modify_external_catalogs: true
I20251211 20:52:32.391562 893 tablet_service.cc:1505] Processing CreateTablet for tablet 664c6a3afe5b4a4caf87dede87f199ba (DEFAULT_TABLE table=default.Table_c2f02a2463774499a7858b0a53ee965d [id=c342962bfd9242c6811897d1c980dfe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.391805 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_505604d692d54bdea110e89bb79743c4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:32.391985 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 664c6a3afe5b4a4caf87dede87f199ba. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:32.392328 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_505604d692d54bdea110e89bb79743c4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.392895 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_dbd8238e6f3c477085d6f3d231cc68fb" } new_table_name: "default.Table_72a1c37a786747d8910b240c2a831487" modify_external_catalogs: true
W20251211 20:52:32.393816 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4721ef8d42714525ba28461e11e30dec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.394227 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 88970b575ba943129729eceef5636519 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:32.396761 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 6663f13691a24fb991b48f543d9de76b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:32.399151 891 tablet_service.cc:1505] Processing CreateTablet for tablet a4ccf9a4ecbc4ca6978a189cf3a15893 (DEFAULT_TABLE table=default.Table_aab9c3f649474051bcf63611d32aa48a [id=bafad2cf86e6404aaaba777365ad4838]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.399573 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4ccf9a4ecbc4ca6978a189cf3a15893. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.400417 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 5ddf7c1972564ef88e3c0a0bd1b430e0
I20251211 20:52:32.401680 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0042a65523294a70b215f6e796d70cd5" } new_table_name: "default.Table_6b98cb90147a43c4aee6b39240c5edc4" modify_external_catalogs: true
I20251211 20:52:32.402436 885 tablet_service.cc:1505] Processing CreateTablet for tablet a5b2f4079315477bb5862629b7896f80 (DEFAULT_TABLE table=default.Table_e887667aa38d47e9814a79f7b0e6990d [id=9d71d585608f49f48128aa54ecd0b724]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.402812 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5b2f4079315477bb5862629b7896f80. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:32.403368 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b072b759d1254859b0b8d401e3078792 consensus state has no leader
I20251211 20:52:32.404520 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_a891ad881584489e87e69014ed59fed9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:32.405074 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a891ad881584489e87e69014ed59fed9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.406124 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_fd6097253c774917bd3fdf776d468873" } new_table_name: "default.Table_0d7c51baaa1541b98af0c9b474cf8d66" modify_external_catalogs: true
I20251211 20:52:32.406949 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 3621735fcb294e3a8c0180ba9ef86442 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:32.407243 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 5ddf7c1972564ef88e3c0a0bd1b430e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:32.407498 884 tablet_service.cc:1505] Processing CreateTablet for tablet bc4e7362e6d045da81bf39f4135f500e (DEFAULT_TABLE table=default.Table_c0ed4563b1d44eafa92065d3ec8adb93 [id=9e7c08f833e14a90b17a9b9f60b702de]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.407948 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc4e7362e6d045da81bf39f4135f500e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.408008 740 tablet_service.cc:1505] Processing CreateTablet for tablet 48820386e41b47129a13bdd5768f021b (DEFAULT_TABLE table=default.Table_4b43e2845ad847e39b035931723fe49f [id=6876565d3a9c48ce841b830cf75139ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.408931 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_46118198e87e4c158f00055e6867664b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:32.409614 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_46118198e87e4c158f00055e6867664b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:32.409762 592 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 49a33084c0fc460eb9592224b6dfc279 consensus state has no leader
I20251211 20:52:32.410458 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ea40e3f0df5c4db7b27261d32241d8d0" } new_table_name: "default.Table_c64075b948a646458740b55e6970fd94" modify_external_catalogs: true
I20251211 20:52:32.413594 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b06a07477d3146c288f85dad5e264daa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:32.413641 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_5a2e3b6cc2734852af9a0a9b8f51a69b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:32.414177 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b06a07477d3146c288f85dad5e264daa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.414943 875 tablet_service.cc:1505] Processing CreateTablet for tablet 88970b575ba943129729eceef5636519 (DEFAULT_TABLE table=default.Table_b377fea34b4145719629f5ca57b25a41 [id=df63f62424bf4da28edb775ecc32ad0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.415056 741 tablet_service.cc:1505] Processing CreateTablet for tablet e709051411354522bdb327c34dbf533a (DEFAULT_TABLE table=default.Table_bd0dcfadc59f4ffa81d0004ae868ed3e [id=c07b39a3869a4f3781ebccf30c95d5e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.415148 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_0144495ce89d45d2bd9b065864b4f3b7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:32.415364 875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88970b575ba943129729eceef5636519. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:32.415422 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a2e3b6cc2734852af9a0a9b8f51a69b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:32.415951 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0144495ce89d45d2bd9b065864b4f3b7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.418586 876 tablet_service.cc:1505] Processing CreateTablet for tablet 43044ba5660345eaa2b4e6562cc8d448 (DEFAULT_TABLE table=default.Table_54b89a8f30cf414590cfd7a5773bf326 [id=c3d36e0944a74f6eb98b9bb510d8352a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.418792 1667 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:52:32.418988 876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43044ba5660345eaa2b4e6562cc8d448. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.420409 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_ee325d5e9bb44666b4065c86f5d298e1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:32.420948 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ee325d5e9bb44666b4065c86f5d298e1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.421773 883 tablet_service.cc:1505] Processing CreateTablet for tablet cc43ee61e78249bfa00b11f3a1fa21a3 (DEFAULT_TABLE table=default.Table_fde35da450a941b38489c3954c1a3839 [id=25f259d8c2e34d63b7857492d92d8c1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.422137 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc43ee61e78249bfa00b11f3a1fa21a3. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:32.422371 592 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d43210ec61af423ebde624397dea9f2e consensus state has no leader
I20251211 20:52:32.425060 880 tablet_service.cc:1505] Processing CreateTablet for tablet 83067b139f504bc191a515feed0ec771 (DEFAULT_TABLE table=default.Table_8f7e347b9ac64793b1073dfa955ebe21 [id=a6ab64c3e0d94462b716f2d153bc3314]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.425426 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83067b139f504bc191a515feed0ec771. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.426309 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c5819ae1abb48fe8a83af9067726d02. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:32.426817 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 834b84a557b04e78b5d2c8c84520a644 consensus state has no leader
I20251211 20:52:32.427490 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e709051411354522bdb327c34dbf533a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.428951 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48820386e41b47129a13bdd5768f021b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.429037 1165 tablet_replica.cc:333] T 8b89ae724b7f41c98741a41d75592da7 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:32.429260 1165 raft_consensus.cc:2243] T 8b89ae724b7f41c98741a41d75592da7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:32.429414 1165 raft_consensus.cc:2272] T 8b89ae724b7f41c98741a41d75592da7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:32.429966 1165 ts_tablet_manager.cc:1916] T 8b89ae724b7f41c98741a41d75592da7 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:32.431326 884 tablet_service.cc:1505] Processing CreateTablet for tablet 06ba98a726424dd89450053c5c5793f4 (DEFAULT_TABLE table=default.Table_bdb9aa9507d94416bbc3c8fe9cb11fa7 [id=367894d4c46a4cc6bde351a099a635b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.431762 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06ba98a726424dd89450053c5c5793f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.435071 893 tablet_service.cc:1505] Processing CreateTablet for tablet e8db5a11a69e4554bc689c38ab57b28a (DEFAULT_TABLE table=default.Table_4e49f78c60234aea972f8f5068e89b9f [id=6cf77cf74a6f48bcb7113538bfa28ffd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.435446 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8db5a11a69e4554bc689c38ab57b28a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.436475 892 tablet_service.cc:1505] Processing CreateTablet for tablet ab4e7a9062134e489d6f9abe6ff26fb4 (DEFAULT_TABLE table=default.Table_8488ced67fe54a25918bbd9c8c9934e8 [id=b958532650374d609199cab9c07bb64b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.436786 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab4e7a9062134e489d6f9abe6ff26fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.437230 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet b8828385cec34d70b717ae7f3813b9c3 (table default.Table_8352c9da1e324dd4847fd22078f22e53 [id=9dc54153dda4495684c2b07732795bfc]) successfully deleted
I20251211 20:52:32.438462 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_813136dbd41e4469b377741cec73e21d" } new_table_name: "default.Table_181fcc88844b493a8e4ce62302564583" modify_external_catalogs: true
I20251211 20:52:32.443187 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_48dfd045c6744e7db15278a2ebb7f434"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:32.443787 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_48dfd045c6744e7db15278a2ebb7f434 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.445098 1696 raft_consensus.cc:493] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:32.445261 1696 raft_consensus.cc:515] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.445984 1696 leader_election.cc:290] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:32.446311 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9adaee28edd74df68b44b25d505fdfcb" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:32.446485 592 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3dde7ed495e64909bd9784dc5eefb04c consensus state has no leader
W20251211 20:52:32.447717 711 leader_election.cc:343] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:32.449435 1166 tablet_bootstrap.cc:492] T 8139acb0ae324d0f973ce877edc1cd04 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:32.458129 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 88970b575ba943129729eceef5636519 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:32.459611 1024 tablet_service.cc:1505] Processing CreateTablet for tablet 48820386e41b47129a13bdd5768f021b (DEFAULT_TABLE table=default.Table_4b43e2845ad847e39b035931723fe49f [id=6876565d3a9c48ce841b830cf75139ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.459669 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 3c5819ae1abb48fe8a83af9067726d02 (DEFAULT_TABLE table=default.Table_015f1fc2445947f38320ea7699cc7388 [id=34516c0e51a64f3d88a7d5744160367c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.460026 1012 tablet_service.cc:1505] Processing CreateTablet for tablet e709051411354522bdb327c34dbf533a (DEFAULT_TABLE table=default.Table_bd0dcfadc59f4ffa81d0004ae868ed3e [id=c07b39a3869a4f3781ebccf30c95d5e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.461773 1752 raft_consensus.cc:493] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:32.462076 1752 raft_consensus.cc:515] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.463706 1752 leader_election.cc:290] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:32.466912 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c5819ae1abb48fe8a83af9067726d02. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:32.468323 847 leader_election.cc:343] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 4b736a1b347d44b398980654caa21332
I20251211 20:52:32.471277 1165 ts_tablet_manager.cc:1929] T 8b89ae724b7f41c98741a41d75592da7 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:32.471448 1165 log.cc:1199] T 8b89ae724b7f41c98741a41d75592da7 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/8b89ae724b7f41c98741a41d75592da7
I20251211 20:52:32.471833 1165 ts_tablet_manager.cc:1950] T 8b89ae724b7f41c98741a41d75592da7 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:32.472062 889 tablet_service.cc:1505] Processing CreateTablet for tablet dfd194cf00f549c8937cb6ec40987240 (DEFAULT_TABLE table=default.Table_8db9acb9d35149fdabcb9a1d24061b46 [id=d919c6132b3c466e8a1404f981b003b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.473006 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 8b89ae724b7f41c98741a41d75592da7 (table default.Table_c607094f0e4d4af48d2e54d4d728ae13 [id=9dae0e60f978448a840aa497da35966a]) successfully deleted
I20251211 20:52:32.473071 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfd194cf00f549c8937cb6ec40987240. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.474220 1165 tablet_replica.cc:333] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:32.474268 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48820386e41b47129a13bdd5768f021b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.476197 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 5ddf7c1972564ef88e3c0a0bd1b430e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:32.476469 1165 raft_consensus.cc:2243] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:32.476688 1165 raft_consensus.cc:2272] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:32.477232 1165 ts_tablet_manager.cc:1916] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:32.479745 1765 tablet.cc:1722] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:32.482154 1667 fs_manager.cc:714] Time spent opening directory manager: real 0.060s user 0.003s sys 0.007s
I20251211 20:52:32.489729 740 tablet_service.cc:1505] Processing CreateTablet for tablet 80df719919a44aba9b4e3c62d6830cf3 (DEFAULT_TABLE table=default.Table_a9542664049c4b818c73d1958c2f2030 [id=73ce59a7d7144af9b792a9efb1d5622f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.490259 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80df719919a44aba9b4e3c62d6830cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.501852 1012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e709051411354522bdb327c34dbf533a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.507776 1019 tablet_service.cc:1552] Processing DeleteTablet for tablet 6663f13691a24fb991b48f543d9de76b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:32.513900 1028 tablet_service.cc:1552] Processing DeleteTablet for tablet 12aaf618d6474f599458026308d7afdd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:32.513796 741 tablet_service.cc:1505] Processing CreateTablet for tablet 84ed434868aa437a83455b1605deb26c (DEFAULT_TABLE table=default.Table_5a5a43caab0f4ed3a42c60ab127ca9bd [id=6f55593998744f1da2224fd4c60fb61e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.514300 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84ed434868aa437a83455b1605deb26c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.516620 1166 tablet_bootstrap.cc:654] T 8139acb0ae324d0f973ce877edc1cd04 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:32.519817 1028 tablet_service.cc:1505] Processing CreateTablet for tablet 80df719919a44aba9b4e3c62d6830cf3 (DEFAULT_TABLE table=default.Table_a9542664049c4b818c73d1958c2f2030 [id=73ce59a7d7144af9b792a9efb1d5622f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.521639 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80df719919a44aba9b4e3c62d6830cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.522497 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 84ed434868aa437a83455b1605deb26c (DEFAULT_TABLE table=default.Table_5a5a43caab0f4ed3a42c60ab127ca9bd [id=6f55593998744f1da2224fd4c60fb61e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.523005 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84ed434868aa437a83455b1605deb26c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.525422 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 5ddf7c1972564ef88e3c0a0bd1b430e0 (DEFAULT_TABLE table=default.Table_018c974d7bdf4d5abae02a0ca19b05d9 [id=24180c942caa48c889a78b5751867bc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.525905 739 tablet_service.cc:1505] Processing CreateTablet for tablet 7d1cc9809ab843e693a179a35c53ce25 (DEFAULT_TABLE table=default.Table_0e678448f89e48f0ab54479b58c50c84 [id=0db63781ab2d4f70899fc95c5188658d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.526053 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ddf7c1972564ef88e3c0a0bd1b430e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.526289 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d1cc9809ab843e693a179a35c53ce25. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.534416 741 tablet_service.cc:1505] Processing CreateTablet for tablet 5ddf7c1972564ef88e3c0a0bd1b430e0 (DEFAULT_TABLE table=default.Table_018c974d7bdf4d5abae02a0ca19b05d9 [id=24180c942caa48c889a78b5751867bc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.535101 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ddf7c1972564ef88e3c0a0bd1b430e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.542830 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_60c2303340ad4078a0bf2c796be9faf3" } modify_external_catalogs: true
I20251211 20:52:32.543056 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_60c2303340ad4078a0bf2c796be9faf3" } modify_external_catalogs: true
I20251211 20:52:32.550665 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet c8214c97ed1640f5965aa5c4b999e2f1
I20251211 20:52:32.551853 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet c8214c97ed1640f5965aa5c4b999e2f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:32.554085 592 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_0455012740604013898f53488fd5c371 [id=d770b26c251e41e38f9720b37852cb11] alter complete (version 1)
I20251211 20:52:32.556092 745 tablet_service.cc:1552] Processing DeleteTablet for tablet c8214c97ed1640f5965aa5c4b999e2f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:32.556813 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 3f1840e4b5214ddb878935a79b214eae from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:32.556977 1764 tablet.cc:1722] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:32.559330 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_03298b88200d42148accc9f0dffbf739"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:32.559973 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_03298b88200d42148accc9f0dffbf739 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.562666 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 7d1cc9809ab843e693a179a35c53ce25 (DEFAULT_TABLE table=default.Table_0e678448f89e48f0ab54479b58c50c84 [id=0db63781ab2d4f70899fc95c5188658d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.568883 745 tablet_service.cc:1505] Processing CreateTablet for tablet ba120d28abc54553ae15532de90db719 (DEFAULT_TABLE table=default.Table_c496f43b462445bab2d5da82a2f69156 [id=0904c1adf1e145818effef1232fda6c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.569514 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba120d28abc54553ae15532de90db719. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.576618 1165 ts_tablet_manager.cc:1929] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:32.576871 1165 log.cc:1199] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/c910246b3d5d4e7e8a71d9f18e94d24c
I20251211 20:52:32.577313 1165 ts_tablet_manager.cc:1950] T c910246b3d5d4e7e8a71d9f18e94d24c P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:32.578351 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet c910246b3d5d4e7e8a71d9f18e94d24c (table default.Table_c2300bb8bfe44c689969a2b233d69353 [id=996745d9420c4589a52eca6bad10d6a9]) successfully deleted
I20251211 20:52:32.578765 1165 tablet_replica.cc:333] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:32.579023 1165 raft_consensus.cc:2243] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:32.579273 1165 raft_consensus.cc:2272] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:32.583788 1012 tablet_service.cc:1505] Processing CreateTablet for tablet ba120d28abc54553ae15532de90db719 (DEFAULT_TABLE table=default.Table_c496f43b462445bab2d5da82a2f69156 [id=0904c1adf1e145818effef1232fda6c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.585431 592 master_service.cc:954] ReplaceTablet: received request to replace tablet 4f807596fa6a4e66b89566b5d5dd1080 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:32.585677 1012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba120d28abc54553ae15532de90db719. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.587281 740 tablet_service.cc:1505] Processing CreateTablet for tablet 87c4ec07e82d4bf79b4eeefe732674ff (DEFAULT_TABLE table=default.Table_5fbbb8bea9fb4160a6ef7f69d736549d [id=a58ccb217e734813a81f290602e09bef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.587846 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87c4ec07e82d4bf79b4eeefe732674ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.591972 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d1cc9809ab843e693a179a35c53ce25. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.600471 739 tablet_service.cc:1505] Processing CreateTablet for tablet 3dde7ed495e64909bd9784dc5eefb04c (DEFAULT_TABLE table=default.Table_181fcc88844b493a8e4ce62302564583 [id=0030e37866fb4d5bb80ad274605749cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.600992 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dde7ed495e64909bd9784dc5eefb04c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.605082 1164 ts_tablet_manager.cc:1929] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:32.607283 592 catalog_manager.cc:5654] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3). New cstate: current_term: 1 leader_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: HEALTHY } } }
I20251211 20:52:32.607934 592 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet c4e048ca28824329b924845ccf5067cf (table default.Table_f541a176465249d8aaba954bc23bcf01 [id=6b55ad645bec4a7daefc996978420295]). Expected version 4 got 0
I20251211 20:52:32.608747 1164 log.cc:1199] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/27466459d8e6411ab87ac474fb617735
I20251211 20:52:32.609248 1164 ts_tablet_manager.cc:1950] T 27466459d8e6411ab87ac474fb617735 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
W20251211 20:52:32.609344 982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.1:42177, user_credentials={real_user=slave}} blocked reactor thread for 276465us
I20251211 20:52:32.609797 1164 tablet_replica.cc:333] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:32.610028 1164 raft_consensus.cc:2243] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:32.610229 1164 raft_consensus.cc:2272] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:32.610821 1164 ts_tablet_manager.cc:1916] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:32.614017 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 27466459d8e6411ab87ac474fb617735 (table default.Table_265379fa17b74d0f954a2a2c4de113c8 [id=18f8d722faaa4293a112525ff0c96216]) successfully deleted
I20251211 20:52:32.614787 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 87c4ec07e82d4bf79b4eeefe732674ff (DEFAULT_TABLE table=default.Table_5fbbb8bea9fb4160a6ef7f69d736549d [id=a58ccb217e734813a81f290602e09bef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.616580 1165 ts_tablet_manager.cc:1916] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:32.616936 595 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet c4e048ca28824329b924845ccf5067cf (table default.Table_f541a176465249d8aaba954bc23bcf01 [id=6b55ad645bec4a7daefc996978420295]). Expected version 4 got 0
I20251211 20:52:32.622062 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87c4ec07e82d4bf79b4eeefe732674ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.616494 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 3dde7ed495e64909bd9784dc5eefb04c (DEFAULT_TABLE table=default.Table_181fcc88844b493a8e4ce62302564583 [id=0030e37866fb4d5bb80ad274605749cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.628026 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dde7ed495e64909bd9784dc5eefb04c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.639166 1165 ts_tablet_manager.cc:1929] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:32.639446 1165 log.cc:1199] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/19d0aa259ac94f36b7f6a24d11a3d420
I20251211 20:52:32.641083 1165 ts_tablet_manager.cc:1950] T 19d0aa259ac94f36b7f6a24d11a3d420 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:32.642268 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 19d0aa259ac94f36b7f6a24d11a3d420 (table default.Table_ef1c4ca915474d569c91296213346105 [id=9dd03042ff0948b99b39b899255848fe]) successfully deleted
I20251211 20:52:32.643635 880 tablet_service.cc:1505] Processing CreateTablet for tablet 845adb6038fc4c4dbc94418d8ebe3fec (DEFAULT_TABLE table=default.Table_4e2726de74fd437796c8e0e6dd8b3af0 [id=6bca2b1cbc244d34b21efa75f651d35f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.644222 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 845adb6038fc4c4dbc94418d8ebe3fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.646515 1165 tablet_replica.cc:333] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:32.647323 1165 raft_consensus.cc:2243] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:32.647619 1165 raft_consensus.cc:2272] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:32.649003 882 tablet_service.cc:1505] Processing CreateTablet for tablet 4fc56df5ccb94198b5b0e516eeae6941 (DEFAULT_TABLE table=default.Table_494ee7d3e0dc4a47b47082643a3134ce [id=42422ac9805e49cbab71e262e73098b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.649550 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fc56df5ccb94198b5b0e516eeae6941. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.649896 1165 ts_tablet_manager.cc:1916] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:32.658813 1164 ts_tablet_manager.cc:1929] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20251211 20:52:32.666067 710 leader_election.cc:343] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 9adaee28edd74df68b44b25d505fdfcb
I20251211 20:52:32.666241 710 leader_election.cc:304] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:32.666718 1574 raft_consensus.cc:2749] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:32.674926 1165 ts_tablet_manager.cc:1929] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:32.675071 1163 ts_tablet_manager.cc:1929] T d5f0ba3ab16240f9822be69f941f5da2 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:32.675215 1163 log.cc:1199] T d5f0ba3ab16240f9822be69f941f5da2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/d5f0ba3ab16240f9822be69f941f5da2
I20251211 20:52:32.686030 740 tablet_service.cc:1505] Processing CreateTablet for tablet e3f711f02af843adb4f5c3c750924e33 (DEFAULT_TABLE table=default.Table_ee325d5e9bb44666b4065c86f5d298e1 [id=83a61d23e80946439a14096ab3ab1bd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.687953 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3f711f02af843adb4f5c3c750924e33. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.688211 741 tablet_service.cc:1505] Processing CreateTablet for tablet bb0eda1a984840679c5aa87a58523552 (DEFAULT_TABLE table=default.Table_46118198e87e4c158f00055e6867664b [id=6583487fed754a28a5bb4f14cdbef5a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.688580 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb0eda1a984840679c5aa87a58523552. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.687112 745 tablet_service.cc:1505] Processing CreateTablet for tablet f94b656e3c1844509fdae36e33b263e8 (DEFAULT_TABLE table=default.Table_0144495ce89d45d2bd9b065864b4f3b7 [id=e23f4305669a474aafa18928f130de3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.691771 1586 raft_consensus.cc:493] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:32.691962 1586 raft_consensus.cc:515] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.692129 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f94b656e3c1844509fdae36e33b263e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.692929 1586 leader_election.cc:290] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
W20251211 20:52:32.667961 847 leader_election.cc:343] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 4b736a1b347d44b398980654caa21332
I20251211 20:52:32.669099 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 88970b575ba943129729eceef5636519 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:32.669286 876 tablet_service.cc:1552] Processing DeleteTablet for tablet 6663f13691a24fb991b48f543d9de76b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:32.669865 887 tablet_service.cc:1505] Processing CreateTablet for tablet 5ddf7c1972564ef88e3c0a0bd1b430e0 (DEFAULT_TABLE table=default.Table_018c974d7bdf4d5abae02a0ca19b05d9 [id=24180c942caa48c889a78b5751867bc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.670255 886 tablet_service.cc:1505] Processing CreateTablet for tablet 7d1cc9809ab843e693a179a35c53ce25 (DEFAULT_TABLE table=default.Table_0e678448f89e48f0ab54479b58c50c84 [id=0db63781ab2d4f70899fc95c5188658d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.670446 878 tablet_service.cc:1552] Processing DeleteTablet for tablet c8214c97ed1640f5965aa5c4b999e2f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:32.670640 875 tablet_service.cc:1505] Processing CreateTablet for tablet 48820386e41b47129a13bdd5768f021b (DEFAULT_TABLE table=default.Table_4b43e2845ad847e39b035931723fe49f [id=6876565d3a9c48ce841b830cf75139ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.670965 890 tablet_service.cc:1505] Processing CreateTablet for tablet 84ed434868aa437a83455b1605deb26c (DEFAULT_TABLE table=default.Table_5a5a43caab0f4ed3a42c60ab127ca9bd [id=6f55593998744f1da2224fd4c60fb61e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.671156 877 tablet_service.cc:1552] Processing DeleteTablet for tablet 12aaf618d6474f599458026308d7afdd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:32.671437 881 tablet_service.cc:1505] Processing CreateTablet for tablet 80df719919a44aba9b4e3c62d6830cf3 (DEFAULT_TABLE table=default.Table_a9542664049c4b818c73d1958c2f2030 [id=73ce59a7d7144af9b792a9efb1d5622f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.671795 888 tablet_service.cc:1505] Processing CreateTablet for tablet ba0e3a6b2699493ba9b9627b7d944b74 (DEFAULT_TABLE table=default.Table_cce138cd7e6f4a91aeb7632af1b61ade [id=7f521f77d3234e5a9b9612598e00fc94]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.674090 894 tablet_service.cc:1505] Processing CreateTablet for tablet ba120d28abc54553ae15532de90db719 (DEFAULT_TABLE table=default.Table_c496f43b462445bab2d5da82a2f69156 [id=0904c1adf1e145818effef1232fda6c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.695783 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2630ad57845a4c31b20c8a104dd27335" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:32.696255 1029 tablet_service.cc:1505] Processing CreateTablet for tablet bb0eda1a984840679c5aa87a58523552 (DEFAULT_TABLE table=default.Table_46118198e87e4c158f00055e6867664b [id=6583487fed754a28a5bb4f14cdbef5a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.696719 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb0eda1a984840679c5aa87a58523552. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:32.697808 982 leader_election.cc:343] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:32.676826 885 tablet_service.cc:1505] Processing CreateTablet for tablet 87c4ec07e82d4bf79b4eeefe732674ff (DEFAULT_TABLE table=default.Table_5fbbb8bea9fb4160a6ef7f69d736549d [id=a58ccb217e734813a81f290602e09bef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.678269 879 tablet_service.cc:1505] Processing CreateTablet for tablet e709051411354522bdb327c34dbf533a (DEFAULT_TABLE table=default.Table_bd0dcfadc59f4ffa81d0004ae868ed3e [id=c07b39a3869a4f3781ebccf30c95d5e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.703727 1165 log.cc:1199] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/c4b76a7e84d344268567c551236ce772
I20251211 20:52:32.704175 1165 ts_tablet_manager.cc:1950] T c4b76a7e84d344268567c551236ce772 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:32.705695 1773 tablet.cc:1722] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:32.706105 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d1cc9809ab843e693a179a35c53ce25. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.707338 1165 tablet_replica.cc:333] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:32.707540 1165 raft_consensus.cc:2243] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:32.707742 1165 raft_consensus.cc:2272] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:32.707868 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet c4b76a7e84d344268567c551236ce772 (table default.Table_afd3c9ee35ad41caa1717a299443409e [id=b0ebaab8cd5a4deb9a5ea1e7bd3ccec3]) successfully deleted
W20251211 20:52:32.708204 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet f9a3fdf17f274c15ac0bdfa9c174f649 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f9a3fdf17f274c15ac0bdfa9c174f649 already in progress: creating tablet
I20251211 20:52:32.708281 1165 ts_tablet_manager.cc:1916] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:32.708438 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 01104975fd8b470aa026578b97ff6b8d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 01104975fd8b470aa026578b97ff6b8d already in progress: creating tablet
W20251211 20:52:32.708740 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 14e2650f8ec1482ea9511b989fdc963c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14e2650f8ec1482ea9511b989fdc963c already in progress: creating tablet
W20251211 20:52:32.708942 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 3acfef6f082641a199b80eefcfc772ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3acfef6f082641a199b80eefcfc772ac already in progress: creating tablet
I20251211 20:52:32.709460 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ddf7c1972564ef88e3c0a0bd1b430e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.709976 875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48820386e41b47129a13bdd5768f021b. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:32.710100 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 648f43e95e034f2188a20e1b837f2818 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 648f43e95e034f2188a20e1b837f2818 already in progress: creating tablet
I20251211 20:52:32.711278 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80df719919a44aba9b4e3c62d6830cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.712623 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_3bccb9c2f8e04bc3aff742806d2b634a" } modify_external_catalogs: true
I20251211 20:52:32.712838 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_3bccb9c2f8e04bc3aff742806d2b634a" } modify_external_catalogs: true
I20251211 20:52:32.712831 740 tablet_service.cc:1505] Processing CreateTablet for tablet bbe6e6220eb14e59a821805c438230c9 (DEFAULT_TABLE table=default.Table_03298b88200d42148accc9f0dffbf739 [id=b7429ea93123473ab5d795d5f3d5a5f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.713224 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbe6e6220eb14e59a821805c438230c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.713546 847 leader_election.cc:304] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:32.714238 1164 log.cc:1199] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/bd8fd6c977204c90b558827fc0b15f43
I20251211 20:52:32.715353 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f14b182e182d40afa389182bafa08912" } modify_external_catalogs: true
I20251211 20:52:32.715534 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f14b182e182d40afa389182bafa08912" } modify_external_catalogs: true
I20251211 20:52:32.716516 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84ed434868aa437a83455b1605deb26c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.716359 883 tablet_service.cc:1505] Processing CreateTablet for tablet 3dde7ed495e64909bd9784dc5eefb04c (DEFAULT_TABLE table=default.Table_181fcc88844b493a8e4ce62302564583 [id=0030e37866fb4d5bb80ad274605749cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.718080 1586 raft_consensus.cc:493] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:32.718295 1586 raft_consensus.cc:515] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.718664 884 tablet_service.cc:1505] Processing CreateTablet for tablet 3c5819ae1abb48fe8a83af9067726d02 (DEFAULT_TABLE table=default.Table_015f1fc2445947f38320ea7699cc7388 [id=34516c0e51a64f3d88a7d5744160367c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.719019 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 845b5ac4a5984c0cb4ab188925103dc9
I20251211 20:52:32.720077 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 845b5ac4a5984c0cb4ab188925103dc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:32.723426 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 13ceabb190494e718a52892faaa53986
I20251211 20:52:32.724256 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 13ceabb190494e718a52892faaa53986 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:32.724977 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 13ceabb190494e718a52892faaa53986 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:32.726579 1586 leader_election.cc:290] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:32.727207 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f7f52d2547534ef99255c5d3f59410c8" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:32.727353 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c496f43b462445bab2d5da82a2f69156" } modify_external_catalogs: true
I20251211 20:52:32.721868 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2630ad57845a4c31b20c8a104dd27335" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:32.727694 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c496f43b462445bab2d5da82a2f69156" } modify_external_catalogs: true
W20251211 20:52:32.728178 983 leader_election.cc:343] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:32.728307 983 leader_election.cc:304] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:32.728578 1020 tablet_service.cc:1505] Processing CreateTablet for tablet bbe6e6220eb14e59a821805c438230c9 (DEFAULT_TABLE table=default.Table_03298b88200d42148accc9f0dffbf739 [id=b7429ea93123473ab5d795d5f3d5a5f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.729696 1028 tablet_service.cc:1505] Processing CreateTablet for tablet f94b656e3c1844509fdae36e33b263e8 (DEFAULT_TABLE table=default.Table_0144495ce89d45d2bd9b065864b4f3b7 [id=e23f4305669a474aafa18928f130de3a]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:32.728878 982 leader_election.cc:343] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:32.729053 1012 tablet_service.cc:1552] Processing DeleteTablet for tablet 845b5ac4a5984c0cb4ab188925103dc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:32.729805 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbe6e6220eb14e59a821805c438230c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.728781 1586 raft_consensus.cc:2749] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:32.731204 745 tablet_service.cc:1505] Processing CreateTablet for tablet 72af55c8045b4f8f8e116a5782794d81 (DEFAULT_TABLE table=default.Table_5a2e3b6cc2734852af9a0a9b8f51a69b [id=b6c4b72063b3428593d5650a160dc66b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.731552 1752 raft_consensus.cc:2749] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:32.731654 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72af55c8045b4f8f8e116a5782794d81. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.734560 878 tablet_service.cc:1505] Processing CreateTablet for tablet bb0eda1a984840679c5aa87a58523552 (DEFAULT_TABLE table=default.Table_46118198e87e4c158f00055e6867664b [id=6583487fed754a28a5bb4f14cdbef5a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.739632 1013 tablet_service.cc:1505] Processing CreateTablet for tablet 72af55c8045b4f8f8e116a5782794d81 (DEFAULT_TABLE table=default.Table_5a2e3b6cc2734852af9a0a9b8f51a69b [id=b6c4b72063b3428593d5650a160dc66b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.739862 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet ba120d28abc54553ae15532de90db719
I20251211 20:52:32.740434 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72af55c8045b4f8f8e116a5782794d81. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.740979 745 tablet_service.cc:1552] Processing DeleteTablet for tablet ba120d28abc54553ae15532de90db719 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:32.743038 1012 tablet_service.cc:1552] Processing DeleteTablet for tablet ba120d28abc54553ae15532de90db719 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:32.743522 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f94b656e3c1844509fdae36e33b263e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.743788 745 tablet_service.cc:1505] Processing CreateTablet for tablet e53583fc6ae74e5a8293c611291a331b (DEFAULT_TABLE table=default.Table_48dfd045c6744e7db15278a2ebb7f434 [id=901b88b003d04e4783a9c09923910fdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.744266 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e53583fc6ae74e5a8293c611291a331b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.744910 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_868d96d23ca040158aaea84effd252cb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:32.745203 877 tablet_service.cc:1505] Processing CreateTablet for tablet e3f711f02af843adb4f5c3c750924e33 (DEFAULT_TABLE table=default.Table_ee325d5e9bb44666b4065c86f5d298e1 [id=83a61d23e80946439a14096ab3ab1bd9]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:32.745647 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_868d96d23ca040158aaea84effd252cb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.745857 1012 tablet_service.cc:1505] Processing CreateTablet for tablet e53583fc6ae74e5a8293c611291a331b (DEFAULT_TABLE table=default.Table_48dfd045c6744e7db15278a2ebb7f434 [id=901b88b003d04e4783a9c09923910fdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.747094 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_7325cceb834541c895db5f104ae375aa" } modify_external_catalogs: true
I20251211 20:52:32.747340 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_7325cceb834541c895db5f104ae375aa" } modify_external_catalogs: true
W20251211 20:52:32.748448 847 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 51853.6us
I20251211 20:52:32.749666 1160 tablet_bootstrap.cc:492] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:32.750473 740 tablet_service.cc:1505] Processing CreateTablet for tablet 449921ea74f04a9c88eb9e094e8226c8 (DEFAULT_TABLE table=default.Table_b06a07477d3146c288f85dad5e264daa [id=5fdca585035347b183d092e5a3858dc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.750597 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f7f52d2547534ef99255c5d3f59410c8" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:32.750985 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 449921ea74f04a9c88eb9e094e8226c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.754755 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba0e3a6b2699493ba9b9627b7d944b74. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.755731 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_79cef56b856548afb009a80ec288ad1b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:32.756582 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb0eda1a984840679c5aa87a58523552. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:32.756785 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_79cef56b856548afb009a80ec288ad1b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.759384 741 tablet_service.cc:1505] Processing CreateTablet for tablet 59acb8474e114bf7ae7f1f043d611459 (DEFAULT_TABLE table=default.Table_a891ad881584489e87e69014ed59fed9 [id=ed1d0670426945aba888b818ad657225]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.760306 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59acb8474e114bf7ae7f1f043d611459. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.761488 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3f711f02af843adb4f5c3c750924e33. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.763245 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dde7ed495e64909bd9784dc5eefb04c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.768574 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_568264fe4a14485bb77966a76aefdf9a" } new_table_name: "default.Table_2c4e58d88571405eab18aee814981780" modify_external_catalogs: true
I20251211 20:52:32.769434 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e709051411354522bdb327c34dbf533a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.771569 739 tablet_service.cc:1505] Processing CreateTablet for tablet 6ad6ab53aad14cc2b6d201acfecf6723 (DEFAULT_TABLE table=default.Table_505604d692d54bdea110e89bb79743c4 [id=c3c2945eb3d748528f9f69cbe76f978f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.772079 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ad6ab53aad14cc2b6d201acfecf6723. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:32.774560 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ee7b93b0b87541d9af6eb9e68b4cea41 consensus state has no leader
I20251211 20:52:32.775245 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba120d28abc54553ae15532de90db719. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.776682 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet c26c3d0170f64a41ae8944c6668cc5f3
I20251211 20:52:32.777449 891 tablet_service.cc:1505] Processing CreateTablet for tablet f94b656e3c1844509fdae36e33b263e8 (DEFAULT_TABLE table=default.Table_0144495ce89d45d2bd9b065864b4f3b7 [id=e23f4305669a474aafa18928f130de3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.777555 746 tablet_service.cc:1552] Processing DeleteTablet for tablet c26c3d0170f64a41ae8944c6668cc5f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:32.777959 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f94b656e3c1844509fdae36e33b263e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.778901 1164 ts_tablet_manager.cc:1950] T bd8fd6c977204c90b558827fc0b15f43 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:32.779006 594 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet c4e048ca28824329b924845ccf5067cf (table default.Table_f541a176465249d8aaba954bc23bcf01 [id=6b55ad645bec4a7daefc996978420295]). Expected version 4 got 0
I20251211 20:52:32.779373 594 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet dd425601d89148eb95525f796a3abf1e (table default.Table_d4afd6ed615a41ac8f6714c8aac27857 [id=c842792b01cb4c689fcfd3d796151508]). Expected version 1 got 0
I20251211 20:52:32.779496 1164 tablet_replica.cc:333] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:32.779806 1164 raft_consensus.cc:2243] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:32.780050 1164 raft_consensus.cc:2272] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:32.780681 1164 ts_tablet_manager.cc:1916] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:32.781160 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_f5ee1b1b44c645f28737720bc385a577"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:32.781339 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8ba21562eb054a09aa7b9e22a79433f2" } new_table_name: "default.Table_c1f9dca5c2144d50b67681dcc6800e77" modify_external_catalogs: true
I20251211 20:52:32.781378 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_11314b6f726648c3bda96c9c1c7da29e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:32.782217 1160 ts_tablet_manager.cc:1403] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.525s user 0.008s sys 0.006s
W20251211 20:52:32.782310 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f5ee1b1b44c645f28737720bc385a577 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.782943 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_fbf522a147ec4a04950c849072ebb619"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:32.783175 1160 raft_consensus.cc:359] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.783365 1160 raft_consensus.cc:385] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20251211 20:52:32.783390 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fbf522a147ec4a04950c849072ebb619 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.783428 1160 raft_consensus.cc:740] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:32.783465 1768 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:52:32.783758 1160 consensus_queue.cc:260] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.785056 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_78f3b1065a0a43d891bb775169c0eaca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:32.785534 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_78f3b1065a0a43d891bb775169c0eaca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.785666 1160 ts_tablet_manager.cc:1434] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251211 20:52:32.786015 1160 tablet_bootstrap.cc:492] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:32.787235 1165 ts_tablet_manager.cc:1929] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:32.787285 880 tablet_service.cc:1505] Processing CreateTablet for tablet 36d91772201d4c5b928d9dee3f583eb4 (DEFAULT_TABLE table=default.Table_b3b56f469dc54741924e5b55045fa626 [id=471a1295a9534492aff0ae89a5362a1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.787469 1165 log.cc:1199] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/9adaee28edd74df68b44b25d505fdfcb
I20251211 20:52:32.787820 1160 tablet_bootstrap.cc:654] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:32.787881 1165 ts_tablet_manager.cc:1950] T 9adaee28edd74df68b44b25d505fdfcb P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:32.790529 1667 fs_manager.cc:730] Time spent opening block manager: real 0.304s user 0.004s sys 0.003s
I20251211 20:52:32.790817 1667 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/data,/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/wal
uuid: "11f5f94732f049d0be4c97cebfffcb05"
format_stamp: "Formatted at 2025-12-11 20:52:18 on dist-test-slave-fp9b"
I20251211 20:52:32.792156 1667 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:52:32.675545 1163 ts_tablet_manager.cc:1950] T d5f0ba3ab16240f9822be69f941f5da2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:32.804829 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_72a1c37a786747d8910b240c2a831487" } new_table_name: "default.Table_b56de1e85a1b42ea81308593884dadb8" modify_external_catalogs: true
I20251211 20:52:32.805864 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet bd8fd6c977204c90b558827fc0b15f43 (table default.Table_5b8aa62e6fdf46089c4c8ae541856fbe [id=e5caa22bce84422290707816a93bd3f5]) successfully deleted
I20251211 20:52:32.807262 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 9adaee28edd74df68b44b25d505fdfcb (table default.Table_4c974eff9ab442d499f0405abc8a4acd [id=35caba098ab545f7859c004af8568ef7]) successfully deleted
I20251211 20:52:32.808602 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_98f173a3b5454e16ac5680ec2465a09c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:32.810061 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 3621735fcb294e3a8c0180ba9ef86442 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3621735fcb294e3a8c0180ba9ef86442 already in progress: creating tablet
W20251211 20:52:32.810294 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 12aaf618d6474f599458026308d7afdd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12aaf618d6474f599458026308d7afdd already in progress: creating tablet
W20251211 20:52:32.811868 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 88970b575ba943129729eceef5636519 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 88970b575ba943129729eceef5636519 already in progress: creating tablet
I20251211 20:52:32.809774 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_f06fbcca762d4b93b6dd39878d6d58fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:32.811940 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_87e2d779c1a74a57a8eb93d7163c1660" } new_table_name: "default.Table_9aa05c766d874348a9e0eb2c016058d3" modify_external_catalogs: true
W20251211 20:52:32.813755 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f06fbcca762d4b93b6dd39878d6d58fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.813459 745 tablet_service.cc:1505] Processing CreateTablet for tablet d1b3710a104e4480b4913e3df1bb31e0 (DEFAULT_TABLE table=default.Table_4721ef8d42714525ba28461e11e30dec [id=146fd60d3483482c8f46de7c44f3cc90]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:32.815922 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_11314b6f726648c3bda96c9c1c7da29e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:32.819052 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_98f173a3b5454e16ac5680ec2465a09c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:32.825131 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 6663f13691a24fb991b48f543d9de76b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6663f13691a24fb991b48f543d9de76b already in progress: creating tablet
I20251211 20:52:32.825340 1012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e53583fc6ae74e5a8293c611291a331b. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:32.825951 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 5ddf7c1972564ef88e3c0a0bd1b430e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5ddf7c1972564ef88e3c0a0bd1b430e0 already in progress: creating tablet
W20251211 20:52:32.825978 1168 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.645s user 0.158s sys 0.229s
W20251211 20:52:32.826115 1168 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.645s user 0.158s sys 0.229s
W20251211 20:52:32.826198 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet c8214c97ed1640f5965aa5c4b999e2f1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c8214c97ed1640f5965aa5c4b999e2f1 already in progress: creating tablet
W20251211 20:52:32.827510 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 845b5ac4a5984c0cb4ab188925103dc9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 845b5ac4a5984c0cb4ab188925103dc9 already in progress: creating tablet
I20251211 20:52:32.828331 1168 tablet_bootstrap.cc:492] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:32.829258 1168 ts_tablet_manager.cc:1403] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.652s user 0.160s sys 0.230s
I20251211 20:52:32.830111 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_35ba9d3f180646d8b597a9c14b01cf37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:32.830680 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_35ba9d3f180646d8b597a9c14b01cf37 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:32.831387 1168 raft_consensus.cc:359] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.832265 1168 raft_consensus.cc:385] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:32.832495 1168 raft_consensus.cc:740] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
W20251211 20:52:32.832676 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b072b759d1254859b0b8d401e3078792 consensus state has no leader
W20251211 20:52:32.833297 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 13ceabb190494e718a52892faaa53986 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 13ceabb190494e718a52892faaa53986 already in progress: creating tablet
I20251211 20:52:32.833158 1168 consensus_queue.cc:260] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.834939 1168 ts_tablet_manager.cc:1434] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20251211 20:52:32.836848 1168 tablet_bootstrap.cc:492] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:32.837579 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet ba120d28abc54553ae15532de90db719 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba120d28abc54553ae15532de90db719 already in progress: creating tablet
W20251211 20:52:32.837850 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet c26c3d0170f64a41ae8944c6668cc5f3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c26c3d0170f64a41ae8944c6668cc5f3 already in progress: creating tablet
I20251211 20:52:32.841149 1168 tablet_bootstrap.cc:654] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:32.863669 1011 tablet_service.cc:1505] Processing CreateTablet for tablet e3f711f02af843adb4f5c3c750924e33 (DEFAULT_TABLE table=default.Table_ee325d5e9bb44666b4065c86f5d298e1 [id=83a61d23e80946439a14096ab3ab1bd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.864235 1011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3f711f02af843adb4f5c3c750924e33. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.865209 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet d5f0ba3ab16240f9822be69f941f5da2 (table default.Table_d73d156809724a399ad27846bfcc460b [id=baf6ca043ec048b894cf9716fad105b2]) successfully deleted
W20251211 20:52:32.865643 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 01104975fd8b470aa026578b97ff6b8d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 01104975fd8b470aa026578b97ff6b8d already in progress: creating tablet
W20251211 20:52:32.867173 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 14e2650f8ec1482ea9511b989fdc963c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14e2650f8ec1482ea9511b989fdc963c already in progress: creating tablet
W20251211 20:52:32.867437 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 3acfef6f082641a199b80eefcfc772ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3acfef6f082641a199b80eefcfc772ac already in progress: creating tablet
W20251211 20:52:32.867673 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet f9a3fdf17f274c15ac0bdfa9c174f649 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f9a3fdf17f274c15ac0bdfa9c174f649 already in progress: creating tablet
W20251211 20:52:32.867738 983 leader_election.cc:343] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:32.788035 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87c4ec07e82d4bf79b4eeefe732674ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.791676 893 tablet_service.cc:1552] Processing DeleteTablet for tablet 5ddf7c1972564ef88e3c0a0bd1b430e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:32.849056 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 845b5ac4a5984c0cb4ab188925103dc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:32.860599 877 tablet_service.cc:1505] Processing CreateTablet for tablet 72af55c8045b4f8f8e116a5782794d81 (DEFAULT_TABLE table=default.Table_5a2e3b6cc2734852af9a0a9b8f51a69b [id=b6c4b72063b3428593d5650a160dc66b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.865531 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 13ceabb190494e718a52892faaa53986 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:32.868417 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 449921ea74f04a9c88eb9e094e8226c8 (DEFAULT_TABLE table=default.Table_b06a07477d3146c288f85dad5e264daa [id=5fdca585035347b183d092e5a3858dc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.870122 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36d91772201d4c5b928d9dee3f583eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.870491 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 6ad6ab53aad14cc2b6d201acfecf6723 (DEFAULT_TABLE table=default.Table_505604d692d54bdea110e89bb79743c4 [id=c3c2945eb3d748528f9f69cbe76f978f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.871649 983 leader_election.cc:304] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:32.872099 1166 tablet_bootstrap.cc:492] T 8139acb0ae324d0f973ce877edc1cd04 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:32.872247 1166 ts_tablet_manager.cc:1403] T 8139acb0ae324d0f973ce877edc1cd04 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.454s user 0.006s sys 0.003s
I20251211 20:52:32.873492 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1b3710a104e4480b4913e3df1bb31e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.874706 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 59acb8474e114bf7ae7f1f043d611459 (DEFAULT_TABLE table=default.Table_a891ad881584489e87e69014ed59fed9 [id=ed1d0670426945aba888b818ad657225]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.875919 1166 raft_consensus.cc:359] T 8139acb0ae324d0f973ce877edc1cd04 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.876112 1166 raft_consensus.cc:385] T 8139acb0ae324d0f973ce877edc1cd04 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:32.876603 1166 raft_consensus.cc:740] T 8139acb0ae324d0f973ce877edc1cd04 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:32.876868 1166 consensus_queue.cc:260] T 8139acb0ae324d0f973ce877edc1cd04 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.879474 1163 tablet_replica.cc:333] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:32.882354 883 tablet_service.cc:1552] Processing DeleteTablet for tablet ba120d28abc54553ae15532de90db719 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:32.885510 893 tablet_service.cc:1552] Processing DeleteTablet for tablet c26c3d0170f64a41ae8944c6668cc5f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:32.886608 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c5819ae1abb48fe8a83af9067726d02. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:32.886812 983 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 49444.1us
I20251211 20:52:32.887425 1586 raft_consensus.cc:2749] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:32.887820 1024 tablet_service.cc:1505] Processing CreateTablet for tablet d1b3710a104e4480b4913e3df1bb31e0 (DEFAULT_TABLE table=default.Table_4721ef8d42714525ba28461e11e30dec [id=146fd60d3483482c8f46de7c44f3cc90]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.888461 889 tablet_service.cc:1505] Processing CreateTablet for tablet e53583fc6ae74e5a8293c611291a331b (DEFAULT_TABLE table=default.Table_48dfd045c6744e7db15278a2ebb7f434 [id=901b88b003d04e4783a9c09923910fdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.891345 886 tablet_service.cc:1505] Processing CreateTablet for tablet 449921ea74f04a9c88eb9e094e8226c8 (DEFAULT_TABLE table=default.Table_b06a07477d3146c288f85dad5e264daa [id=5fdca585035347b183d092e5a3858dc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.896660 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72af55c8045b4f8f8e116a5782794d81. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.899848 1166 ts_tablet_manager.cc:1434] T 8139acb0ae324d0f973ce877edc1cd04 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251211 20:52:32.900213 1166 tablet_bootstrap.cc:492] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:32.897799 879 tablet_service.cc:1505] Processing CreateTablet for tablet bbe6e6220eb14e59a821805c438230c9 (DEFAULT_TABLE table=default.Table_03298b88200d42148accc9f0dffbf739 [id=b7429ea93123473ab5d795d5f3d5a5f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.906487 883 tablet_service.cc:1505] Processing CreateTablet for tablet 59acb8474e114bf7ae7f1f043d611459 (DEFAULT_TABLE table=default.Table_a891ad881584489e87e69014ed59fed9 [id=ed1d0670426945aba888b818ad657225]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.908453 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet c26c3d0170f64a41ae8944c6668cc5f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:32.908892 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 449921ea74f04a9c88eb9e094e8226c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.909168 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ad6ab53aad14cc2b6d201acfecf6723. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.909412 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1b3710a104e4480b4913e3df1bb31e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.911993 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59acb8474e114bf7ae7f1f043d611459. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.918099 1166 tablet_bootstrap.cc:654] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:32.927192 1163 raft_consensus.cc:2243] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:32.929188 1163 raft_consensus.cc:2272] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:32.929855 1163 ts_tablet_manager.cc:1916] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:32.907902 914 raft_consensus.cc:1275] T c4e048ca28824329b924845ccf5067cf P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Refusing update from remote peer e715a811a82446ce8d5bb4aafd5b81c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251211 20:52:32.899681 881 tablet_service.cc:1505] Processing CreateTablet for tablet d1b3710a104e4480b4913e3df1bb31e0 (DEFAULT_TABLE table=default.Table_4721ef8d42714525ba28461e11e30dec [id=146fd60d3483482c8f46de7c44f3cc90]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.910346 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 449921ea74f04a9c88eb9e094e8226c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.915207 888 tablet_service.cc:1505] Processing CreateTablet for tablet 6ad6ab53aad14cc2b6d201acfecf6723 (DEFAULT_TABLE table=default.Table_505604d692d54bdea110e89bb79743c4 [id=c3c2945eb3d748528f9f69cbe76f978f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:32.936290 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ad6ab53aad14cc2b6d201acfecf6723. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.936596 1586 consensus_queue.cc:1048] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:32.937507 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1b3710a104e4480b4913e3df1bb31e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.949669 1166 tablet_bootstrap.cc:492] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:32.950008 1166 ts_tablet_manager.cc:1403] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.050s user 0.001s sys 0.003s
I20251211 20:52:32.951424 1166 raft_consensus.cc:359] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.957587 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e53583fc6ae74e5a8293c611291a331b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.962875 1166 raft_consensus.cc:385] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:32.963021 1166 raft_consensus.cc:740] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:32.893630 776 raft_consensus.cc:1275] T c4e048ca28824329b924845ccf5067cf P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Refusing update from remote peer e715a811a82446ce8d5bb4aafd5b81c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251211 20:52:32.982144 1164 ts_tablet_manager.cc:1929] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:32.987447 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbe6e6220eb14e59a821805c438230c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:32.987474 1164 log.cc:1199] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/2260850f32b541aab1128818f144b5a7
I20251211 20:52:32.988783 1586 consensus_queue.cc:1048] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20251211 20:52:32.990013 1164 ts_tablet_manager.cc:1950] T 2260850f32b541aab1128818f144b5a7 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:32.990487 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 540) took 1072 ms. Trace:
I20251211 20:52:32.990778 1164 rpcz_store.cc:276] 1211 20:52:31.917741 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:31.917992 (+ 251us) service_pool.cc:225] Handling call
1211 20:52:32.779432 (+861440us) ts_tablet_manager.cc:1151] Deleting tablet 2260850f32b541aab1128818f144b5a7
1211 20:52:32.779439 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:32.897656 (+118217us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:32.981981 (+ 84325us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:32.990431 (+ 8450us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":859468}
W20251211 20:52:32.991348 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4fe5c534b572491c911aae01eaa79bae consensus state has no leader
I20251211 20:52:32.992878 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 2260850f32b541aab1128818f144b5a7 (table default.Table_d8c241666f7840b8bf0cb7672482aa55 [id=81bc0640c53f4710895d487f7c6be071]) successfully deleted
I20251211 20:52:32.989212 1166 consensus_queue.cc:260] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:32.997004 1164 tablet_replica.cc:333] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:32.998167 1166 ts_tablet_manager.cc:1434] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.048s user 0.000s sys 0.002s
I20251211 20:52:33.000257 1166 tablet_bootstrap.cc:492] T ef4fada332cf4b5c9da26ba5f908b4c6 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:33.001992 1160 tablet_bootstrap.cc:492] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:33.003921 1166 tablet_bootstrap.cc:654] T ef4fada332cf4b5c9da26ba5f908b4c6 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.005218 1586 raft_consensus.cc:493] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:33.005995 1586 raft_consensus.cc:515] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.006668 1586 leader_election.cc:290] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:33.007180 1164 raft_consensus.cc:2243] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:33.007819 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "55c40b2464f8444ca6c9fa0400f7f7f2" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:33.013705 1773 tablet.cc:1722] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
W20251211 20:52:33.015117 982 leader_election.cc:343] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:33.015842 1160 ts_tablet_manager.cc:1403] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.217s user 0.004s sys 0.001s
I20251211 20:52:33.017576 594 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet dd425601d89148eb95525f796a3abf1e (table default.Table_d4afd6ed615a41ac8f6714c8aac27857 [id=c842792b01cb4c689fcfd3d796151508]). Expected version 1 got 0
I20251211 20:52:33.017673 1164 raft_consensus.cc:2272] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:33.018299 1164 ts_tablet_manager.cc:1916] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:33.018689 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_5fbbb8bea9fb4160a6ef7f69d736549d" } new_table_name: "default.Table_a06aeab492cf4bd59dbc1274c6c6827d" modify_external_catalogs: true
I20251211 20:52:33.019752 1764 tablet.cc:1722] T c4e048ca28824329b924845ccf5067cf P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20251211 20:52:33.021391 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59acb8474e114bf7ae7f1f043d611459. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.023814 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 0246f1df99684d0da43d97acbf1182a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:33.026286 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_5a5a43caab0f4ed3a42c60ab127ca9bd" } modify_external_catalogs: true
I20251211 20:52:33.026484 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_5a5a43caab0f4ed3a42c60ab127ca9bd" } modify_external_catalogs: true
I20251211 20:52:33.035514 589 master_service.cc:954] ReplaceTablet: received request to replace tablet 22273287aa9245a2a66131fed2c8861c from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:33.036744 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_6829fee154a74021a1410d7535c0c89f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:33.037089 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5bd801124df84a6c96f4912993f1bdfd consensus state has no leader
I20251211 20:52:33.037388 596 master_service.cc:954] ReplaceTablet: received request to replace tablet bc32a778e5424850b23ccaf74bb9df07 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:33.038013 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_0040a1e388eb4e58a352a1a3884345c4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:33.038184 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ccd52fe248db4491820429e77b22f465" } modify_external_catalogs: true
I20251211 20:52:33.038365 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ccd52fe248db4491820429e77b22f465" } modify_external_catalogs: true
W20251211 20:52:33.038491 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0040a1e388eb4e58a352a1a3884345c4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:33.037395 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6829fee154a74021a1410d7535c0c89f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:33.043090 1168 tablet_bootstrap.cc:492] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:33.043329 1168 ts_tablet_manager.cc:1403] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.207s user 0.042s sys 0.110s
I20251211 20:52:33.044205 1168 raft_consensus.cc:359] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.044497 1168 raft_consensus.cc:385] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.044636 1168 raft_consensus.cc:740] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.044906 1168 consensus_queue.cc:260] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.045094 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "55c40b2464f8444ca6c9fa0400f7f7f2" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:33.046422 1168 ts_tablet_manager.cc:1434] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:33.047282 1168 tablet_bootstrap.cc:492] T ee893e33255745debf73c486d8c70267 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:33.049963 1168 tablet_bootstrap.cc:654] T ee893e33255745debf73c486d8c70267 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.058131 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0e1298b5dbe74fdebb9368056208f205" } modify_external_catalogs: true
I20251211 20:52:33.058460 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b70392ad5fb6451482100395256494bd" } modify_external_catalogs: true
I20251211 20:52:33.058689 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0e1298b5dbe74fdebb9368056208f205" } modify_external_catalogs: true
I20251211 20:52:33.059329 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b70392ad5fb6451482100395256494bd" } modify_external_catalogs: true
I20251211 20:52:33.060535 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c14c424a6284406d968649c9991c7a92" } modify_external_catalogs: true
I20251211 20:52:33.060684 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c14c424a6284406d968649c9991c7a92" } modify_external_catalogs: true
I20251211 20:52:33.061094 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_bc06d56f49324a42b6e5a68cb9ef1434"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:33.061462 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 87c4ec07e82d4bf79b4eeefe732674ff consensus state has no leader
W20251211 20:52:33.061657 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc06d56f49324a42b6e5a68cb9ef1434 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:33.061866 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_31cc05174eeb48948593096a3613f70c" } new_table_name: "default.Table_e02933dd91ce4fe7a40d1c3b2f9b2d4b" modify_external_catalogs: true
I20251211 20:52:33.065071 1667 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:52:33.065229 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 8139acb0ae324d0f973ce877edc1cd04 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:33.066792 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ae7f102bdee3470990e863235f08d636" } new_table_name: "default.Table_c817dec9a7674f699b3366fc302ead57" modify_external_catalogs: true
I20251211 20:52:33.067775 1667 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:52:33.068097 1667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:52:33.071852 590 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 2630ad57845a4c31b20c8a104dd27335 (table default.Table_4d70f35e05524680b3d72b2e488adbd5 [id=61b2de16fee74487823292913de1b685]). Expected version 3 got 0
I20251211 20:52:33.074570 1011 tablet_service.cc:1505] Processing CreateTablet for tablet d0673c37c2fd47eea8ab10d9e91b8d7a (DEFAULT_TABLE table=default.Table_79cef56b856548afb009a80ec288ad1b [id=a4bb425aab7d4a9183630f3db61a6310]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.075501 745 tablet_service.cc:1505] Processing CreateTablet for tablet 5a42998dbedc465f9e39e0aa45285368 (DEFAULT_TABLE table=default.Table_868d96d23ca040158aaea84effd252cb [id=1aaf20327cd24cc682293c44cb5295f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.060056 1160 raft_consensus.cc:359] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.075953 1160 raft_consensus.cc:385] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.075911 741 tablet_service.cc:1505] Processing CreateTablet for tablet d0673c37c2fd47eea8ab10d9e91b8d7a (DEFAULT_TABLE table=default.Table_79cef56b856548afb009a80ec288ad1b [id=a4bb425aab7d4a9183630f3db61a6310]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.078727 590 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 2630ad57845a4c31b20c8a104dd27335 (table default.Table_4d70f35e05524680b3d72b2e488adbd5 [id=61b2de16fee74487823292913de1b685]). Expected version 3 got 0
I20251211 20:52:33.078948 590 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 3eac21a07ba34e1fa275e25ee9dcabb1 (table default.Table_d571120dedcb48cd9d4b75314be3a654 [id=9700f9fe6e3044748e34954bef8a0038]). Expected version 2 got 0
I20251211 20:52:33.079150 590 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet c4e048ca28824329b924845ccf5067cf (table default.Table_f541a176465249d8aaba954bc23bcf01 [id=6b55ad645bec4a7daefc996978420295]). Expected version 4 got 0
I20251211 20:52:33.082420 1013 tablet_service.cc:1505] Processing CreateTablet for tablet 5a42998dbedc465f9e39e0aa45285368 (DEFAULT_TABLE table=default.Table_868d96d23ca040158aaea84effd252cb [id=1aaf20327cd24cc682293c44cb5295f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.082429 885 tablet_service.cc:1505] Processing CreateTablet for tablet d0673c37c2fd47eea8ab10d9e91b8d7a (DEFAULT_TABLE table=default.Table_79cef56b856548afb009a80ec288ad1b [id=a4bb425aab7d4a9183630f3db61a6310]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.082895 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0673c37c2fd47eea8ab10d9e91b8d7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.087627 1160 raft_consensus.cc:740] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.088009 1160 consensus_queue.cc:260] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.089951 1160 ts_tablet_manager.cc:1434] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.073s user 0.001s sys 0.001s
I20251211 20:52:33.090310 1160 tablet_bootstrap.cc:492] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:33.091960 891 tablet_service.cc:1505] Processing CreateTablet for tablet 5a42998dbedc465f9e39e0aa45285368 (DEFAULT_TABLE table=default.Table_868d96d23ca040158aaea84effd252cb [id=1aaf20327cd24cc682293c44cb5295f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.108050 1160 tablet_bootstrap.cc:654] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.112210 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a42998dbedc465f9e39e0aa45285368. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:33.114821 983 leader_election.cc:343] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:33.118916 1011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0673c37c2fd47eea8ab10d9e91b8d7a. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:33.121937 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e4647105ab57454d883ebfcde7bde891 consensus state has no leader
I20251211 20:52:33.122140 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 84ed434868aa437a83455b1605deb26c
I20251211 20:52:33.122289 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet bd2bf1113dfe4b67b263b56f3d3db07d
W20251211 20:52:33.122953 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 645b08b33676420c96e880de3d460829 consensus state has no leader
I20251211 20:52:33.123392 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a42998dbedc465f9e39e0aa45285368. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.123865 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet c112714331bf4bd7a1ddcca17f338d4f
I20251211 20:52:33.124122 740 tablet_service.cc:1552] Processing DeleteTablet for tablet bd2bf1113dfe4b67b263b56f3d3db07d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:33.124838 739 tablet_service.cc:1552] Processing DeleteTablet for tablet 84ed434868aa437a83455b1605deb26c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:33.119987 983 leader_election.cc:304] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:33.125213 983 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 78938.3us
I20251211 20:52:33.126555 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0673c37c2fd47eea8ab10d9e91b8d7a. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:33.128881 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet bd2bf1113dfe4b67b263b56f3d3db07d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd2bf1113dfe4b67b263b56f3d3db07d already in progress: creating tablet
W20251211 20:52:33.129696 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 84ed434868aa437a83455b1605deb26c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84ed434868aa437a83455b1605deb26c already in progress: creating tablet
I20251211 20:52:33.134512 881 tablet_service.cc:1552] Processing DeleteTablet for tablet bd2bf1113dfe4b67b263b56f3d3db07d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:33.135272 886 tablet_service.cc:1552] Processing DeleteTablet for tablet c112714331bf4bd7a1ddcca17f338d4f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:33.136446 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 84ed434868aa437a83455b1605deb26c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:33.138263 1586 raft_consensus.cc:2749] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:33.141582 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a42998dbedc465f9e39e0aa45285368. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.143345 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet 55c40b2464f8444ca6c9fa0400f7f7f2 with delete_type TABLET_DATA_DELETED (replaced by tablet fb030a62796248869d46b682656eaeb0) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:33.143568 1012 tablet_service.cc:1552] Processing DeleteTablet for tablet c112714331bf4bd7a1ddcca17f338d4f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:33.144177 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet bd2bf1113dfe4b67b263b56f3d3db07d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:33.144343 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 84ed434868aa437a83455b1605deb26c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:33.145145 739 tablet_service.cc:1552] Processing DeleteTablet for tablet c112714331bf4bd7a1ddcca17f338d4f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:33.146744 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet c112714331bf4bd7a1ddcca17f338d4f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c112714331bf4bd7a1ddcca17f338d4f already in progress: creating tablet
I20251211 20:52:33.148859 1163 ts_tablet_manager.cc:1929] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:33.149226 1163 log.cc:1199] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/f7f52d2547534ef99255c5d3f59410c8
I20251211 20:52:33.166509 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 84b2a67c836844ceb3628cc3b7d8ba10
I20251211 20:52:33.172549 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 84b2a67c836844ceb3628cc3b7d8ba10 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:33.177171 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet a9b2aa21ab124b28aebc33adba0583d2
I20251211 20:52:33.177217 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 84b2a67c836844ceb3628cc3b7d8ba10 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:33.181077 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet a9b2aa21ab124b28aebc33adba0583d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:33.185699 888 tablet_service.cc:1552] Processing DeleteTablet for tablet a9b2aa21ab124b28aebc33adba0583d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:33.186761 873 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:43876 (request call id 115) took 1057 ms. Trace:
I20251211 20:52:33.186941 873 rpcz_store.cc:276] 1211 20:52:32.129267 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:32.129336 (+ 69us) service_pool.cc:225] Handling call
1211 20:52:32.254101 (+124765us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x6160000d6130. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9006227df9 00007f9008d4f938 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:32.297804 (+ 43703us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x6160000d2154. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:32.715277 (+417473us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x616000023e54. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:33.186692 (+471415us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":861174528}
I20251211 20:52:33.187497 1163 ts_tablet_manager.cc:1950] T f7f52d2547534ef99255c5d3f59410c8 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:33.188699 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 566) took 1120 ms. Trace:
I20251211 20:52:33.189148 1163 rpcz_store.cc:276] 1211 20:52:32.068210 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:32.068377 (+ 167us) service_pool.cc:225] Handling call
1211 20:52:32.867955 (+799578us) ts_tablet_manager.cc:1151] Deleting tablet f7f52d2547534ef99255c5d3f59410c8
1211 20:52:32.867966 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:32.927508 (+ 59542us) spinlock_profiling.cc:243] Waited 43.1 ms on lock 0x616000127f54. stack: 00007f39ea6b1ee6 00007f39ea6b1ca0 00007f39e968d3ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f39f657c1e0 00007f39f78ca0b3 00007f39fa48a757 00007f39fa4bd744 00007f39fa49e240 00007f39fa49dfcd 0000000000a8970c 00007f39ea6f820f 00007f39ea6fb52b 00007f39ea6fb3a8
1211 20:52:33.139832 (+212324us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:33.148155 (+ 8323us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:33.188639 (+ 40484us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":99175168,"tablet-delete.queue_time_us":799357}
I20251211 20:52:33.190834 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet f7f52d2547534ef99255c5d3f59410c8 (table default.Table_b3e8496af4d340218ce858c81796a29c [id=ecb6d90c41224c2d8f771a78922eaf05]) successfully deleted
I20251211 20:52:33.191424 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 565) took 1125 ms. Trace:
I20251211 20:52:33.195407 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_63cd6bb6292e4535afd64845a12e0149" } new_table_name: "default.Table_742d698db63f432aa7b88ceccf6dc0fc" modify_external_catalogs: true
I20251211 20:52:33.196025 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_68f5e21962ce48b89834785bf8dd332f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:33.196574 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_68f5e21962ce48b89834785bf8dd332f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:33.196892 740 tablet_service.cc:1552] Processing DeleteTablet for tablet a9b2aa21ab124b28aebc33adba0583d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:33.197352 739 tablet_service.cc:1552] Processing DeleteTablet for tablet 84b2a67c836844ceb3628cc3b7d8ba10 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:33.198773 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet a9b2aa21ab124b28aebc33adba0583d2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a9b2aa21ab124b28aebc33adba0583d2 already in progress: creating tablet
W20251211 20:52:33.199080 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 84b2a67c836844ceb3628cc3b7d8ba10 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84b2a67c836844ceb3628cc3b7d8ba10 already in progress: creating tablet
I20251211 20:52:33.201592 594 master_service.cc:954] ReplaceTablet: received request to replace tablet 062f980b65504d3baf99ae400e521775 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:33.203414 596 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_f541a176465249d8aaba954bc23bcf01 [id=6b55ad645bec4a7daefc996978420295] alter complete (version 4)
I20251211 20:52:33.212072 1164 ts_tablet_manager.cc:1929] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:33.212801 885 tablet_service.cc:1552] Processing DeleteTablet for tablet c38c1a4c0e574eb0957de127d41c26ad with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:33.213122 1164 log.cc:1199] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/78922f26cedb4bb989aa1c1cb40bfa3a
I20251211 20:52:33.214576 1164 ts_tablet_manager.cc:1950] T 78922f26cedb4bb989aa1c1cb40bfa3a P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:33.214987 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 547) took 1266 ms. Trace:
I20251211 20:52:33.215924 1164 rpcz_store.cc:276] 1211 20:52:31.948499 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:31.948550 (+ 51us) service_pool.cc:225] Handling call
1211 20:52:32.991652 (+1043102us) ts_tablet_manager.cc:1151] Deleting tablet 78922f26cedb4bb989aa1c1cb40bfa3a
1211 20:52:32.991689 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:33.098913 (+107224us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:33.195922 (+ 97009us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:33.214947 (+ 19025us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":892030}
I20251211 20:52:33.224113 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 78922f26cedb4bb989aa1c1cb40bfa3a (table default.Table_736d6c2eb4dd4eb9a72dca6b9ed2073f [id=289e09e5a445461a9d0949c41ddcbd18]) successfully deleted
I20251211 20:52:33.217455 1160 tablet_bootstrap.cc:492] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:33.228076 1164 tablet_replica.cc:333] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:33.228964 1160 ts_tablet_manager.cc:1403] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.135s user 0.002s sys 0.004s
I20251211 20:52:33.229880 1160 raft_consensus.cc:359] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.231747 1166 tablet_bootstrap.cc:492] T ef4fada332cf4b5c9da26ba5f908b4c6 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:33.231977 1166 ts_tablet_manager.cc:1403] T ef4fada332cf4b5c9da26ba5f908b4c6 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.232s user 0.011s sys 0.015s
I20251211 20:52:33.232913 1166 raft_consensus.cc:359] T ef4fada332cf4b5c9da26ba5f908b4c6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.233094 1166 raft_consensus.cc:385] T ef4fada332cf4b5c9da26ba5f908b4c6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.233177 1166 raft_consensus.cc:740] T ef4fada332cf4b5c9da26ba5f908b4c6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.233410 1166 consensus_queue.cc:260] T ef4fada332cf4b5c9da26ba5f908b4c6 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.233937 1166 ts_tablet_manager.cc:1434] T ef4fada332cf4b5c9da26ba5f908b4c6 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:33.234278 1166 tablet_bootstrap.cc:492] T bc32a778e5424850b23ccaf74bb9df07 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:33.237298 1166 tablet_bootstrap.cc:654] T bc32a778e5424850b23ccaf74bb9df07 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.237726 1164 raft_consensus.cc:2243] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:33.240604 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_a076c34f6d2c4e38b6a1078d322e151c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:33.241230 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a076c34f6d2c4e38b6a1078d322e151c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:33.243110 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b7695e75618444e5a8e1ae1e2fd02d5b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:33.243659 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b7695e75618444e5a8e1ae1e2fd02d5b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:33.231721 1163 rpcz_store.cc:276] 1211 20:52:32.066412 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:32.150992 (+ 84580us) service_pool.cc:225] Handling call
1211 20:52:33.191230 (+1040238us) ts_tablet_manager.cc:1151] Deleting tablet 648f43e95e034f2188a20e1b837f2818
1211 20:52:33.191246 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:33.191384 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1039980}
I20251211 20:52:33.244757 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_25aba30fb89c47c5b77403382971cf65"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:33.245169 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 648f43e95e034f2188a20e1b837f2818 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 648f43e95e034f2188a20e1b837f2818 already in progress: creating tablet
W20251211 20:52:33.245255 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_25aba30fb89c47c5b77403382971cf65 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:33.245471 588 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0144495ce89d45d2bd9b065864b4f3b7" } new_table_name: "default.Table_64fa73fbf85e4a40834ce3d6d7d84b11" modify_external_catalogs: true
I20251211 20:52:33.245631 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_9fb97b3359de4806bf0557739a5da4fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:33.246119 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9fb97b3359de4806bf0557739a5da4fd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:33.247109 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_c993b3973aa64eabb16fe02f9514b161"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:33.247534 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 577) took 1023 ms. Trace:
I20251211 20:52:33.247679 1163 rpcz_store.cc:276] 1211 20:52:32.223715 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:32.224202 (+ 487us) service_pool.cc:225] Handling call
1211 20:52:33.245282 (+1021080us) ts_tablet_manager.cc:1151] Deleting tablet 3621735fcb294e3a8c0180ba9ef86442
1211 20:52:33.245295 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:33.247491 (+ 2196us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":949918}
W20251211 20:52:33.248411 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 3621735fcb294e3a8c0180ba9ef86442 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3621735fcb294e3a8c0180ba9ef86442 already in progress: creating tablet
W20251211 20:52:33.248612 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c993b3973aa64eabb16fe02f9514b161 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:33.250108 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 88970b575ba943129729eceef5636519 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 88970b575ba943129729eceef5636519 already in progress: creating tablet
W20251211 20:52:33.250864 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 5ddf7c1972564ef88e3c0a0bd1b430e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5ddf7c1972564ef88e3c0a0bd1b430e0 already in progress: creating tablet
W20251211 20:52:33.251884 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 6663f13691a24fb991b48f543d9de76b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6663f13691a24fb991b48f543d9de76b already in progress: creating tablet
W20251211 20:52:33.252954 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 12aaf618d6474f599458026308d7afdd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12aaf618d6474f599458026308d7afdd already in progress: creating tablet
I20251211 20:52:33.253657 1164 raft_consensus.cc:2272] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:33.253909 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet c8214c97ed1640f5965aa5c4b999e2f1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c8214c97ed1640f5965aa5c4b999e2f1 already in progress: creating tablet
W20251211 20:52:33.254910 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 13ceabb190494e718a52892faaa53986 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 13ceabb190494e718a52892faaa53986 already in progress: creating tablet
W20251211 20:52:33.255889 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 845b5ac4a5984c0cb4ab188925103dc9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 845b5ac4a5984c0cb4ab188925103dc9 already in progress: creating tablet
W20251211 20:52:33.257119 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet ba120d28abc54553ae15532de90db719 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba120d28abc54553ae15532de90db719 already in progress: creating tablet
W20251211 20:52:33.257650 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet c26c3d0170f64a41ae8944c6668cc5f3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c26c3d0170f64a41ae8944c6668cc5f3 already in progress: creating tablet
I20251211 20:52:33.258178 1163 tablet_replica.cc:333] T 8139acb0ae324d0f973ce877edc1cd04 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:33.258227 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_e345037da95a4c16b6e8b650f0ba3911"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:33.259145 1163 raft_consensus.cc:2243] T 8139acb0ae324d0f973ce877edc1cd04 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:33.259346 1163 raft_consensus.cc:2272] T 8139acb0ae324d0f973ce877edc1cd04 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:33.259306 1160 raft_consensus.cc:385] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.258893 1166 tablet_bootstrap.cc:492] T bc32a778e5424850b23ccaf74bb9df07 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:33.259647 1166 ts_tablet_manager.cc:1403] T bc32a778e5424850b23ccaf74bb9df07 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.007s
I20251211 20:52:33.260134 1164 ts_tablet_manager.cc:1916] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:33.260481 1166 raft_consensus.cc:359] T bc32a778e5424850b23ccaf74bb9df07 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.260710 1166 raft_consensus.cc:385] T bc32a778e5424850b23ccaf74bb9df07 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.260821 1166 raft_consensus.cc:740] T bc32a778e5424850b23ccaf74bb9df07 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.261104 1166 consensus_queue.cc:260] T bc32a778e5424850b23ccaf74bb9df07 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.265096 1166 ts_tablet_manager.cc:1434] T bc32a778e5424850b23ccaf74bb9df07 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251211 20:52:33.265237 1160 raft_consensus.cc:740] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.265653 1160 consensus_queue.cc:260] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.266206 1160 ts_tablet_manager.cc:1434] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.037s user 0.001s sys 0.001s
I20251211 20:52:33.269039 1160 tablet_bootstrap.cc:492] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:33.273659 1758 raft_consensus.cc:493] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:33.273809 1758 raft_consensus.cc:515] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.274477 1758 leader_election.cc:290] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:33.275926 1166 tablet_bootstrap.cc:492] T 05385f20e88c46108e09a78814811209 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:33.276137 847 leader_election.cc:343] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: c38c1a4c0e574eb0957de127d41c26ad
W20251211 20:52:33.276346 847 leader_election.cc:343] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: c38c1a4c0e574eb0957de127d41c26ad
I20251211 20:52:33.276434 847 leader_election.cc:304] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:33.276796 1758 raft_consensus.cc:2749] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:33.277923 1166 tablet_bootstrap.cc:654] T 05385f20e88c46108e09a78814811209 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.279469 1163 ts_tablet_manager.cc:1916] T 8139acb0ae324d0f973ce877edc1cd04 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:33.282622 891 tablet_service.cc:1552] Processing DeleteTablet for tablet dc40b9826ea84728910b1fd61ce62a25 with delete_type TABLET_DATA_DELETED (replaced by tablet bc32a778e5424850b23ccaf74bb9df07) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:33.288946 1011 tablet_service.cc:1552] Processing DeleteTablet for tablet bc32a778e5424850b23ccaf74bb9df07 with delete_type TABLET_DATA_DELETED (replaced by tablet 6a206abd6ea2420089cb07714e0d19c0) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:33.292631 1163 ts_tablet_manager.cc:1929] T 8139acb0ae324d0f973ce877edc1cd04 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:33.293252 1160 tablet_bootstrap.cc:654] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.299666 1163 log.cc:1199] T 8139acb0ae324d0f973ce877edc1cd04 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/8139acb0ae324d0f973ce877edc1cd04
W20251211 20:52:33.297268 591 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b529b38d754f46ba8cb60b19e9705256 consensus state has no leader
I20251211 20:52:33.301249 1163 ts_tablet_manager.cc:1950] T 8139acb0ae324d0f973ce877edc1cd04 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
W20251211 20:52:33.301416 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e345037da95a4c16b6e8b650f0ba3911 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:33.303036 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 8139acb0ae324d0f973ce877edc1cd04 (table default.Table_d282d0b8f49e4f44998af55480ad3cb8 [id=959101c0af144f52a5680b0c9d4cbd20]) successfully deleted
I20251211 20:52:33.303277 1792 tablet.cc:1722] T c4e048ca28824329b924845ccf5067cf P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20251211 20:52:33.303366 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_505604d692d54bdea110e89bb79743c4" } new_table_name: "default.Table_0605520db16d4320bafd3ffe2e5395f0" modify_external_catalogs: true
I20251211 20:52:33.305254 1163 tablet_replica.cc:333] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:33.305987 1163 raft_consensus.cc:2243] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:33.306703 1163 raft_consensus.cc:2272] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:33.308722 1163 ts_tablet_manager.cc:1916] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:33.313341 1168 tablet_bootstrap.cc:492] T ee893e33255745debf73c486d8c70267 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:33.313542 1168 ts_tablet_manager.cc:1403] T ee893e33255745debf73c486d8c70267 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.267s user 0.103s sys 0.092s
I20251211 20:52:33.314415 1168 raft_consensus.cc:359] T ee893e33255745debf73c486d8c70267 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.314638 1168 raft_consensus.cc:385] T ee893e33255745debf73c486d8c70267 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.314731 1168 raft_consensus.cc:740] T ee893e33255745debf73c486d8c70267 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.314958 1758 raft_consensus.cc:493] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:33.314968 1168 consensus_queue.cc:260] T ee893e33255745debf73c486d8c70267 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.315133 1758 raft_consensus.cc:515] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.315438 1168 ts_tablet_manager.cc:1434] T ee893e33255745debf73c486d8c70267 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:33.315795 1168 tablet_bootstrap.cc:492] T 27a08a64d92342d08ab4406e8006a97b P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:33.315817 1758 leader_election.cc:290] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:33.317507 1168 tablet_bootstrap.cc:654] T 27a08a64d92342d08ab4406e8006a97b P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:33.318593 847 leader_election.cc:343] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 0246f1df99684d0da43d97acbf1182a0
W20251211 20:52:33.325033 847 leader_election.cc:343] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 0246f1df99684d0da43d97acbf1182a0
I20251211 20:52:33.325222 847 leader_election.cc:304] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:33.325719 1168 tablet_bootstrap.cc:492] T 27a08a64d92342d08ab4406e8006a97b P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:33.325760 1758 raft_consensus.cc:2749] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:33.325903 1168 ts_tablet_manager.cc:1403] T 27a08a64d92342d08ab4406e8006a97b P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20251211 20:52:33.326673 1168 raft_consensus.cc:359] T 27a08a64d92342d08ab4406e8006a97b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.326881 1168 raft_consensus.cc:385] T 27a08a64d92342d08ab4406e8006a97b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.327071 1168 raft_consensus.cc:740] T 27a08a64d92342d08ab4406e8006a97b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.327299 1168 consensus_queue.cc:260] T 27a08a64d92342d08ab4406e8006a97b P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.327873 1168 ts_tablet_manager.cc:1434] T 27a08a64d92342d08ab4406e8006a97b P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:33.328176 1168 tablet_bootstrap.cc:492] T ae1abef61473443b9922eb13f2ae92fa P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:33.328642 592 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet c4e048ca28824329b924845ccf5067cf (table default.Table_f541a176465249d8aaba954bc23bcf01 [id=6b55ad645bec4a7daefc996978420295]). Expected version 4 got 0
I20251211 20:52:33.329576 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_7287bc8f28a64682b352f9adae2ec224"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:33.330149 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7287bc8f28a64682b352f9adae2ec224 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:33.330452 1168 tablet_bootstrap.cc:654] T ae1abef61473443b9922eb13f2ae92fa P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.332551 1163 ts_tablet_manager.cc:1929] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:33.332677 1163 log.cc:1199] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/55c40b2464f8444ca6c9fa0400f7f7f2
I20251211 20:52:33.334940 1163 ts_tablet_manager.cc:1950] T 55c40b2464f8444ca6c9fa0400f7f7f2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:33.336337 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 55c40b2464f8444ca6c9fa0400f7f7f2 (table default.Table_069bb219e15e43be995ff011e2289a84 [id=bc0b945e86cf43fdb99d56658ca1576c]) successfully deleted
W20251211 20:52:33.338940 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet c112714331bf4bd7a1ddcca17f338d4f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c112714331bf4bd7a1ddcca17f338d4f already in progress: creating tablet
I20251211 20:52:33.340095 1166 tablet_bootstrap.cc:492] T 05385f20e88c46108e09a78814811209 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
W20251211 20:52:33.340669 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet bd2bf1113dfe4b67b263b56f3d3db07d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd2bf1113dfe4b67b263b56f3d3db07d already in progress: creating tablet
W20251211 20:52:33.341586 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 84ed434868aa437a83455b1605deb26c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84ed434868aa437a83455b1605deb26c already in progress: creating tablet
I20251211 20:52:33.341835 1160 tablet_bootstrap.cc:492] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:33.342000 1160 ts_tablet_manager.cc:1403] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.073s user 0.001s sys 0.002s
I20251211 20:52:33.342206 1166 ts_tablet_manager.cc:1403] T 05385f20e88c46108e09a78814811209 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.066s user 0.010s sys 0.010s
W20251211 20:52:33.342694 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 84b2a67c836844ceb3628cc3b7d8ba10 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84b2a67c836844ceb3628cc3b7d8ba10 already in progress: creating tablet
I20251211 20:52:33.342826 1160 raft_consensus.cc:359] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.343019 1160 raft_consensus.cc:385] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20251211 20:52:33.344825 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet a9b2aa21ab124b28aebc33adba0583d2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a9b2aa21ab124b28aebc33adba0583d2 already in progress: creating tablet
I20251211 20:52:33.345255 1163 tablet_replica.cc:333] T bc32a778e5424850b23ccaf74bb9df07 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:33.345701 1163 raft_consensus.cc:2243] T bc32a778e5424850b23ccaf74bb9df07 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:33.346181 1163 raft_consensus.cc:2272] T bc32a778e5424850b23ccaf74bb9df07 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:33.347479 1163 ts_tablet_manager.cc:1916] T bc32a778e5424850b23ccaf74bb9df07 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:33.350677 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_3fe66404826142609775fed9970bbb7a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:33.351125 745 tablet_service.cc:1552] Processing DeleteTablet for tablet ee893e33255745debf73c486d8c70267 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:33.351257 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3fe66404826142609775fed9970bbb7a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:33.351452 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 27a08a64d92342d08ab4406e8006a97b with delete_type TABLET_DATA_DELETED (replaced by tablet bfb52935296443c58190d4d06e2e6e9c) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:33.351722 1165 tablet_replica.cc:333] T ee893e33255745debf73c486d8c70267 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:33.351922 1165 raft_consensus.cc:2243] T ee893e33255745debf73c486d8c70267 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:33.352116 1165 raft_consensus.cc:2272] T ee893e33255745debf73c486d8c70267 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:33.353397 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_1ccc66e4393445059e9653a319360528"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:33.353639 1165 ts_tablet_manager.cc:1916] T ee893e33255745debf73c486d8c70267 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:33.354084 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1ccc66e4393445059e9653a319360528 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:33.363293 1165 ts_tablet_manager.cc:1929] T ee893e33255745debf73c486d8c70267 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:33.364473 1165 log.cc:1199] T ee893e33255745debf73c486d8c70267 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/ee893e33255745debf73c486d8c70267
I20251211 20:52:33.365116 1165 ts_tablet_manager.cc:1950] T ee893e33255745debf73c486d8c70267 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:33.368371 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet ee893e33255745debf73c486d8c70267 (table default.Table_da9a07e5af4a433c8e02a128e8843084 [id=3940282e8e794b21b982be1f171b677d]) successfully deleted
I20251211 20:52:33.368659 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f4fa28c6ade240c69a5b4a98b44eac5a" } modify_external_catalogs: true
I20251211 20:52:33.368861 1165 tablet_replica.cc:333] T 27a08a64d92342d08ab4406e8006a97b P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:33.368873 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f4fa28c6ade240c69a5b4a98b44eac5a" } modify_external_catalogs: true
I20251211 20:52:33.369074 1165 raft_consensus.cc:2243] T 27a08a64d92342d08ab4406e8006a97b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:33.369249 1165 raft_consensus.cc:2272] T 27a08a64d92342d08ab4406e8006a97b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:33.371775 1165 ts_tablet_manager.cc:1916] T 27a08a64d92342d08ab4406e8006a97b P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:33.371874 1164 ts_tablet_manager.cc:1929] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:33.372073 1164 log.cc:1199] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/4b736a1b347d44b398980654caa21332
I20251211 20:52:33.372481 1164 ts_tablet_manager.cc:1950] T 4b736a1b347d44b398980654caa21332 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:33.372867 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 546) took 1425 ms. Trace:
I20251211 20:52:33.373013 1164 rpcz_store.cc:276] 1211 20:52:31.947160 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:31.947215 (+ 55us) service_pool.cc:225] Handling call
1211 20:52:33.225901 (+1278686us) ts_tablet_manager.cc:1151] Deleting tablet 4b736a1b347d44b398980654caa21332
1211 20:52:33.225915 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:33.296506 (+ 70591us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:33.371742 (+ 75236us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:33.372817 (+ 1075us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1101923}
I20251211 20:52:33.373585 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 4b736a1b347d44b398980654caa21332 (table default.Table_f9014306b36e47b98c8e4c6070ea4a37 [id=4e64e54fb4fc42d48679371e4bc1efe5]) successfully deleted
I20251211 20:52:33.374112 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 557) took 1070 ms. Trace:
I20251211 20:52:33.374257 1164 rpcz_store.cc:276] 1211 20:52:32.303355 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:32.303402 (+ 47us) service_pool.cc:225] Handling call
1211 20:52:33.373944 (+1070542us) ts_tablet_manager.cc:1151] Deleting tablet 3acfef6f082641a199b80eefcfc772ac
1211 20:52:33.373954 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:33.374060 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1070298}
W20251211 20:52:33.374781 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 3acfef6f082641a199b80eefcfc772ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3acfef6f082641a199b80eefcfc772ac already in progress: creating tablet
I20251211 20:52:33.375212 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 558) took 1071 ms. Trace:
I20251211 20:52:33.346419 1166 raft_consensus.cc:359] T 05385f20e88c46108e09a78814811209 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.377254 1166 raft_consensus.cc:385] T 05385f20e88c46108e09a78814811209 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.378602 1166 raft_consensus.cc:740] T 05385f20e88c46108e09a78814811209 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
W20251211 20:52:33.379347 588 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f94b656e3c1844509fdae36e33b263e8 consensus state has no leader
W20251211 20:52:33.381827 581 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet b529b38d754f46ba8cb60b19e9705256 (table default.Table_742d698db63f432aa7b88ceccf6dc0fc [id=6e5fe0a78e4b48f98dac028802f1ddf4], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet b529b38d754f46ba8cb60b19e9705256 consensus state has no leader
I20251211 20:52:33.381635 1166 consensus_queue.cc:260] T 05385f20e88c46108e09a78814811209 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.386583 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b7f9abd9a3b04fd4a6ef3b477330ea9e" } modify_external_catalogs: true
I20251211 20:52:33.386847 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b7f9abd9a3b04fd4a6ef3b477330ea9e" } modify_external_catalogs: true
I20251211 20:52:33.394161 1163 ts_tablet_manager.cc:1929] T bc32a778e5424850b23ccaf74bb9df07 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:33.387737 1160 raft_consensus.cc:740] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.394320 1163 log.cc:1199] T bc32a778e5424850b23ccaf74bb9df07 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/bc32a778e5424850b23ccaf74bb9df07
I20251211 20:52:33.394670 1163 ts_tablet_manager.cc:1950] T bc32a778e5424850b23ccaf74bb9df07 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:33.396019 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet bc32a778e5424850b23ccaf74bb9df07 (table default.Table_e6252813479645a98ce2709dc7aba798 [id=db38368266194271a02ad265fd991439]) successfully deleted
I20251211 20:52:33.396104 1160 consensus_queue.cc:260] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.396746 1166 ts_tablet_manager.cc:1434] T 05385f20e88c46108e09a78814811209 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.054s user 0.003s sys 0.000s
I20251211 20:52:33.397239 597 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet dd425601d89148eb95525f796a3abf1e (table default.Table_d4afd6ed615a41ac8f6714c8aac27857 [id=c842792b01cb4c689fcfd3d796151508]). Expected version 1 got 0
I20251211 20:52:33.398248 1758 raft_consensus.cc:493] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:33.398384 1758 raft_consensus.cc:515] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.398954 1758 leader_election.cc:290] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:33.400355 1165 ts_tablet_manager.cc:1929] T 27a08a64d92342d08ab4406e8006a97b P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:33.401041 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8488ced67fe54a25918bbd9c8c9934e8" } new_table_name: "default.Table_fa0592c7846d4e4883138ca9e597a452" modify_external_catalogs: true
I20251211 20:52:33.402640 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 05385f20e88c46108e09a78814811209 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:33.404217 1166 tablet_bootstrap.cc:492] T 3c578d81e8dd41aaa0f707c4e3fed225 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:33.406330 1166 tablet_bootstrap.cc:654] T 3c578d81e8dd41aaa0f707c4e3fed225 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.411065 1166 tablet_bootstrap.cc:492] T 3c578d81e8dd41aaa0f707c4e3fed225 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:33.411384 1165 log.cc:1199] T 27a08a64d92342d08ab4406e8006a97b P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/27a08a64d92342d08ab4406e8006a97b
I20251211 20:52:33.411999 1165 ts_tablet_manager.cc:1950] T 27a08a64d92342d08ab4406e8006a97b P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:33.411872 1164 rpcz_store.cc:276] 1211 20:52:32.303440 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:32.303509 (+ 69us) service_pool.cc:225] Handling call
1211 20:52:33.375070 (+1071561us) ts_tablet_manager.cc:1151] Deleting tablet 648f43e95e034f2188a20e1b837f2818
1211 20:52:33.375079 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:33.375163 (+ 84us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1071384}
W20251211 20:52:33.412477 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 648f43e95e034f2188a20e1b837f2818 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 648f43e95e034f2188a20e1b837f2818 already in progress: creating tablet
I20251211 20:52:33.412909 1166 ts_tablet_manager.cc:1403] T 3c578d81e8dd41aaa0f707c4e3fed225 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20251211 20:52:33.415346 1166 raft_consensus.cc:359] T 3c578d81e8dd41aaa0f707c4e3fed225 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.416610 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 27a08a64d92342d08ab4406e8006a97b (table default.Table_3721b5725e8c4297a392d3986a081128 [id=383392d835da4c50848068d83261cecc]) successfully deleted
I20251211 20:52:33.416769 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 554) took 1113 ms. Trace:
I20251211 20:52:33.416867 1164 rpcz_store.cc:276] 1211 20:52:32.303069 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:32.303693 (+ 624us) service_pool.cc:225] Handling call
1211 20:52:33.414124 (+1110431us) ts_tablet_manager.cc:1151] Deleting tablet f9a3fdf17f274c15ac0bdfa9c174f649
1211 20:52:33.414135 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:33.416654 (+ 2519us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1110241}
W20251211 20:52:33.417367 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet f9a3fdf17f274c15ac0bdfa9c174f649 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f9a3fdf17f274c15ac0bdfa9c174f649 already in progress: creating tablet
I20251211 20:52:33.417870 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 556) took 1114 ms. Trace:
I20251211 20:52:33.417989 1164 rpcz_store.cc:276] 1211 20:52:32.303235 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:32.304076 (+ 841us) service_pool.cc:225] Handling call
1211 20:52:33.417647 (+1113571us) ts_tablet_manager.cc:1151] Deleting tablet 14e2650f8ec1482ea9511b989fdc963c
1211 20:52:33.417659 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:33.417774 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1113444}
I20251211 20:52:33.418730 1166 raft_consensus.cc:385] T 3c578d81e8dd41aaa0f707c4e3fed225 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.419036 1166 raft_consensus.cc:740] T 3c578d81e8dd41aaa0f707c4e3fed225 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.419473 1166 consensus_queue.cc:260] T 3c578d81e8dd41aaa0f707c4e3fed225 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.421983 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_aab9c3f649474051bcf63611d32aa48a" } modify_external_catalogs: true
I20251211 20:52:33.422196 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_aab9c3f649474051bcf63611d32aa48a" } modify_external_catalogs: true
I20251211 20:52:33.424032 1166 ts_tablet_manager.cc:1434] T 3c578d81e8dd41aaa0f707c4e3fed225 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
W20251211 20:52:33.424540 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 14e2650f8ec1482ea9511b989fdc963c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14e2650f8ec1482ea9511b989fdc963c already in progress: creating tablet
I20251211 20:52:33.424885 1163 tablet_replica.cc:333] T 05385f20e88c46108e09a78814811209 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:33.425027 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 555) took 1121 ms. Trace:
I20251211 20:52:33.425119 1164 rpcz_store.cc:276] 1211 20:52:32.303145 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:32.303910 (+ 765us) service_pool.cc:225] Handling call
1211 20:52:33.424855 (+1120945us) ts_tablet_manager.cc:1151] Deleting tablet 01104975fd8b470aa026578b97ff6b8d
1211 20:52:33.424867 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:33.424973 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1113052}
W20251211 20:52:33.425549 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 01104975fd8b470aa026578b97ff6b8d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 01104975fd8b470aa026578b97ff6b8d already in progress: creating tablet
I20251211 20:52:33.427801 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 569) took 1020 ms. Trace:
I20251211 20:52:33.427918 1164 rpcz_store.cc:276] 1211 20:52:32.406812 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:32.406859 (+ 47us) service_pool.cc:225] Handling call
1211 20:52:33.427626 (+1020767us) ts_tablet_manager.cc:1151] Deleting tablet 3621735fcb294e3a8c0180ba9ef86442
1211 20:52:33.427638 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:33.427745 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1020536}
W20251211 20:52:33.428303 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 3621735fcb294e3a8c0180ba9ef86442 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3621735fcb294e3a8c0180ba9ef86442 already in progress: creating tablet
I20251211 20:52:33.429437 1163 raft_consensus.cc:2243] T 05385f20e88c46108e09a78814811209 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:33.429947 1163 raft_consensus.cc:2272] T 05385f20e88c46108e09a78814811209 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:33.425930 1166 tablet_bootstrap.cc:492] T 40722aef4e194ff094d271a079bda9d9 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:33.431387 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 88970b575ba943129729eceef5636519 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 88970b575ba943129729eceef5636519 already in progress: creating tablet
I20251211 20:52:33.432301 1163 ts_tablet_manager.cc:1916] T 05385f20e88c46108e09a78814811209 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:33.432610 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 6663f13691a24fb991b48f543d9de76b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6663f13691a24fb991b48f543d9de76b already in progress: creating tablet
W20251211 20:52:33.433583 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet c8214c97ed1640f5965aa5c4b999e2f1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c8214c97ed1640f5965aa5c4b999e2f1 already in progress: creating tablet
I20251211 20:52:33.434438 1166 tablet_bootstrap.cc:654] T 40722aef4e194ff094d271a079bda9d9 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:33.435849 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 12aaf618d6474f599458026308d7afdd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12aaf618d6474f599458026308d7afdd already in progress: creating tablet
W20251211 20:52:33.439662 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 5ddf7c1972564ef88e3c0a0bd1b430e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5ddf7c1972564ef88e3c0a0bd1b430e0 already in progress: creating tablet
W20251211 20:52:33.440521 591 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6ad6ab53aad14cc2b6d201acfecf6723 consensus state has no leader
W20251211 20:52:33.443403 847 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.1:42177, user_credentials={real_user=slave}} blocked reactor thread for 45720.2us
W20251211 20:52:33.445258 847 leader_election.cc:343] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: dc40b9826ea84728910b1fd61ce62a25
W20251211 20:52:33.445508 847 leader_election.cc:343] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: dc40b9826ea84728910b1fd61ce62a25
I20251211 20:52:33.445650 847 leader_election.cc:304] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:33.445916 1758 raft_consensus.cc:2749] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:33.447288 1166 tablet_bootstrap.cc:492] T 40722aef4e194ff094d271a079bda9d9 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:33.451211 891 tablet_service.cc:1552] Processing DeleteTablet for tablet b8828385cec34d70b717ae7f3813b9c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:23 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:33.448357 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 3c578d81e8dd41aaa0f707c4e3fed225 with delete_type TABLET_DATA_DELETED (replaced by tablet b115d737d7fc483d9f5a1f040b511bae) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:33.451881 1166 ts_tablet_manager.cc:1403] T 40722aef4e194ff094d271a079bda9d9 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.002s
W20251211 20:52:33.452001 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 845b5ac4a5984c0cb4ab188925103dc9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 845b5ac4a5984c0cb4ab188925103dc9 already in progress: creating tablet
W20251211 20:52:33.452641 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 13ceabb190494e718a52892faaa53986 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 13ceabb190494e718a52892faaa53986 already in progress: creating tablet
I20251211 20:52:33.452772 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_06ad70486e504f4ca804a4470a0b65a1" } new_table_name: "default.Table_819c4206e91e41358ac71e82185b4140" modify_external_catalogs: true
W20251211 20:52:33.453199 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet ba120d28abc54553ae15532de90db719 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba120d28abc54553ae15532de90db719 already in progress: creating tablet
I20251211 20:52:33.453734 1160 ts_tablet_manager.cc:1434] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.112s user 0.005s sys 0.005s
I20251211 20:52:33.453703 1166 raft_consensus.cc:359] T 40722aef4e194ff094d271a079bda9d9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.455631 1160 tablet_bootstrap.cc:492] T 6a206b37e43247b5babe3113eab2c9fb P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:33.457626 1166 raft_consensus.cc:385] T 40722aef4e194ff094d271a079bda9d9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.457965 1166 raft_consensus.cc:740] T 40722aef4e194ff094d271a079bda9d9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.458410 1166 consensus_queue.cc:260] T 40722aef4e194ff094d271a079bda9d9 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.464524 1166 ts_tablet_manager.cc:1434] T 40722aef4e194ff094d271a079bda9d9 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20251211 20:52:33.468573 1764 tablet.cc:1722] T dd425601d89148eb95525f796a3abf1e P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:33.468709 1166 tablet_bootstrap.cc:492] T d762b4afb5ff447dac3468393826205a P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:33.470687 1163 ts_tablet_manager.cc:1929] T 05385f20e88c46108e09a78814811209 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:33.471534 1163 log.cc:1199] T 05385f20e88c46108e09a78814811209 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/05385f20e88c46108e09a78814811209
I20251211 20:52:33.472159 1163 ts_tablet_manager.cc:1950] T 05385f20e88c46108e09a78814811209 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:33.473776 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 05385f20e88c46108e09a78814811209 (table default.Table_2a5ff353268344dba84653a0fd41e4a0 [id=c06795252c09401b99e58c702574ac05]) successfully deleted
W20251211 20:52:33.474308 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet c26c3d0170f64a41ae8944c6668cc5f3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c26c3d0170f64a41ae8944c6668cc5f3 already in progress: creating tablet
I20251211 20:52:33.474941 1166 tablet_bootstrap.cc:654] T d762b4afb5ff447dac3468393826205a P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.475525 1164 tablet_replica.cc:333] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:33.475546 1163 tablet_replica.cc:333] T 3c578d81e8dd41aaa0f707c4e3fed225 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:33.477154 1163 raft_consensus.cc:2243] T 3c578d81e8dd41aaa0f707c4e3fed225 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:33.477568 1163 raft_consensus.cc:2272] T 3c578d81e8dd41aaa0f707c4e3fed225 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:33.478202 1792 tablet.cc:1722] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:33.478727 1163 ts_tablet_manager.cc:1916] T 3c578d81e8dd41aaa0f707c4e3fed225 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:33.481336 1164 raft_consensus.cc:2243] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:33.481858 1164 raft_consensus.cc:2272] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:33.482662 1164 ts_tablet_manager.cc:1916] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:33.480909 1160 tablet_bootstrap.cc:654] T 6a206b37e43247b5babe3113eab2c9fb P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.492931 1696 raft_consensus.cc:493] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:33.493287 1696 raft_consensus.cc:515] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.494268 1773 tablet.cc:1722] T dd425601d89148eb95525f796a3abf1e P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:33.494575 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2630ad57845a4c31b20c8a104dd27335" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:33.495205 711 leader_election.cc:343] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:33.495731 1696 leader_election.cc:290] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:33.498035 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2630ad57845a4c31b20c8a104dd27335" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:33.498263 1046 raft_consensus.cc:2468] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:33.498838 710 leader_election.cc:304] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:33.499137 1828 raft_consensus.cc:2804] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:33.499228 1828 raft_consensus.cc:493] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:33.499433 1828 raft_consensus.cc:3060] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:33.503371 1828 raft_consensus.cc:515] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.508582 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2630ad57845a4c31b20c8a104dd27335" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:33.509296 711 leader_election.cc:343] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:33.509752 1163 ts_tablet_manager.cc:1929] T 3c578d81e8dd41aaa0f707c4e3fed225 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:33.510375 1163 log.cc:1199] T 3c578d81e8dd41aaa0f707c4e3fed225 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/3c578d81e8dd41aaa0f707c4e3fed225
I20251211 20:52:33.511075 1163 ts_tablet_manager.cc:1950] T 3c578d81e8dd41aaa0f707c4e3fed225 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:33.512152 1828 leader_election.cc:290] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:33.515043 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 3c578d81e8dd41aaa0f707c4e3fed225 (table default.Table_6d94053661f142b79292e19752df02df [id=c5be59a2c3934b3789641ceeb2d4d6e2]) successfully deleted
I20251211 20:52:33.515386 1168 tablet_bootstrap.cc:492] T ae1abef61473443b9922eb13f2ae92fa P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:33.515570 1168 ts_tablet_manager.cc:1403] T ae1abef61473443b9922eb13f2ae92fa P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.187s user 0.043s sys 0.074s
I20251211 20:52:33.516435 1168 raft_consensus.cc:359] T ae1abef61473443b9922eb13f2ae92fa P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.516661 1168 raft_consensus.cc:385] T ae1abef61473443b9922eb13f2ae92fa P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.516739 1168 raft_consensus.cc:740] T ae1abef61473443b9922eb13f2ae92fa P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.516983 1168 consensus_queue.cc:260] T ae1abef61473443b9922eb13f2ae92fa P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.517529 1168 ts_tablet_manager.cc:1434] T ae1abef61473443b9922eb13f2ae92fa P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:33.517823 1168 tablet_bootstrap.cc:492] T 5fd2a6cf80a04f20aef8ff9c84bc88db P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:33.512131 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2630ad57845a4c31b20c8a104dd27335" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:33.518009 1044 raft_consensus.cc:3060] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:33.519676 1168 tablet_bootstrap.cc:654] T 5fd2a6cf80a04f20aef8ff9c84bc88db P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.523067 1044 raft_consensus.cc:2468] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:33.523594 710 leader_election.cc:304] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:33.523885 1828 raft_consensus.cc:2804] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:33.524029 1828 raft_consensus.cc:697] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:33.525213 1828 consensus_queue.cc:237] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.527709 1168 tablet_bootstrap.cc:492] T 5fd2a6cf80a04f20aef8ff9c84bc88db P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:33.527863 1168 ts_tablet_manager.cc:1403] T 5fd2a6cf80a04f20aef8ff9c84bc88db P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20251211 20:52:33.528679 1168 raft_consensus.cc:359] T 5fd2a6cf80a04f20aef8ff9c84bc88db P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.528870 1168 raft_consensus.cc:385] T 5fd2a6cf80a04f20aef8ff9c84bc88db P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.528962 1168 raft_consensus.cc:740] T 5fd2a6cf80a04f20aef8ff9c84bc88db P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.529161 1168 consensus_queue.cc:260] T 5fd2a6cf80a04f20aef8ff9c84bc88db P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.529664 1168 ts_tablet_manager.cc:1434] T 5fd2a6cf80a04f20aef8ff9c84bc88db P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:33.529948 1168 tablet_bootstrap.cc:492] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:33.531646 1168 tablet_bootstrap.cc:654] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.533025 1166 tablet_bootstrap.cc:492] T d762b4afb5ff447dac3468393826205a P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:33.534323 1166 ts_tablet_manager.cc:1403] T d762b4afb5ff447dac3468393826205a P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.066s user 0.005s sys 0.010s
I20251211 20:52:33.534549 1168 tablet_bootstrap.cc:492] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:33.534706 1168 ts_tablet_manager.cc:1403] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251211 20:52:33.535475 1168 raft_consensus.cc:359] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.535706 1168 raft_consensus.cc:385] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.535778 1168 raft_consensus.cc:740] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.535995 1168 consensus_queue.cc:260] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.536450 1168 ts_tablet_manager.cc:1434] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:33.536563 1160 tablet_bootstrap.cc:492] T 6a206b37e43247b5babe3113eab2c9fb P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:33.536726 1160 ts_tablet_manager.cc:1403] T 6a206b37e43247b5babe3113eab2c9fb P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.081s user 0.004s sys 0.000s
I20251211 20:52:33.536756 1168 tablet_bootstrap.cc:492] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:33.537484 1160 raft_consensus.cc:359] T 6a206b37e43247b5babe3113eab2c9fb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.537683 1160 raft_consensus.cc:385] T 6a206b37e43247b5babe3113eab2c9fb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.538026 1160 raft_consensus.cc:740] T 6a206b37e43247b5babe3113eab2c9fb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.538260 1160 consensus_queue.cc:260] T 6a206b37e43247b5babe3113eab2c9fb P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.538358 1168 tablet_bootstrap.cc:654] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.539145 1166 raft_consensus.cc:359] T d762b4afb5ff447dac3468393826205a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.539923 1166 raft_consensus.cc:385] T d762b4afb5ff447dac3468393826205a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.540163 1166 raft_consensus.cc:740] T d762b4afb5ff447dac3468393826205a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.540504 1166 consensus_queue.cc:260] T d762b4afb5ff447dac3468393826205a P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.541471 1168 tablet_bootstrap.cc:492] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:33.541615 1168 ts_tablet_manager.cc:1403] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251211 20:52:33.541810 1160 ts_tablet_manager.cc:1434] T 6a206b37e43247b5babe3113eab2c9fb P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251211 20:52:33.542718 1168 raft_consensus.cc:359] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.542884 1168 raft_consensus.cc:385] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.542966 1168 raft_consensus.cc:740] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.543159 1168 consensus_queue.cc:260] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.543725 1168 ts_tablet_manager.cc:1434] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:33.543987 1168 tablet_bootstrap.cc:492] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:33.546149 1168 tablet_bootstrap.cc:654] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.546952 1160 tablet_bootstrap.cc:492] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:33.547366 1166 ts_tablet_manager.cc:1434] T d762b4afb5ff447dac3468393826205a P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20251211 20:52:33.548266 1166 tablet_bootstrap.cc:492] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:33.548589 1168 tablet_bootstrap.cc:492] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:33.548748 1168 ts_tablet_manager.cc:1403] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251211 20:52:33.549559 1168 raft_consensus.cc:359] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.549746 1168 raft_consensus.cc:385] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.549829 1168 raft_consensus.cc:740] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.550032 1168 consensus_queue.cc:260] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.550468 1168 ts_tablet_manager.cc:1434] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:33.550763 1168 tablet_bootstrap.cc:492] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:33.552386 1168 tablet_bootstrap.cc:654] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.552415 1160 tablet_bootstrap.cc:654] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.554308 1166 tablet_bootstrap.cc:654] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.555339 1168 tablet_bootstrap.cc:492] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:33.555490 1168 ts_tablet_manager.cc:1403] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20251211 20:52:33.556301 1168 raft_consensus.cc:359] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.556464 1168 raft_consensus.cc:385] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.556556 1168 raft_consensus.cc:740] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.556598 1164 ts_tablet_manager.cc:1929] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:33.556774 1164 log.cc:1199] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/0246f1df99684d0da43d97acbf1182a0
I20251211 20:52:33.556751 1168 consensus_queue.cc:260] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.557345 1168 ts_tablet_manager.cc:1434] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:33.557745 1168 tablet_bootstrap.cc:492] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:33.559355 1168 tablet_bootstrap.cc:654] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.557752 1164 ts_tablet_manager.cc:1950] T 0246f1df99684d0da43d97acbf1182a0 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:33.560595 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 0246f1df99684d0da43d97acbf1182a0 (table default.Table_03fdaef4fcc144f485f238db0e76fab4 [id=4264180c89854054b14e98d5a49984fa]) successfully deleted
I20251211 20:52:33.562256 1166 tablet_bootstrap.cc:492] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:33.562872 1166 ts_tablet_manager.cc:1403] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.002s
W20251211 20:52:33.563946 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet bd2bf1113dfe4b67b263b56f3d3db07d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd2bf1113dfe4b67b263b56f3d3db07d already in progress: creating tablet
W20251211 20:52:33.564604 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet c112714331bf4bd7a1ddcca17f338d4f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c112714331bf4bd7a1ddcca17f338d4f already in progress: creating tablet
I20251211 20:52:33.563992 1166 raft_consensus.cc:359] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.565423 1166 raft_consensus.cc:385] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.565692 1166 raft_consensus.cc:740] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
W20251211 20:52:33.566844 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 84ed434868aa437a83455b1605deb26c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84ed434868aa437a83455b1605deb26c already in progress: creating tablet
I20251211 20:52:33.566821 1166 consensus_queue.cc:260] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:33.567612 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 84b2a67c836844ceb3628cc3b7d8ba10 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84b2a67c836844ceb3628cc3b7d8ba10 already in progress: creating tablet
I20251211 20:52:33.567633 1813 raft_consensus.cc:493] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:33.567766 1813 raft_consensus.cc:515] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:33.568194 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet a9b2aa21ab124b28aebc33adba0583d2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a9b2aa21ab124b28aebc33adba0583d2 already in progress: creating tablet
I20251211 20:52:33.568496 1813 leader_election.cc:290] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:33.569693 847 leader_election.cc:343] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: b8828385cec34d70b717ae7f3813b9c3
W20251211 20:52:33.569918 847 leader_election.cc:343] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: b8828385cec34d70b717ae7f3813b9c3
I20251211 20:52:33.570010 847 leader_election.cc:304] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:33.570688 1166 ts_tablet_manager.cc:1434] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20251211 20:52:33.571005 1813 raft_consensus.cc:2749] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:33.571362 1164 tablet_replica.cc:333] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:33.571626 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 06b8f8e46e524545aeab6c931efaf116 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:33.572778 1166 tablet_bootstrap.cc:492] T 9f8aa28c1c9b4fc58dfc128952c180ba P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:33.573029 1164 raft_consensus.cc:2243] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:33.573213 1164 raft_consensus.cc:2272] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:33.576321 1168 tablet_bootstrap.cc:492] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:33.576508 1168 ts_tablet_manager.cc:1403] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.000s
I20251211 20:52:33.577085 1164 ts_tablet_manager.cc:1916] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:33.577266 1168 raft_consensus.cc:359] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.577458 1168 raft_consensus.cc:385] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.577540 1168 raft_consensus.cc:740] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.577961 1168 consensus_queue.cc:260] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.578454 1168 ts_tablet_manager.cc:1434] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:33.578773 1168 tablet_bootstrap.cc:492] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:33.580476 1168 tablet_bootstrap.cc:654] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.582016 1160 tablet_bootstrap.cc:492] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:33.582177 1160 ts_tablet_manager.cc:1403] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.035s user 0.004s sys 0.000s
I20251211 20:52:33.583070 1160 raft_consensus.cc:359] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.583261 1160 raft_consensus.cc:385] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.583341 1160 raft_consensus.cc:740] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.583575 1160 consensus_queue.cc:260] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.585443 1166 tablet_bootstrap.cc:654] T 9f8aa28c1c9b4fc58dfc128952c180ba P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.586122 1160 ts_tablet_manager.cc:1434] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:33.586432 1160 tablet_bootstrap.cc:492] T 8b89ae724b7f41c98741a41d75592da7 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:33.588153 1160 tablet_bootstrap.cc:654] T 8b89ae724b7f41c98741a41d75592da7 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.591832 1168 tablet_bootstrap.cc:492] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:33.591985 1168 ts_tablet_manager.cc:1403] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20251211 20:52:33.592973 1168 raft_consensus.cc:359] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.593179 1168 raft_consensus.cc:385] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.593268 1168 raft_consensus.cc:740] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.593515 1168 consensus_queue.cc:260] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.594053 1168 ts_tablet_manager.cc:1434] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:33.594347 1168 tablet_bootstrap.cc:492] T d30376baf75b4761971f91da3493a142 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:33.596086 1168 tablet_bootstrap.cc:654] T d30376baf75b4761971f91da3493a142 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.597905 1164 ts_tablet_manager.cc:1929] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:33.604053 1164 log.cc:1199] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/c38c1a4c0e574eb0957de127d41c26ad
I20251211 20:52:33.604790 1164 ts_tablet_manager.cc:1950] T c38c1a4c0e574eb0957de127d41c26ad P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:33.604875 1166 tablet_bootstrap.cc:492] T 9f8aa28c1c9b4fc58dfc128952c180ba P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:33.605381 1166 ts_tablet_manager.cc:1403] T 9f8aa28c1c9b4fc58dfc128952c180ba P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.033s user 0.006s sys 0.000s
W20251211 20:52:33.606333 591 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 861d2d70a24946699dccd65cab05195e consensus state has no leader
I20251211 20:52:33.606292 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet c38c1a4c0e574eb0957de127d41c26ad (table default.Table_88975c1ad9604f50b7f69fbbdacabce3 [id=52f34b88d367454286943a27138b20f2]) successfully deleted
W20251211 20:52:33.606600 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ab4e7a9062134e489d6f9abe6ff26fb4 consensus state has no leader
I20251211 20:52:33.608021 1164 tablet_replica.cc:333] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:33.608475 1164 raft_consensus.cc:2243] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:33.609427 1164 raft_consensus.cc:2272] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:33.609210 1160 tablet_bootstrap.cc:492] T 8b89ae724b7f41c98741a41d75592da7 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:33.609647 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_70ec351f0d724592bcfb6bf8b469d84d" } modify_external_catalogs: true
I20251211 20:52:33.609807 1160 ts_tablet_manager.cc:1403] T 8b89ae724b7f41c98741a41d75592da7 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.023s user 0.005s sys 0.000s
I20251211 20:52:33.609817 1166 raft_consensus.cc:359] T 9f8aa28c1c9b4fc58dfc128952c180ba P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.610092 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_70ec351f0d724592bcfb6bf8b469d84d" } modify_external_catalogs: true
I20251211 20:52:33.610430 1166 raft_consensus.cc:385] T 9f8aa28c1c9b4fc58dfc128952c180ba P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.610559 1160 raft_consensus.cc:359] T 8b89ae724b7f41c98741a41d75592da7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.610772 1160 raft_consensus.cc:385] T 8b89ae724b7f41c98741a41d75592da7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.610476 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_bc06d56f49324a42b6e5a68cb9ef1434" } modify_external_catalogs: true
I20251211 20:52:33.610845 1160 raft_consensus.cc:740] T 8b89ae724b7f41c98741a41d75592da7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.610891 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_bc06d56f49324a42b6e5a68cb9ef1434" } modify_external_catalogs: true
I20251211 20:52:33.610796 1166 raft_consensus.cc:740] T 9f8aa28c1c9b4fc58dfc128952c180ba P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.611138 1160 consensus_queue.cc:260] T 8b89ae724b7f41c98741a41d75592da7 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.611254 1166 consensus_queue.cc:260] T 9f8aa28c1c9b4fc58dfc128952c180ba P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.611804 1160 ts_tablet_manager.cc:1434] T 8b89ae724b7f41c98741a41d75592da7 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:33.612125 1160 tablet_bootstrap.cc:492] T ee893e33255745debf73c486d8c70267 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:33.614305 1164 ts_tablet_manager.cc:1916] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:33.615463 1166 ts_tablet_manager.cc:1434] T 9f8aa28c1c9b4fc58dfc128952c180ba P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20251211 20:52:33.616775 1166 tablet_bootstrap.cc:492] T 822abda3c85848728205b71847ef7029 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:33.615355 1160 tablet_bootstrap.cc:654] T ee893e33255745debf73c486d8c70267 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.622601 1166 tablet_bootstrap.cc:654] T 822abda3c85848728205b71847ef7029 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.624253 590 master_service.cc:954] ReplaceTablet: received request to replace tablet 6a206b37e43247b5babe3113eab2c9fb from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:33.624951 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_91f5eb149cce4e40a34b953dac528702"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:33.625496 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91f5eb149cce4e40a34b953dac528702 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:33.626343 1168 tablet_bootstrap.cc:492] T d30376baf75b4761971f91da3493a142 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:33.627255 1168 ts_tablet_manager.cc:1403] T d30376baf75b4761971f91da3493a142 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.033s user 0.013s sys 0.015s
I20251211 20:52:33.628106 1168 raft_consensus.cc:359] T d30376baf75b4761971f91da3493a142 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.628296 1168 raft_consensus.cc:385] T d30376baf75b4761971f91da3493a142 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.628360 1168 raft_consensus.cc:740] T d30376baf75b4761971f91da3493a142 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.628582 1168 consensus_queue.cc:260] T d30376baf75b4761971f91da3493a142 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.629107 1168 ts_tablet_manager.cc:1434] T d30376baf75b4761971f91da3493a142 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:33.629464 1168 tablet_bootstrap.cc:492] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:33.632056 1168 tablet_bootstrap.cc:654] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.629846 1160 tablet_bootstrap.cc:492] T ee893e33255745debf73c486d8c70267 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:33.642208 1160 ts_tablet_manager.cc:1403] T ee893e33255745debf73c486d8c70267 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.030s user 0.005s sys 0.000s
I20251211 20:52:33.642944 592 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_d4afd6ed615a41ac8f6714c8aac27857 [id=c842792b01cb4c689fcfd3d796151508] alter complete (version 1)
I20251211 20:52:33.643276 1160 raft_consensus.cc:359] T ee893e33255745debf73c486d8c70267 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.643481 1160 raft_consensus.cc:385] T ee893e33255745debf73c486d8c70267 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.643568 1160 raft_consensus.cc:740] T ee893e33255745debf73c486d8c70267 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.643816 1160 consensus_queue.cc:260] T ee893e33255745debf73c486d8c70267 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.644345 1160 ts_tablet_manager.cc:1434] T ee893e33255745debf73c486d8c70267 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251211 20:52:33.644646 1160 tablet_bootstrap.cc:492] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:33.623569 1164 ts_tablet_manager.cc:1929] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:33.646358 1164 log.cc:1199] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/dc40b9826ea84728910b1fd61ce62a25
I20251211 20:52:33.646757 1164 ts_tablet_manager.cc:1950] T dc40b9826ea84728910b1fd61ce62a25 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:33.648610 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 062f980b65504d3baf99ae400e521775
I20251211 20:52:33.648674 1166 tablet_bootstrap.cc:492] T 822abda3c85848728205b71847ef7029 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:33.648841 1166 ts_tablet_manager.cc:1403] T 822abda3c85848728205b71847ef7029 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.002s
I20251211 20:52:33.649101 594 catalog_manager.cc:6724] ReplaceTablet: tablet 062f980b65504d3baf99ae400e521775 deleted and replaced by tablet 33fd09ea06ee47908b85911e4ee79708
I20251211 20:52:33.649684 1166 raft_consensus.cc:359] T 822abda3c85848728205b71847ef7029 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.649911 1166 raft_consensus.cc:385] T 822abda3c85848728205b71847ef7029 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.650041 1166 raft_consensus.cc:740] T 822abda3c85848728205b71847ef7029 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.650177 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 062f980b65504d3baf99ae400e521775 with delete_type TABLET_DATA_DELETED (replaced by tablet 33fd09ea06ee47908b85911e4ee79708) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:33.650782 1166 consensus_queue.cc:260] T 822abda3c85848728205b71847ef7029 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.650954 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 062f980b65504d3baf99ae400e521775 with delete_type TABLET_DATA_DELETED (replaced by tablet 33fd09ea06ee47908b85911e4ee79708) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:33.651233 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 062f980b65504d3baf99ae400e521775 with delete_type TABLET_DATA_DELETED (replaced by tablet 33fd09ea06ee47908b85911e4ee79708) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:33.651682 1160 tablet_bootstrap.cc:654] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:33.652107 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 062f980b65504d3baf99ae400e521775 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 062f980b65504d3baf99ae400e521775 already in progress: creating tablet
I20251211 20:52:33.652428 1166 ts_tablet_manager.cc:1434] T 822abda3c85848728205b71847ef7029 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
W20251211 20:52:33.652544 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 062f980b65504d3baf99ae400e521775 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 062f980b65504d3baf99ae400e521775 already in progress: creating tablet
I20251211 20:52:33.652720 1166 tablet_bootstrap.cc:492] T 28ef13be3b84410d9c786c43502a3478 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:33.652998 1164 tablet_replica.cc:333] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:33.653139 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet dc40b9826ea84728910b1fd61ce62a25 (table default.Table_e6252813479645a98ce2709dc7aba798 [id=db38368266194271a02ad265fd991439]) successfully deleted
I20251211 20:52:33.654836 597 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 79bd9916d0434e88ac0cbd242ec2b525
I20251211 20:52:33.655956 1164 raft_consensus.cc:2243] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:33.656258 1166 tablet_bootstrap.cc:654] T 28ef13be3b84410d9c786c43502a3478 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.659747 1164 raft_consensus.cc:2272] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:33.660634 1164 ts_tablet_manager.cc:1916] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:33.662010 1168 tablet_bootstrap.cc:492] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:33.662185 1168 ts_tablet_manager.cc:1403] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.009s
I20251211 20:52:33.663046 1168 raft_consensus.cc:359] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.663277 1168 raft_consensus.cc:385] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.663400 1168 raft_consensus.cc:740] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.663868 1168 consensus_queue.cc:260] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.664414 1168 ts_tablet_manager.cc:1434] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251211 20:52:33.664710 1168 tablet_bootstrap.cc:492] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:33.666168 1586 raft_consensus.cc:493] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:33.666402 1168 tablet_bootstrap.cc:654] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.666327 1586 raft_consensus.cc:515] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.666957 1586 leader_election.cc:290] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:33.667861 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3eac21a07ba34e1fa275e25ee9dcabb1" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:33.668078 776 raft_consensus.cc:2468] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 0.
I20251211 20:52:33.668612 982 leader_election.cc:304] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:33.669157 1586 raft_consensus.cc:2804] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:33.669251 1586 raft_consensus.cc:493] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:33.669335 1586 raft_consensus.cc:3060] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:33.671788 1828 raft_consensus.cc:493] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:33.671914 1828 raft_consensus.cc:515] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.672518 1828 leader_election.cc:290] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:33.673684 710 leader_election.cc:343] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 88066d94d5d34e0fab20da9fb6ef178e
I20251211 20:52:33.674387 1586 raft_consensus.cc:515] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.674749 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "88066d94d5d34e0fab20da9fb6ef178e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:33.675432 711 leader_election.cc:343] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:33.675432 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3eac21a07ba34e1fa275e25ee9dcabb1" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:33.675613 711 leader_election.cc:304] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:33.675734 776 raft_consensus.cc:3060] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:33.676151 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3eac21a07ba34e1fa275e25ee9dcabb1" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:33.677130 1160 tablet_bootstrap.cc:492] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:33.677338 1160 ts_tablet_manager.cc:1403] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.033s user 0.001s sys 0.003s
I20251211 20:52:33.678220 1160 raft_consensus.cc:359] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.678505 1160 raft_consensus.cc:385] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.678618 1160 raft_consensus.cc:740] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.678701 891 tablet_service.cc:1505] Processing CreateTablet for tablet d3b383ace72748d4bf21678894815c4f (DEFAULT_TABLE table=default.Table_0040a1e388eb4e58a352a1a3884345c4 [id=c2fb490b2d5d44f793304b8939018ca2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.678887 776 raft_consensus.cc:2468] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 1.
I20251211 20:52:33.679163 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3b383ace72748d4bf21678894815c4f. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:33.679546 983 leader_election.cc:343] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:33.680119 741 tablet_service.cc:1505] Processing CreateTablet for tablet d3b383ace72748d4bf21678894815c4f (DEFAULT_TABLE table=default.Table_0040a1e388eb4e58a352a1a3884345c4 [id=c2fb490b2d5d44f793304b8939018ca2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.680505 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3b383ace72748d4bf21678894815c4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.681025 1164 ts_tablet_manager.cc:1929] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:33.681156 1164 log.cc:1199] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/b8828385cec34d70b717ae7f3813b9c3
I20251211 20:52:33.681537 1164 ts_tablet_manager.cc:1950] T b8828385cec34d70b717ae7f3813b9c3 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:33.682498 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet b8828385cec34d70b717ae7f3813b9c3 (table default.Table_8352c9da1e324dd4847fd22078f22e53 [id=9dc54153dda4495684c2b07732795bfc]) successfully deleted
I20251211 20:52:33.683696 1828 raft_consensus.cc:2749] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:33.678905 1160 consensus_queue.cc:260] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.685273 1160 ts_tablet_manager.cc:1434] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.008s user 0.000s sys 0.001s
I20251211 20:52:33.685642 1160 tablet_bootstrap.cc:492] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:33.685600 1013 tablet_service.cc:1505] Processing CreateTablet for tablet d3b383ace72748d4bf21678894815c4f (DEFAULT_TABLE table=default.Table_0040a1e388eb4e58a352a1a3884345c4 [id=c2fb490b2d5d44f793304b8939018ca2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.685761 982 leader_election.cc:304] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:33.686039 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3b383ace72748d4bf21678894815c4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.686683 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3eac21a07ba34e1fa275e25ee9dcabb1" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:33.687286 983 leader_election.cc:343] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:33.687407 1160 tablet_bootstrap.cc:654] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.687553 1586 leader_election.cc:290] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Requested vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:33.687803 1586 raft_consensus.cc:2804] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:33.687994 1586 raft_consensus.cc:697] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 LEADER]: Becoming Leader. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Running, Role: LEADER
I20251211 20:52:33.691334 1586 consensus_queue.cc:237] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.700532 1166 tablet_bootstrap.cc:492] T 28ef13be3b84410d9c786c43502a3478 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
W20251211 20:52:33.700501 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 062f980b65504d3baf99ae400e521775 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 062f980b65504d3baf99ae400e521775 already in progress: creating tablet
I20251211 20:52:33.704936 741 tablet_service.cc:1505] Processing CreateTablet for tablet 6365572dac1449428ed371cf4e3545de (DEFAULT_TABLE table=default.Table_98f173a3b5454e16ac5680ec2465a09c [id=ab5b964850544c718f621b5ec87e3372]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.705508 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6365572dac1449428ed371cf4e3545de. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.711156 1166 ts_tablet_manager.cc:1403] T 28ef13be3b84410d9c786c43502a3478 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.054s user 0.004s sys 0.003s
I20251211 20:52:33.712456 1166 raft_consensus.cc:359] T 28ef13be3b84410d9c786c43502a3478 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.713367 891 tablet_service.cc:1505] Processing CreateTablet for tablet 6365572dac1449428ed371cf4e3545de (DEFAULT_TABLE table=default.Table_98f173a3b5454e16ac5680ec2465a09c [id=ab5b964850544c718f621b5ec87e3372]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.713852 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6365572dac1449428ed371cf4e3545de. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.714874 885 tablet_service.cc:1505] Processing CreateTablet for tablet 26b2f69913b44a4281b8d61097e780e4 (DEFAULT_TABLE table=default.Table_11314b6f726648c3bda96c9c1c7da29e [id=6ce96b52af404551a7e7e5186d54ee8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.715329 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26b2f69913b44a4281b8d61097e780e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.716142 1011 tablet_service.cc:1505] Processing CreateTablet for tablet 26b2f69913b44a4281b8d61097e780e4 (DEFAULT_TABLE table=default.Table_11314b6f726648c3bda96c9c1c7da29e [id=6ce96b52af404551a7e7e5186d54ee8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.717063 745 tablet_service.cc:1505] Processing CreateTablet for tablet 26b2f69913b44a4281b8d61097e780e4 (DEFAULT_TABLE table=default.Table_11314b6f726648c3bda96c9c1c7da29e [id=6ce96b52af404551a7e7e5186d54ee8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.717432 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26b2f69913b44a4281b8d61097e780e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.717557 883 tablet_service.cc:1505] Processing CreateTablet for tablet 2e29df291d8d40daba83e1d9f97f902b (DEFAULT_TABLE table=default.Table_f5ee1b1b44c645f28737720bc385a577 [id=f8a38441b37d41eab85bad3ba16b4c05]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.717913 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e29df291d8d40daba83e1d9f97f902b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.718029 739 tablet_service.cc:1505] Processing CreateTablet for tablet 2e29df291d8d40daba83e1d9f97f902b (DEFAULT_TABLE table=default.Table_f5ee1b1b44c645f28737720bc385a577 [id=f8a38441b37d41eab85bad3ba16b4c05]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.718411 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e29df291d8d40daba83e1d9f97f902b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.724391 1166 raft_consensus.cc:385] T 28ef13be3b84410d9c786c43502a3478 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.724596 1166 raft_consensus.cc:740] T 28ef13be3b84410d9c786c43502a3478 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.725816 1166 consensus_queue.cc:260] T 28ef13be3b84410d9c786c43502a3478 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.726421 1166 ts_tablet_manager.cc:1434] T 28ef13be3b84410d9c786c43502a3478 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20251211 20:52:33.729411 1166 tablet_bootstrap.cc:492] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:33.731305 1160 tablet_bootstrap.cc:492] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:33.732095 1160 ts_tablet_manager.cc:1403] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.046s user 0.008s sys 0.003s
I20251211 20:52:33.733028 1160 raft_consensus.cc:359] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.735373 1160 raft_consensus.cc:385] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.735571 1160 raft_consensus.cc:740] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.735914 1160 consensus_queue.cc:260] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.736462 1011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26b2f69913b44a4281b8d61097e780e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.738858 1828 raft_consensus.cc:493] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:33.739010 1828 raft_consensus.cc:515] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.739714 1828 leader_election.cc:290] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:33.740514 1166 tablet_bootstrap.cc:654] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.741051 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dc5de12701714fcd81b9dba6973a28b5" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:33.741261 1044 raft_consensus.cc:2468] T dc5de12701714fcd81b9dba6973a28b5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:33.740513 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dc5de12701714fcd81b9dba6973a28b5" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:33.742841 711 leader_election.cc:343] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:33.742957 1160 ts_tablet_manager.cc:1434] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.011s user 0.001s sys 0.000s
I20251211 20:52:33.743268 1160 tablet_bootstrap.cc:492] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:33.743271 710 leader_election.cc:304] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:33.743796 1828 raft_consensus.cc:2804] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:33.743917 1828 raft_consensus.cc:493] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:33.743996 1828 raft_consensus.cc:3060] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:33.746548 1160 tablet_bootstrap.cc:654] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.746683 1828 raft_consensus.cc:515] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.748234 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dc5de12701714fcd81b9dba6973a28b5" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:33.748849 711 leader_election.cc:343] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:33.749370 1828 leader_election.cc:290] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:33.749644 1013 tablet_service.cc:1505] Processing CreateTablet for tablet 6365572dac1449428ed371cf4e3545de (DEFAULT_TABLE table=default.Table_98f173a3b5454e16ac5680ec2465a09c [id=ab5b964850544c718f621b5ec87e3372]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.750028 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dc5de12701714fcd81b9dba6973a28b5" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:33.750233 1044 raft_consensus.cc:3060] T dc5de12701714fcd81b9dba6973a28b5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:33.750348 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6365572dac1449428ed371cf4e3545de. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.753268 1044 raft_consensus.cc:2468] T dc5de12701714fcd81b9dba6973a28b5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:33.754232 710 leader_election.cc:304] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:33.754817 1828 raft_consensus.cc:2804] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:33.754956 1828 raft_consensus.cc:697] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:33.755241 1828 consensus_queue.cc:237] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.757587 1011 tablet_service.cc:1505] Processing CreateTablet for tablet 2e29df291d8d40daba83e1d9f97f902b (DEFAULT_TABLE table=default.Table_f5ee1b1b44c645f28737720bc385a577 [id=f8a38441b37d41eab85bad3ba16b4c05]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.757906 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 06b8f8e46e524545aeab6c931efaf116
I20251211 20:52:33.758040 1011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e29df291d8d40daba83e1d9f97f902b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.758427 595 catalog_manager.cc:6724] ReplaceTablet: tablet 06b8f8e46e524545aeab6c931efaf116 deleted and replaced by tablet 7bc03ed8c35740f58fe48ddbaa120ea0
I20251211 20:52:33.758983 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 215f8dc74a71482daa2ca80e15010f51
I20251211 20:52:33.759136 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 06b8f8e46e524545aeab6c931efaf116 with delete_type TABLET_DATA_DELETED (replaced by tablet 7bc03ed8c35740f58fe48ddbaa120ea0) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:33.759541 588 master_service.cc:954] ReplaceTablet: received request to replace tablet e709051411354522bdb327c34dbf533a from {username='slave'} at 127.0.0.1:44772
W20251211 20:52:33.760015 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 06b8f8e46e524545aeab6c931efaf116 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06b8f8e46e524545aeab6c931efaf116 already in progress: creating tablet
I20251211 20:52:33.760601 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 215f8dc74a71482daa2ca80e15010f51 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:33.760636 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 06b8f8e46e524545aeab6c931efaf116 with delete_type TABLET_DATA_DELETED (replaced by tablet 7bc03ed8c35740f58fe48ddbaa120ea0) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:33.760878 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 215f8dc74a71482daa2ca80e15010f51 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:33.761224 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 215f8dc74a71482daa2ca80e15010f51 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 215f8dc74a71482daa2ca80e15010f51 already in progress: creating tablet
W20251211 20:52:33.761747 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 06b8f8e46e524545aeab6c931efaf116 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06b8f8e46e524545aeab6c931efaf116 already in progress: creating tablet
I20251211 20:52:33.761796 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 215f8dc74a71482daa2ca80e15010f51 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:33.761993 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 215f8dc74a71482daa2ca80e15010f51 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 215f8dc74a71482daa2ca80e15010f51 already in progress: creating tablet
W20251211 20:52:33.762413 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 215f8dc74a71482daa2ca80e15010f51 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 215f8dc74a71482daa2ca80e15010f51 already in progress: creating tablet
I20251211 20:52:33.762622 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 5262c2241e524395a73c0eb368917169
I20251211 20:52:33.762676 739 tablet_service.cc:1552] Processing DeleteTablet for tablet 06b8f8e46e524545aeab6c931efaf116 with delete_type TABLET_DATA_DELETED (replaced by tablet 7bc03ed8c35740f58fe48ddbaa120ea0) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:33.763398 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet a4ccf9a4ecbc4ca6978a189cf3a15893
I20251211 20:52:33.764010 596 master_service.cc:954] ReplaceTablet: received request to replace tablet 0246f1df99684d0da43d97acbf1182a0 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:33.764248 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 5262c2241e524395a73c0eb368917169 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:33.764914 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 06b8f8e46e524545aeab6c931efaf116 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06b8f8e46e524545aeab6c931efaf116 already in progress: creating tablet
W20251211 20:52:33.765012 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 5262c2241e524395a73c0eb368917169 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5262c2241e524395a73c0eb368917169 already in progress: creating tablet
I20251211 20:52:33.765539 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 5262c2241e524395a73c0eb368917169 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:33.765607 740 tablet_service.cc:1552] Processing DeleteTablet for tablet a4ccf9a4ecbc4ca6978a189cf3a15893 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:33.766054 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 5262c2241e524395a73c0eb368917169 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:33.766263 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet a4ccf9a4ecbc4ca6978a189cf3a15893 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:33.766393 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 5262c2241e524395a73c0eb368917169 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5262c2241e524395a73c0eb368917169 already in progress: creating tablet
W20251211 20:52:33.766633 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet a4ccf9a4ecbc4ca6978a189cf3a15893 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a4ccf9a4ecbc4ca6978a189cf3a15893 already in progress: creating tablet
I20251211 20:52:33.766850 891 tablet_service.cc:1552] Processing DeleteTablet for tablet a4ccf9a4ecbc4ca6978a189cf3a15893 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:33.767381 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet a4ccf9a4ecbc4ca6978a189cf3a15893 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a4ccf9a4ecbc4ca6978a189cf3a15893 already in progress: creating tablet
I20251211 20:52:33.768179 592 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 9f8aa28c1c9b4fc58dfc128952c180ba (table default.Table_bf7d402a36be44cfb6b79041906d26a1 [id=0327e686e7ed4475a3e8cda8721a3a3a]). Expected version 1 got 0
I20251211 20:52:33.768384 592 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 8feee01fc0aa4b32be926ff2b617b894 (table default.Table_265379fa17b74d0f954a2a2c4de113c8 [id=18f8d722faaa4293a112525ff0c96216]). Expected version 2 got 0
W20251211 20:52:33.769639 592 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 8feee01fc0aa4b32be926ff2b617b894 (table default.Table_265379fa17b74d0f954a2a2c4de113c8 [id=18f8d722faaa4293a112525ff0c96216], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 8feee01fc0aa4b32be926ff2b617b894 consensus state has no leader
I20251211 20:52:33.771888 1160 tablet_bootstrap.cc:492] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:33.772049 1160 ts_tablet_manager.cc:1403] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.029s user 0.000s sys 0.003s
I20251211 20:52:33.773334 1166 tablet_bootstrap.cc:492] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:33.773528 1166 ts_tablet_manager.cc:1403] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.047s user 0.004s sys 0.003s
I20251211 20:52:33.773564 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_015ac77359ed47db98e074fa602a0141"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:33.774044 597 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 2630ad57845a4c31b20c8a104dd27335 (table default.Table_4d70f35e05524680b3d72b2e488adbd5 [id=61b2de16fee74487823292913de1b685]). Expected version 3 got 0
W20251211 20:52:33.774251 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_015ac77359ed47db98e074fa602a0141 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:33.774314 1166 raft_consensus.cc:359] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.774483 1166 raft_consensus.cc:385] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.774578 1166 raft_consensus.cc:740] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.775763 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_d812d6f6c90842e4821d33226a54ec68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:33.775835 1160 raft_consensus.cc:359] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.775933 885 tablet_service.cc:1552] Processing DeleteTablet for tablet ee893e33255745debf73c486d8c70267 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:33.776093 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b89ae724b7f41c98741a41d75592da7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:33.776204 1164 tablet_replica.cc:333] T ee893e33255745debf73c486d8c70267 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:33.776286 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a206b37e43247b5babe3113eab2c9fb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:33.776311 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d812d6f6c90842e4821d33226a54ec68 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:33.776402 1164 raft_consensus.cc:2243] T ee893e33255745debf73c486d8c70267 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:33.776575 1164 raft_consensus.cc:2272] T ee893e33255745debf73c486d8c70267 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:33.777415 1164 ts_tablet_manager.cc:1916] T ee893e33255745debf73c486d8c70267 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:33.779542 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_bc3ec8261d83495488df9d7b7481de50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:33.779666 589 catalog_manager.cc:5654] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:33.780040 1160 raft_consensus.cc:385] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.780035 589 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 2630ad57845a4c31b20c8a104dd27335 (table default.Table_4d70f35e05524680b3d72b2e488adbd5 [id=61b2de16fee74487823292913de1b685]). Expected version 3 got 0
I20251211 20:52:33.780133 1160 raft_consensus.cc:740] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
W20251211 20:52:33.780149 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc3ec8261d83495488df9d7b7481de50 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:33.780476 589 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 49a33084c0fc460eb9592224b6dfc279 (table default.Table_6b98cb90147a43c4aee6b39240c5edc4 [id=9c24845e4efe475fa792a51ce8a66be6]). Expected version 4 got 0
I20251211 20:52:33.780645 589 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet dc5de12701714fcd81b9dba6973a28b5 (table default.Table_f572100667d148e8a69e7930596f0f52 [id=cb97ccf262fd4c41a180267fd0678cec]). Expected version 2 got 0
I20251211 20:52:33.780877 1160 consensus_queue.cc:260] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.781558 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_d89334f1d2b04caf9c31917b346399b5" } new_table_name: "default.Table_890c9027415a44ccaf95c0adede51992" modify_external_catalogs: true
I20251211 20:52:33.781754 1166 consensus_queue.cc:260] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.775540 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_5a933aeef8dc44b18bb0cf7ddfe412f2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:33.783239 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a933aeef8dc44b18bb0cf7ddfe412f2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:33.775521 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a4f93c6107724875b3de06f538971317" } new_table_name: "default.Table_18262e22c26d4d66ba4c6134fbde0368" modify_external_catalogs: true
I20251211 20:52:33.787089 1166 ts_tablet_manager.cc:1434] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.013s user 0.001s sys 0.001s
I20251211 20:52:33.787297 1164 ts_tablet_manager.cc:1929] T ee893e33255745debf73c486d8c70267 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:33.787431 1164 log.cc:1199] T ee893e33255745debf73c486d8c70267 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/ee893e33255745debf73c486d8c70267
I20251211 20:52:33.787817 1164 ts_tablet_manager.cc:1950] T ee893e33255745debf73c486d8c70267 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:33.788702 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet d762b4afb5ff447dac3468393826205a with delete_type TABLET_DATA_DELETED (replaced by tablet 2197822e0f11484f9679c4858626ac2c) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:33.788763 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet ee893e33255745debf73c486d8c70267 (table default.Table_da9a07e5af4a433c8e02a128e8843084 [id=3940282e8e794b21b982be1f171b677d]) successfully deleted
I20251211 20:52:33.789815 1163 tablet_replica.cc:333] T d762b4afb5ff447dac3468393826205a P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:33.790017 1163 raft_consensus.cc:2243] T d762b4afb5ff447dac3468393826205a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:33.790206 1163 raft_consensus.cc:2272] T d762b4afb5ff447dac3468393826205a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:33.790302 1160 ts_tablet_manager.cc:1434] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.018s user 0.003s sys 0.000s
W20251211 20:52:33.790423 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 5262c2241e524395a73c0eb368917169 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5262c2241e524395a73c0eb368917169 already in progress: creating tablet
I20251211 20:52:33.791158 1160 tablet_bootstrap.cc:492] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:33.791308 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet a4ccf9a4ecbc4ca6978a189cf3a15893 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a4ccf9a4ecbc4ca6978a189cf3a15893 already in progress: creating tablet
I20251211 20:52:33.791360 1163 ts_tablet_manager.cc:1916] T d762b4afb5ff447dac3468393826205a P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:33.794214 592 catalog_manager.cc:5654] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3). New cstate: current_term: 1 leader_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: HEALTHY } } }
I20251211 20:52:33.794790 592 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3eac21a07ba34e1fa275e25ee9dcabb1 (table default.Table_d571120dedcb48cd9d4b75314be3a654 [id=9700f9fe6e3044748e34954bef8a0038]). Expected version 2 got 0
I20251211 20:52:33.797930 1163 ts_tablet_manager.cc:1929] T d762b4afb5ff447dac3468393826205a P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:33.798106 1163 log.cc:1199] T d762b4afb5ff447dac3468393826205a P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/d762b4afb5ff447dac3468393826205a
I20251211 20:52:33.798475 1163 ts_tablet_manager.cc:1950] T d762b4afb5ff447dac3468393826205a P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:33.799630 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet d762b4afb5ff447dac3468393826205a (table default.Table_f6f977edac0f45bea7e94de5bfa51ee4 [id=4a74b9a60cbb4198b55cee630b2f4bef]) successfully deleted
I20251211 20:52:33.800017 1828 raft_consensus.cc:493] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:33.800957 1160 tablet_bootstrap.cc:654] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.800957 1828 raft_consensus.cc:515] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.802301 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cf1201bba4b6421f8d7ce593f4bd892b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:33.802532 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cf1201bba4b6421f8d7ce593f4bd892b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:33.802788 1044 raft_consensus.cc:2468] T cf1201bba4b6421f8d7ce593f4bd892b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
W20251211 20:52:33.802919 711 leader_election.cc:343] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:33.803269 710 leader_election.cc:304] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:33.803644 1696 raft_consensus.cc:2804] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:33.803735 1696 raft_consensus.cc:493] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:33.803831 1696 raft_consensus.cc:3060] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:33.806440 1696 raft_consensus.cc:515] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.806931 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet f91487dd7e0f4c8b8483c2b83945da3b
I20251211 20:52:33.807034 1696 leader_election.cc:290] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:33.807680 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_20456517a58b40fabd534461d2ab6cc0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:33.808292 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_20456517a58b40fabd534461d2ab6cc0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:33.808365 891 tablet_service.cc:1552] Processing DeleteTablet for tablet f91487dd7e0f4c8b8483c2b83945da3b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:33.809653 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cf1201bba4b6421f8d7ce593f4bd892b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:33.810213 711 leader_election.cc:343] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:33.810833 739 tablet_service.cc:1552] Processing DeleteTablet for tablet f91487dd7e0f4c8b8483c2b83945da3b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:33.810907 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cf1201bba4b6421f8d7ce593f4bd892b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:33.811120 1044 raft_consensus.cc:3060] T cf1201bba4b6421f8d7ce593f4bd892b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:33.811687 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet f91487dd7e0f4c8b8483c2b83945da3b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f91487dd7e0f4c8b8483c2b83945da3b already in progress: creating tablet
I20251211 20:52:33.812178 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet f91487dd7e0f4c8b8483c2b83945da3b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:33.814064 1586 raft_consensus.cc:493] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:33.814213 1586 raft_consensus.cc:515] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.814852 1586 leader_election.cc:290] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:33.814054 1044 raft_consensus.cc:2468] T cf1201bba4b6421f8d7ce593f4bd892b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:33.816284 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8feee01fc0aa4b32be926ff2b617b894" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:33.816859 983 leader_election.cc:343] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:33.817771 710 leader_election.cc:304] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:33.817793 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8feee01fc0aa4b32be926ff2b617b894" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:33.818414 982 leader_election.cc:343] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:33.818537 982 leader_election.cc:304] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:33.818868 1586 raft_consensus.cc:2749] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:33.819425 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet e709051411354522bdb327c34dbf533a
I20251211 20:52:33.819712 1696 raft_consensus.cc:2804] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:33.819964 588 catalog_manager.cc:6724] ReplaceTablet: tablet e709051411354522bdb327c34dbf533a deleted and replaced by tablet dfeb47dfaa6445309e28715b1b8db2ae
I20251211 20:52:33.819988 1696 raft_consensus.cc:697] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:33.820271 1696 consensus_queue.cc:237] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.820389 739 tablet_service.cc:1552] Processing DeleteTablet for tablet e709051411354522bdb327c34dbf533a with delete_type TABLET_DATA_DELETED (replaced by tablet dfeb47dfaa6445309e28715b1b8db2ae) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:33.821048 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet e709051411354522bdb327c34dbf533a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e709051411354522bdb327c34dbf533a already in progress: creating tablet
I20251211 20:52:33.821439 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet e709051411354522bdb327c34dbf533a with delete_type TABLET_DATA_DELETED (replaced by tablet dfeb47dfaa6445309e28715b1b8db2ae) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:33.822249 891 tablet_service.cc:1552] Processing DeleteTablet for tablet e709051411354522bdb327c34dbf533a with delete_type TABLET_DATA_DELETED (replaced by tablet dfeb47dfaa6445309e28715b1b8db2ae) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:33.823071 1828 leader_election.cc:290] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:33.824396 1164 tablet_replica.cc:333] T 8b89ae724b7f41c98741a41d75592da7 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:33.825384 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 88066d94d5d34e0fab20da9fb6ef178e with delete_type TABLET_DATA_DELETED (replaced by tablet 6a61cf8f5de14adaa4a1de5bddde9a2f) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:33.825659 745 tablet_service.cc:1552] Processing DeleteTablet for tablet bb5c5523e175406d9f6f9cd2736a2045 with delete_type TABLET_DATA_DELETED (replaced by tablet cce44b80c97948e1af7ac68e77cccfd6) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:33.825907 1165 tablet_replica.cc:333] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:33.826087 1165 raft_consensus.cc:2243] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:33.826270 1165 raft_consensus.cc:2272] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:33.826725 739 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fd2a6cf80a04f20aef8ff9c84bc88db with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:33.827773 745 tablet_service.cc:1552] Processing DeleteTablet for tablet f93006227cc6489e91f3593e11789b54 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:33.827996 739 tablet_service.cc:1552] Processing DeleteTablet for tablet d30376baf75b4761971f91da3493a142 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:33.828518 1165 ts_tablet_manager.cc:1916] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:33.828612 1586 raft_consensus.cc:493] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:33.828769 1586 raft_consensus.cc:515] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.829422 1586 leader_election.cc:290] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:33.830083 1586 raft_consensus.cc:493] T 40722aef4e194ff094d271a079bda9d9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:33.830219 1586 raft_consensus.cc:515] T 40722aef4e194ff094d271a079bda9d9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.830718 1586 leader_election.cc:290] T 40722aef4e194ff094d271a079bda9d9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:33.833009 739 tablet_service.cc:1505] Processing CreateTablet for tablet 098054406a1d459b824a094f50e71e7e (DEFAULT_TABLE table=default.Table_fbf522a147ec4a04950c849072ebb619 [id=d4e1349374a346a5b8be180f9bdc329a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.833411 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 098054406a1d459b824a094f50e71e7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.836354 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 098054406a1d459b824a094f50e71e7e (DEFAULT_TABLE table=default.Table_fbf522a147ec4a04950c849072ebb619 [id=d4e1349374a346a5b8be180f9bdc329a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.836747 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 098054406a1d459b824a094f50e71e7e. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:33.840099 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 10746162da4640e6a2a891467690b770 consensus state has no leader
I20251211 20:52:33.841212 745 tablet_service.cc:1505] Processing CreateTablet for tablet 7b9f569554fb4836ac2b62542c3ecf7e (DEFAULT_TABLE table=default.Table_78f3b1065a0a43d891bb775169c0eaca [id=6ef691749df24251a7a7c5919c219754]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.841583 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b9f569554fb4836ac2b62542c3ecf7e. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:33.841745 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 50f9f29c064c4f938a0f20f302ab5e8a consensus state has no leader
I20251211 20:52:33.843245 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_c1cf2780cc3941688af6d55d57d9e6a8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:33.843832 595 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4b43e2845ad847e39b035931723fe49f" } new_table_name: "default.Table_8747ca05a949428ba103bb1e2088f0f2" modify_external_catalogs: true
I20251211 20:52:33.844273 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 7b9f569554fb4836ac2b62542c3ecf7e (DEFAULT_TABLE table=default.Table_78f3b1065a0a43d891bb775169c0eaca [id=6ef691749df24251a7a7c5919c219754]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.844617 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b9f569554fb4836ac2b62542c3ecf7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.845113 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_3f5d0d2b042c42ed88431232445bef02"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:33.845610 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3f5d0d2b042c42ed88431232445bef02 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:33.846779 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_76916c86042b49f487d21bf7aabfed56"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:33.847087 1857 raft_consensus.cc:493] T ef4fada332cf4b5c9da26ba5f908b4c6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251211 20:52:33.847193 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_76916c86042b49f487d21bf7aabfed56 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:33.847219 1857 raft_consensus.cc:515] T ef4fada332cf4b5c9da26ba5f908b4c6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.847572 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "40722aef4e194ff094d271a079bda9d9" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:33.847875 1857 leader_election.cc:290] T ef4fada332cf4b5c9da26ba5f908b4c6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:33.848110 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6dfa7480e2ab49b9b8a938da0dc6cf0d" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:33.848359 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c1cf2780cc3941688af6d55d57d9e6a8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:33.849525 982 leader_election.cc:343] T 40722aef4e194ff094d271a079bda9d9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:33.849704 982 leader_election.cc:343] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:33.850657 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_d11ab1dc4ef24e4aa5a57ac9876f5eb9" } new_table_name: "default.Table_0ec526100c24487b808171d9734bcfb7" modify_external_catalogs: true
I20251211 20:52:33.851931 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c6fbe03584554d35ae13e59b8d8f0e70" } new_table_name: "default.Table_f3c10706c1294f978865836c1f29549f" modify_external_catalogs: true
I20251211 20:52:33.852124 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ef4fada332cf4b5c9da26ba5f908b4c6" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:33.853056 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_ddc346d7126a4b579fa1f29b6ef6cb87"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:33.853685 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ddc346d7126a4b579fa1f29b6ef6cb87 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:33.855206 590 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 3eac21a07ba34e1fa275e25ee9dcabb1 (table default.Table_d571120dedcb48cd9d4b75314be3a654 [id=9700f9fe6e3044748e34954bef8a0038]). Expected version 2 got 0
I20251211 20:52:33.790028 1166 tablet_bootstrap.cc:492] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:33.860538 1166 tablet_bootstrap.cc:654] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.861068 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 28ef13be3b84410d9c786c43502a3478 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:33.862655 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_fde35da450a941b38489c3954c1a3839" } modify_external_catalogs: true
I20251211 20:52:33.862851 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_fde35da450a941b38489c3954c1a3839" } modify_external_catalogs: true
I20251211 20:52:33.863018 1807 raft_consensus.cc:493] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:33.863159 1807 raft_consensus.cc:515] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.863914 1807 leader_election.cc:290] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:33.864629 1011 tablet_service.cc:1552] Processing DeleteTablet for tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d with delete_type TABLET_DATA_DELETED (replaced by tablet e4647105ab57454d883ebfcde7bde891) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:33.868510 982 leader_election.cc:343] T ef4fada332cf4b5c9da26ba5f908b4c6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:33.869640 710 leader_election.cc:343] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: f93006227cc6489e91f3593e11789b54
W20251211 20:52:33.875154 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet f91487dd7e0f4c8b8483c2b83945da3b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f91487dd7e0f4c8b8483c2b83945da3b already in progress: creating tablet
I20251211 20:52:33.832018 891 tablet_service.cc:1505] Processing CreateTablet for tablet 098054406a1d459b824a094f50e71e7e (DEFAULT_TABLE table=default.Table_fbf522a147ec4a04950c849072ebb619 [id=d4e1349374a346a5b8be180f9bdc329a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.833570 883 tablet_service.cc:1505] Processing CreateTablet for tablet 7b9f569554fb4836ac2b62542c3ecf7e (DEFAULT_TABLE table=default.Table_78f3b1065a0a43d891bb775169c0eaca [id=6ef691749df24251a7a7c5919c219754]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.837105 885 tablet_service.cc:1505] Processing CreateTablet for tablet 2f7b1bc3dd1f4496b3ac6e71652cf012 (DEFAULT_TABLE table=default.Table_35ba9d3f180646d8b597a9c14b01cf37 [id=dacdeefdb1ea4b529b1597557f7b2e40]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.840029 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "40722aef4e194ff094d271a079bda9d9" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:33.840418 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6dfa7480e2ab49b9b8a938da0dc6cf0d" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:33.849536 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ef4fada332cf4b5c9da26ba5f908b4c6" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:33.856781 879 tablet_service.cc:1552] Processing DeleteTablet for tablet 9adaee28edd74df68b44b25d505fdfcb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:33.857779 888 tablet_service.cc:1552] Processing DeleteTablet for tablet c4b76a7e84d344268567c551236ce772 with delete_type TABLET_DATA_DELETED (replaced by tablet 68be21a25c6743f5b86d518d98fbc35e) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:33.876633 983 leader_election.cc:343] T 40722aef4e194ff094d271a079bda9d9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:33.876750 983 leader_election.cc:304] T 40722aef4e194ff094d271a079bda9d9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:33.877032 983 leader_election.cc:343] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:33.877116 983 leader_election.cc:304] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:33.877451 1857 raft_consensus.cc:2749] T 40722aef4e194ff094d271a079bda9d9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:33.877612 1857 raft_consensus.cc:2749] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:33.878482 1165 ts_tablet_manager.cc:1929] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:33.865134 1160 tablet_bootstrap.cc:492] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:33.865625 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f93006227cc6489e91f3593e11789b54" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:33.881054 1160 ts_tablet_manager.cc:1403] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.090s user 0.009s sys 0.006s
I20251211 20:52:33.881906 1160 raft_consensus.cc:359] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.882061 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 098054406a1d459b824a094f50e71e7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.882244 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f7b1bc3dd1f4496b3ac6e71652cf012. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.885243 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b9f569554fb4836ac2b62542c3ecf7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.889566 1160 raft_consensus.cc:385] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.889708 1160 raft_consensus.cc:740] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.889997 1160 consensus_queue.cc:260] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.891667 1160 ts_tablet_manager.cc:1434] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.010s user 0.002s sys 0.001s
I20251211 20:52:33.891978 1160 tablet_bootstrap.cc:492] T 5fd2a6cf80a04f20aef8ff9c84bc88db P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:33.880810 711 leader_election.cc:343] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:33.894256 711 leader_election.cc:304] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:33.894876 1165 log.cc:1199] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/88066d94d5d34e0fab20da9fb6ef178e
I20251211 20:52:33.895367 1165 ts_tablet_manager.cc:1950] T 88066d94d5d34e0fab20da9fb6ef178e P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:33.896624 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 88066d94d5d34e0fab20da9fb6ef178e (table default.Table_b06f9587f78645bb9712c3b080f3c0d4 [id=ea12bdaa894f446da22635f95d97ac2a]) successfully deleted
I20251211 20:52:33.898983 745 tablet_service.cc:1505] Processing CreateTablet for tablet 2f7b1bc3dd1f4496b3ac6e71652cf012 (DEFAULT_TABLE table=default.Table_35ba9d3f180646d8b597a9c14b01cf37 [id=dacdeefdb1ea4b529b1597557f7b2e40]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.899631 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f7b1bc3dd1f4496b3ac6e71652cf012. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.904892 1168 tablet_bootstrap.cc:492] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:33.905087 1168 ts_tablet_manager.cc:1403] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.240s user 0.038s sys 0.031s
I20251211 20:52:33.905920 891 tablet_service.cc:1505] Processing CreateTablet for tablet 042f8bf13a7a4712b5bdc04671cd9938 (DEFAULT_TABLE table=default.Table_6829fee154a74021a1410d7535c0c89f [id=aea8a10b336548f78e1c093ce01a22a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.905948 1168 raft_consensus.cc:359] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.906141 1168 raft_consensus.cc:385] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.906311 1168 raft_consensus.cc:740] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.906392 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 88066d94d5d34e0fab20da9fb6ef178e with delete_type TABLET_DATA_DELETED (replaced by tablet 6a61cf8f5de14adaa4a1de5bddde9a2f) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:33.906556 1168 consensus_queue.cc:260] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.906656 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 042f8bf13a7a4712b5bdc04671cd9938. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.906786 1160 tablet_bootstrap.cc:654] T 5fd2a6cf80a04f20aef8ff9c84bc88db P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.906976 1828 raft_consensus.cc:2749] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:33.907104 1168 ts_tablet_manager.cc:1434] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:33.907418 1168 tablet_bootstrap.cc:492] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:33.909073 1168 tablet_bootstrap.cc:654] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.909505 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 2f7b1bc3dd1f4496b3ac6e71652cf012 (DEFAULT_TABLE table=default.Table_35ba9d3f180646d8b597a9c14b01cf37 [id=dacdeefdb1ea4b529b1597557f7b2e40]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.909938 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f7b1bc3dd1f4496b3ac6e71652cf012. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.911298 1164 raft_consensus.cc:2243] T 8b89ae724b7f41c98741a41d75592da7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
W20251211 20:52:33.912917 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet e709051411354522bdb327c34dbf533a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e709051411354522bdb327c34dbf533a already in progress: creating tablet
I20251211 20:52:33.913079 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 822abda3c85848728205b71847ef7029 with delete_type TABLET_DATA_DELETED (replaced by tablet f9a3fdf17f274c15ac0bdfa9c174f649) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:33.913381 1164 raft_consensus.cc:2272] T 8b89ae724b7f41c98741a41d75592da7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:33.915763 739 tablet_service.cc:1505] Processing CreateTablet for tablet 042f8bf13a7a4712b5bdc04671cd9938 (DEFAULT_TABLE table=default.Table_6829fee154a74021a1410d7535c0c89f [id=aea8a10b336548f78e1c093ce01a22a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.915844 885 tablet_service.cc:1505] Processing CreateTablet for tablet 4cf0cde801bf44449e539d2799b895a7 (DEFAULT_TABLE table=default.Table_f06fbcca762d4b93b6dd39878d6d58fb [id=321235280d1d4fa0bae5cc0333860f85]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.916296 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 042f8bf13a7a4712b5bdc04671cd9938. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.916541 1013 tablet_service.cc:1505] Processing CreateTablet for tablet 042f8bf13a7a4712b5bdc04671cd9938 (DEFAULT_TABLE table=default.Table_6829fee154a74021a1410d7535c0c89f [id=aea8a10b336548f78e1c093ce01a22a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.917150 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 042f8bf13a7a4712b5bdc04671cd9938. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.917567 1160 tablet_bootstrap.cc:492] T 5fd2a6cf80a04f20aef8ff9c84bc88db P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:33.917642 1163 tablet_replica.cc:333] T 28ef13be3b84410d9c786c43502a3478 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:33.917739 1160 ts_tablet_manager.cc:1403] T 5fd2a6cf80a04f20aef8ff9c84bc88db P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.001s
I20251211 20:52:33.917936 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cf0cde801bf44449e539d2799b895a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.918664 1160 raft_consensus.cc:359] T 5fd2a6cf80a04f20aef8ff9c84bc88db P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.918879 1160 raft_consensus.cc:385] T 5fd2a6cf80a04f20aef8ff9c84bc88db P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.918967 1160 raft_consensus.cc:740] T 5fd2a6cf80a04f20aef8ff9c84bc88db P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.919435 1164 ts_tablet_manager.cc:1916] T 8b89ae724b7f41c98741a41d75592da7 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:33.923296 590 catalog_manager.cc:5654] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:33.923911 590 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet dc5de12701714fcd81b9dba6973a28b5 (table default.Table_f572100667d148e8a69e7930596f0f52 [id=cb97ccf262fd4c41a180267fd0678cec]). Expected version 2 got 0
I20251211 20:52:33.924278 590 catalog_manager.cc:5654] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:33.925839 1696 raft_consensus.cc:493] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:33.926038 1696 raft_consensus.cc:515] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.926641 1163 raft_consensus.cc:2243] T 28ef13be3b84410d9c786c43502a3478 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:33.926920 1163 raft_consensus.cc:2272] T 28ef13be3b84410d9c786c43502a3478 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:33.927685 1163 ts_tablet_manager.cc:1916] T 28ef13be3b84410d9c786c43502a3478 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:33.927882 1696 leader_election.cc:290] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:33.926012 1165 tablet_replica.cc:333] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:33.928311 1165 raft_consensus.cc:2243] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:33.928521 1165 raft_consensus.cc:2272] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:33.928478 710 leader_election.cc:343] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: bb5c5523e175406d9f6f9cd2736a2045
I20251211 20:52:33.929383 1165 ts_tablet_manager.cc:1916] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:33.933547 741 tablet_service.cc:1505] Processing CreateTablet for tablet 4cf0cde801bf44449e539d2799b895a7 (DEFAULT_TABLE table=default.Table_f06fbcca762d4b93b6dd39878d6d58fb [id=321235280d1d4fa0bae5cc0333860f85]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.933980 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cf0cde801bf44449e539d2799b895a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.940280 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 4cf0cde801bf44449e539d2799b895a7 (DEFAULT_TABLE table=default.Table_f06fbcca762d4b93b6dd39878d6d58fb [id=321235280d1d4fa0bae5cc0333860f85]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:33.940851 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cf0cde801bf44449e539d2799b895a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:33.944280 1168 tablet_bootstrap.cc:492] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:33.944450 1168 ts_tablet_manager.cc:1403] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.037s user 0.005s sys 0.009s
I20251211 20:52:33.945241 1168 raft_consensus.cc:359] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.945451 1168 raft_consensus.cc:385] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.945547 1168 raft_consensus.cc:740] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.945793 1168 consensus_queue.cc:260] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:33.947240 983 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 70204.8us
W20251211 20:52:33.947571 983 leader_election.cc:343] T ef4fada332cf4b5c9da26ba5f908b4c6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:33.947746 983 leader_election.cc:304] T ef4fada332cf4b5c9da26ba5f908b4c6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:33.949170 1870 raft_consensus.cc:2749] T ef4fada332cf4b5c9da26ba5f908b4c6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:33.927290 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb5c5523e175406d9f6f9cd2736a2045" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:33.929243 1160 consensus_queue.cc:260] T 5fd2a6cf80a04f20aef8ff9c84bc88db P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:33.950218 711 leader_election.cc:343] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:33.950358 711 leader_election.cc:304] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:33.950533 711 raft_consensus.cc:2686] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251211 20:52:33.950779 1160 ts_tablet_manager.cc:1434] T 5fd2a6cf80a04f20aef8ff9c84bc88db P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.033s user 0.001s sys 0.001s
I20251211 20:52:33.951344 1160 tablet_bootstrap.cc:492] T dc5de12701714fcd81b9dba6973a28b5 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:33.953227 1160 tablet_bootstrap.cc:654] T dc5de12701714fcd81b9dba6973a28b5 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.960031 1168 ts_tablet_manager.cc:1434] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.015s user 0.001s sys 0.001s
I20251211 20:52:33.969107 1813 raft_consensus.cc:493] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:33.969300 1813 raft_consensus.cc:515] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.970018 1813 leader_election.cc:290] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:33.972291 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_181fcc88844b493a8e4ce62302564583" } modify_external_catalogs: true
I20251211 20:52:33.972463 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fd2a6cf80a04f20aef8ff9c84bc88db with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:33.972496 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_181fcc88844b493a8e4ce62302564583" } modify_external_catalogs: true
I20251211 20:52:33.975030 1696 raft_consensus.cc:493] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:33.975203 1696 raft_consensus.cc:515] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.975845 1696 leader_election.cc:290] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:33.977139 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "605172f066d242f2abea133c08a145ca" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:33.977667 847 leader_election.cc:343] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 88066d94d5d34e0fab20da9fb6ef178e
W20251211 20:52:33.977895 847 leader_election.cc:343] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 88066d94d5d34e0fab20da9fb6ef178e
I20251211 20:52:33.978008 847 leader_election.cc:304] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
W20251211 20:52:33.978973 710 leader_election.cc:343] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 605172f066d242f2abea133c08a145ca
W20251211 20:52:33.979449 711 leader_election.cc:343] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:33.979598 711 leader_election.cc:304] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:33.980638 1696 raft_consensus.cc:2749] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:33.980931 1813 raft_consensus.cc:2749] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:33.981884 1168 tablet_bootstrap.cc:492] T 804a11c8f7414eb1a6bcff2952be9ae2 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:33.987514 1165 ts_tablet_manager.cc:1929] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:33.988191 1168 tablet_bootstrap.cc:654] T 804a11c8f7414eb1a6bcff2952be9ae2 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:33.988333 1165 log.cc:1199] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/bb5c5523e175406d9f6f9cd2736a2045
I20251211 20:52:33.989032 1165 ts_tablet_manager.cc:1950] T bb5c5523e175406d9f6f9cd2736a2045 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:33.990720 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet bb5c5523e175406d9f6f9cd2736a2045 (table default.Table_32ebd46bd3cc460986098c79c9b67ca7 [id=1bd6b7930f2447659da365b7c06b1640]) successfully deleted
I20251211 20:52:33.990794 1165 tablet_replica.cc:333] T 5fd2a6cf80a04f20aef8ff9c84bc88db P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:33.991859 1165 raft_consensus.cc:2243] T 5fd2a6cf80a04f20aef8ff9c84bc88db P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:33.992130 1165 raft_consensus.cc:2272] T 5fd2a6cf80a04f20aef8ff9c84bc88db P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:33.992383 1166 tablet_bootstrap.cc:492] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:33.992535 1166 ts_tablet_manager.cc:1403] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.203s user 0.025s sys 0.041s
I20251211 20:52:33.993006 1165 ts_tablet_manager.cc:1916] T 5fd2a6cf80a04f20aef8ff9c84bc88db P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:33.993311 1166 raft_consensus.cc:359] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.993500 1166 raft_consensus.cc:385] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:33.993577 1166 raft_consensus.cc:740] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:33.993785 1166 consensus_queue.cc:260] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.994251 1166 ts_tablet_manager.cc:1434] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:33.995146 1166 tablet_bootstrap.cc:492] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:33.995872 595 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 48820386e41b47129a13bdd5768f021b consensus state has no leader
I20251211 20:52:33.997750 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_95bf875841aa45c8bf54c51b5839eb51"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:33.998164 1163 ts_tablet_manager.cc:1929] T 28ef13be3b84410d9c786c43502a3478 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:33.998171 1813 raft_consensus.cc:493] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:33.998260 1163 log.cc:1199] T 28ef13be3b84410d9c786c43502a3478 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/28ef13be3b84410d9c786c43502a3478
W20251211 20:52:33.998358 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_95bf875841aa45c8bf54c51b5839eb51 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:33.998312 1813 raft_consensus.cc:515] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:33.998526 1163 ts_tablet_manager.cc:1950] T 28ef13be3b84410d9c786c43502a3478 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:33.998982 1813 leader_election.cc:290] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:34.000995 1165 ts_tablet_manager.cc:1929] T 5fd2a6cf80a04f20aef8ff9c84bc88db P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.001111 1165 log.cc:1199] T 5fd2a6cf80a04f20aef8ff9c84bc88db P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/5fd2a6cf80a04f20aef8ff9c84bc88db
I20251211 20:52:34.001482 1165 ts_tablet_manager.cc:1950] T 5fd2a6cf80a04f20aef8ff9c84bc88db P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:34.001520 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_43474d2631e0443d9eaecae68a1c110d" } modify_external_catalogs: true
I20251211 20:52:34.001705 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_43474d2631e0443d9eaecae68a1c110d" } modify_external_catalogs: true
I20251211 20:52:34.002523 1165 tablet_replica.cc:333] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:34.003257 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_162bb61ae9c74b42854b233dab7191ee" } modify_external_catalogs: true
I20251211 20:52:34.003969 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_162bb61ae9c74b42854b233dab7191ee" } modify_external_catalogs: true
I20251211 20:52:34.002724 1165 raft_consensus.cc:2243] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.004295 1165 raft_consensus.cc:2272] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.005290 1166 tablet_bootstrap.cc:654] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.006006 1165 ts_tablet_manager.cc:1916] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:34.007009 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6029c1fd563d4023a8cf7cfb8ffa0876 consensus state has no leader
W20251211 20:52:34.007378 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 33fd09ea06ee47908b85911e4ee79708 has no consensus state
I20251211 20:52:34.007779 1813 raft_consensus.cc:493] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:34.007973 1813 raft_consensus.cc:515] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.008610 1813 leader_election.cc:290] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:34.009029 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet cc43ee61e78249bfa00b11f3a1fa21a3
I20251211 20:52:34.010071 891 tablet_service.cc:1552] Processing DeleteTablet for tablet cc43ee61e78249bfa00b11f3a1fa21a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:34.010262 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_2fddb279b6164c2ba9e81fc55dfeb59a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:34.010818 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2fddb279b6164c2ba9e81fc55dfeb59a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.013047 1165 ts_tablet_manager.cc:1929] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.013257 1165 log.cc:1199] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/f93006227cc6489e91f3593e11789b54
I20251211 20:52:34.013415 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2630ad57845a4c31b20c8a104dd27335" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:34.013653 1046 raft_consensus.cc:2393] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb0f1933d9a74fccaa4618047fc05c41 in current term 1: Already voted for candidate 6f8d784fd1cd492bbe3809403ec34919 in this term.
I20251211 20:52:34.013710 1165 ts_tablet_manager.cc:1950] T f93006227cc6489e91f3593e11789b54 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:34.014727 1163 tablet_replica.cc:333] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:34.014834 1165 tablet_replica.cc:333] T d30376baf75b4761971f91da3493a142 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:34.015102 1165 raft_consensus.cc:2243] T d30376baf75b4761971f91da3493a142 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.015336 1165 raft_consensus.cc:2272] T d30376baf75b4761971f91da3493a142 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.015358 1163 raft_consensus.cc:2243] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.015321 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_06bc050f989a4d97a4f76d09b835932d" } modify_external_catalogs: true
I20251211 20:52:34.015512 1163 raft_consensus.cc:2272] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.016099 1165 ts_tablet_manager.cc:1916] T d30376baf75b4761971f91da3493a142 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.015501 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_06bc050f989a4d97a4f76d09b835932d" } modify_external_catalogs: true
I20251211 20:52:34.017583 1163 ts_tablet_manager.cc:1916] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.018896 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2630ad57845a4c31b20c8a104dd27335" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:34.019867 593 master_service.cc:954] ReplaceTablet: received request to replace tablet c74427cb9d6f4a679e43eae796ff8c3f from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:34.020893 1160 tablet_bootstrap.cc:492] T dc5de12701714fcd81b9dba6973a28b5 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:34.021044 1160 ts_tablet_manager.cc:1403] T dc5de12701714fcd81b9dba6973a28b5 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.070s user 0.007s sys 0.004s
I20251211 20:52:34.021090 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_6b058d9ed449475e986f56f95463634a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:34.021638 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6b058d9ed449475e986f56f95463634a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.023039 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_a5b944a25f4c45cf8c70765ea43d1ac2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:34.023921 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a5b944a25f4c45cf8c70765ea43d1ac2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.027695 1696 raft_consensus.cc:493] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:34.027868 1696 raft_consensus.cc:515] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.028489 1696 leader_election.cc:290] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:34.029074 1870 raft_consensus.cc:493] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:34.029225 1870 raft_consensus.cc:515] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:34.029656 710 leader_election.cc:343] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: dcdcff8235814b58b3683104a2ff0cbf
I20251211 20:52:34.029768 1870 leader_election.cc:290] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:34.030514 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "340730a17a834a77bbc20b95aba98e2c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:34.031193 983 leader_election.cc:343] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:34.031931 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "340730a17a834a77bbc20b95aba98e2c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:34.032392 982 leader_election.cc:343] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:34.032495 982 leader_election.cc:304] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:34.032549 1160 raft_consensus.cc:359] T dc5de12701714fcd81b9dba6973a28b5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.032755 1160 raft_consensus.cc:385] T dc5de12701714fcd81b9dba6973a28b5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.032845 1160 raft_consensus.cc:740] T dc5de12701714fcd81b9dba6973a28b5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.032944 1870 raft_consensus.cc:2749] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:34.033053 1160 consensus_queue.cc:260] T dc5de12701714fcd81b9dba6973a28b5 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.032579 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dcdcff8235814b58b3683104a2ff0cbf" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:34.033741 1160 ts_tablet_manager.cc:1434] T dc5de12701714fcd81b9dba6973a28b5 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.013s user 0.001s sys 0.002s
W20251211 20:52:34.034710 711 leader_election.cc:343] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:34.034829 1160 tablet_bootstrap.cc:492] T c910246b3d5d4e7e8a71d9f18e94d24c P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:34.034837 711 leader_election.cc:304] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:34.036475 1696 raft_consensus.cc:2749] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:34.030524 1164 ts_tablet_manager.cc:1929] T 8b89ae724b7f41c98741a41d75592da7 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.036722 1164 log.cc:1199] T 8b89ae724b7f41c98741a41d75592da7 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/8b89ae724b7f41c98741a41d75592da7
I20251211 20:52:34.038211 1165 ts_tablet_manager.cc:1929] T d30376baf75b4761971f91da3493a142 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.040134 1164 ts_tablet_manager.cc:1950] T 8b89ae724b7f41c98741a41d75592da7 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:34.040757 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 340730a17a834a77bbc20b95aba98e2c with delete_type TABLET_DATA_DELETED (replaced by tablet 3b2c6064c5eb4c0f8979c485f5d85a75) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.041491 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 8b89ae724b7f41c98741a41d75592da7 (table default.Table_c607094f0e4d4af48d2e54d4d728ae13 [id=9dae0e60f978448a840aa497da35966a]) successfully deleted
I20251211 20:52:34.041594 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 5fd2a6cf80a04f20aef8ff9c84bc88db (table default.Table_370d3069ceee4b8e8856bd74fe3a1815 [id=10fd0368198c447c95b692b1f3e6a023]) successfully deleted
I20251211 20:52:34.041792 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet f93006227cc6489e91f3593e11789b54 (table default.Table_4cd9b395cfa94302b298a3df4971d631 [id=42b8ff20f93f4d3fa10fe7ff92dc598d]) successfully deleted
I20251211 20:52:34.042073 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 28ef13be3b84410d9c786c43502a3478 (table default.Table_88dcd36b4447481487bc918a05511862 [id=d9453cea23a8430c98a3317856b3cec3]) successfully deleted
W20251211 20:52:34.043977 847 leader_election.cc:343] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: c4b76a7e84d344268567c551236ce772
I20251211 20:52:34.044343 847 leader_election.cc:304] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:34.044711 1813 raft_consensus.cc:3060] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:34.047312 1165 log.cc:1199] T d30376baf75b4761971f91da3493a142 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/d30376baf75b4761971f91da3493a142
I20251211 20:52:34.047870 1165 ts_tablet_manager.cc:1950] T d30376baf75b4761971f91da3493a142 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:34.048270 1160 tablet_bootstrap.cc:654] T c910246b3d5d4e7e8a71d9f18e94d24c P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.043139 739 tablet_service.cc:1552] Processing DeleteTablet for tablet cc43ee61e78249bfa00b11f3a1fa21a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:34.048424 847 leader_election.cc:343] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: c4b76a7e84d344268567c551236ce772
I20251211 20:52:34.048539 1813 raft_consensus.cc:2749] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:34.048687 847 leader_election.cc:304] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:34.049144 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet d30376baf75b4761971f91da3493a142 (table default.Table_cec7f350a9d04bd590db8a29fb8ceb6b [id=62d2078174bb4b5f952896c6ada6a414]) successfully deleted
I20251211 20:52:34.049196 1813 raft_consensus.cc:2749] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:34.049976 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet cc43ee61e78249bfa00b11f3a1fa21a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cc43ee61e78249bfa00b11f3a1fa21a3 already in progress: creating tablet
I20251211 20:52:34.050529 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet cc43ee61e78249bfa00b11f3a1fa21a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.050622 1160 tablet_bootstrap.cc:492] T c910246b3d5d4e7e8a71d9f18e94d24c P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:34.050792 1160 ts_tablet_manager.cc:1403] T c910246b3d5d4e7e8a71d9f18e94d24c P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.000s
I20251211 20:52:34.051278 1164 tablet_replica.cc:333] T 6a206b37e43247b5babe3113eab2c9fb P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:34.051527 1164 raft_consensus.cc:2243] T 6a206b37e43247b5babe3113eab2c9fb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.051617 1160 raft_consensus.cc:359] T c910246b3d5d4e7e8a71d9f18e94d24c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.051801 1160 raft_consensus.cc:385] T c910246b3d5d4e7e8a71d9f18e94d24c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.051853 1164 raft_consensus.cc:2272] T 6a206b37e43247b5babe3113eab2c9fb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.052461 1164 ts_tablet_manager.cc:1916] T 6a206b37e43247b5babe3113eab2c9fb P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.054769 1870 raft_consensus.cc:493] T 9f8aa28c1c9b4fc58dfc128952c180ba P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:34.054920 1870 raft_consensus.cc:515] T 9f8aa28c1c9b4fc58dfc128952c180ba P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.055539 1870 leader_election.cc:290] T 9f8aa28c1c9b4fc58dfc128952c180ba P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:34.056355 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f8aa28c1c9b4fc58dfc128952c180ba" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:34.056895 982 leader_election.cc:343] T 9f8aa28c1c9b4fc58dfc128952c180ba P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:34.057310 1163 ts_tablet_manager.cc:1929] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.057049 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f8aa28c1c9b4fc58dfc128952c180ba" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:34.051900 1160 raft_consensus.cc:740] T c910246b3d5d4e7e8a71d9f18e94d24c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
W20251211 20:52:34.062794 983 leader_election.cc:343] T 9f8aa28c1c9b4fc58dfc128952c180ba P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:34.063720 983 leader_election.cc:304] T 9f8aa28c1c9b4fc58dfc128952c180ba P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:34.064277 1586 raft_consensus.cc:2749] T 9f8aa28c1c9b4fc58dfc128952c180ba P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:34.064550 1163 log.cc:1199] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/6dfa7480e2ab49b9b8a938da0dc6cf0d
I20251211 20:52:34.065095 1163 ts_tablet_manager.cc:1950] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:34.066823 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d (table default.Table_c817dec9a7674f699b3366fc302ead57 [id=d555d1fc6aef4a21b4d3f2804264d030]) successfully deleted
I20251211 20:52:34.067533 1163 tablet_replica.cc:333] T 822abda3c85848728205b71847ef7029 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:34.067993 1163 raft_consensus.cc:2243] T 822abda3c85848728205b71847ef7029 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.070492 1163 raft_consensus.cc:2272] T 822abda3c85848728205b71847ef7029 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.071707 1163 ts_tablet_manager.cc:1916] T 822abda3c85848728205b71847ef7029 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.076040 1168 tablet_bootstrap.cc:492] T 804a11c8f7414eb1a6bcff2952be9ae2 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:34.076243 1168 ts_tablet_manager.cc:1403] T 804a11c8f7414eb1a6bcff2952be9ae2 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.094s user 0.001s sys 0.003s
I20251211 20:52:34.077132 1168 raft_consensus.cc:359] T 804a11c8f7414eb1a6bcff2952be9ae2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.077366 1168 raft_consensus.cc:385] T 804a11c8f7414eb1a6bcff2952be9ae2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.077445 1168 raft_consensus.cc:740] T 804a11c8f7414eb1a6bcff2952be9ae2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.077731 1168 consensus_queue.cc:260] T 804a11c8f7414eb1a6bcff2952be9ae2 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.078593 1828 raft_consensus.cc:493] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:34.078759 1828 raft_consensus.cc:515] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.079376 1828 leader_election.cc:290] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:34.079533 1828 raft_consensus.cc:493] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:34.079702 1828 raft_consensus.cc:515] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.080243 1828 leader_election.cc:290] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:34.082113 710 leader_election.cc:343] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: c85624ad640041c29bb87416e1992a9e
I20251211 20:52:34.083106 1168 ts_tablet_manager.cc:1434] T 804a11c8f7414eb1a6bcff2952be9ae2 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20251211 20:52:34.091217 590 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet dc5de12701714fcd81b9dba6973a28b5 (table default.Table_f572100667d148e8a69e7930596f0f52 [id=cb97ccf262fd4c41a180267fd0678cec]). Expected version 2 got 0
I20251211 20:52:34.091817 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet f2ae084982f44fdf8bfecdd2bc4c01b9
I20251211 20:52:34.092432 594 master_service.cc:954] ReplaceTablet: received request to replace tablet be876471f8bf453aa4467c91891cbbde from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:34.094413 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet aeff1fbabf7845dc9863f59f286ddcec
I20251211 20:52:34.095157 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e7b2a8479479460fa5a4f3c23c6eeee6" } new_table_name: "default.Table_68b9d0d7258a4b5caa1779a4769bdd7f" modify_external_catalogs: true
I20251211 20:52:34.097115 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_842a98db481041b98480204da25a6419"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:34.097712 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_842a98db481041b98480204da25a6419 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.080492 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "49a33084c0fc460eb9592224b6dfc279" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:34.098762 1044 raft_consensus.cc:2468] T 49a33084c0fc460eb9592224b6dfc279 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:34.099705 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 3dde7ed495e64909bd9784dc5eefb04c
I20251211 20:52:34.100400 591 master_service.cc:954] ReplaceTablet: received request to replace tablet b529b38d754f46ba8cb60b19e9705256 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:34.102065 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ee325d5e9bb44666b4065c86f5d298e1" } new_table_name: "default.Table_3fda0e3909b04d63817a6851c0354909" modify_external_catalogs: true
I20251211 20:52:34.103368 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 58e35de0792a464a818170886ca8d8c9
I20251211 20:52:34.108381 1011 tablet_service.cc:1552] Processing DeleteTablet for tablet 58e35de0792a464a818170886ca8d8c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.109185 891 tablet_service.cc:1552] Processing DeleteTablet for tablet f2ae084982f44fdf8bfecdd2bc4c01b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:34.109301 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet f2ae084982f44fdf8bfecdd2bc4c01b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.110047 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet aeff1fbabf7845dc9863f59f286ddcec with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.110231 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 3dde7ed495e64909bd9784dc5eefb04c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.110431 885 tablet_service.cc:1552] Processing DeleteTablet for tablet aeff1fbabf7845dc9863f59f286ddcec with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:34.110977 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet c74427cb9d6f4a679e43eae796ff8c3f
I20251211 20:52:34.111469 593 catalog_manager.cc:6724] ReplaceTablet: tablet c74427cb9d6f4a679e43eae796ff8c3f deleted and replaced by tablet 4e123b96b7534f07ae48a808f47e21a1
I20251211 20:52:34.112520 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet c74427cb9d6f4a679e43eae796ff8c3f with delete_type TABLET_DATA_DELETED (replaced by tablet 4e123b96b7534f07ae48a808f47e21a1) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.118995 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c85624ad640041c29bb87416e1992a9e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:34.119354 593 master_service.cc:954] ReplaceTablet: received request to replace tablet d3b383ace72748d4bf21678894815c4f from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:34.118927 1160 consensus_queue.cc:260] T c910246b3d5d4e7e8a71d9f18e94d24c P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.119975 883 tablet_service.cc:1552] Processing DeleteTablet for tablet c74427cb9d6f4a679e43eae796ff8c3f with delete_type TABLET_DATA_DELETED (replaced by tablet 4e123b96b7534f07ae48a808f47e21a1) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:34.120245 1160 ts_tablet_manager.cc:1434] T c910246b3d5d4e7e8a71d9f18e94d24c P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.069s user 0.000s sys 0.002s
I20251211 20:52:34.120355 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 58e35de0792a464a818170886ca8d8c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:34.120726 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "49a33084c0fc460eb9592224b6dfc279" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:34.128165 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 3dde7ed495e64909bd9784dc5eefb04c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:34.130177 885 tablet_service.cc:1552] Processing DeleteTablet for tablet c910246b3d5d4e7e8a71d9f18e94d24c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:34.130157 1160 tablet_bootstrap.cc:492] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:34.131420 1163 ts_tablet_manager.cc:1929] T 822abda3c85848728205b71847ef7029 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.135828 1163 log.cc:1199] T 822abda3c85848728205b71847ef7029 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/822abda3c85848728205b71847ef7029
I20251211 20:52:34.136543 1163 ts_tablet_manager.cc:1950] T 822abda3c85848728205b71847ef7029 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:34.138677 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 822abda3c85848728205b71847ef7029 (table default.Table_4aad4cf01eff415f86b8446c29977007 [id=01282adb7b834261a7750c350ad0b250]) successfully deleted
I20251211 20:52:34.139811 1160 tablet_bootstrap.cc:654] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.140758 1164 ts_tablet_manager.cc:1929] T 6a206b37e43247b5babe3113eab2c9fb P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.140914 1164 log.cc:1199] T 6a206b37e43247b5babe3113eab2c9fb P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/6a206b37e43247b5babe3113eab2c9fb
I20251211 20:52:34.141266 1163 tablet_replica.cc:333] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:34.141412 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet be876471f8bf453aa4467c91891cbbde
I20251211 20:52:34.141750 1163 raft_consensus.cc:2243] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.141850 1164 ts_tablet_manager.cc:1950] T 6a206b37e43247b5babe3113eab2c9fb P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:34.141968 594 catalog_manager.cc:6724] ReplaceTablet: tablet be876471f8bf453aa4467c91891cbbde deleted and replaced by tablet 7d4e63f54d8443ae8322d4659a165e76
I20251211 20:52:34.142282 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet be876471f8bf453aa4467c91891cbbde with delete_type TABLET_DATA_DELETED (replaced by tablet 7d4e63f54d8443ae8322d4659a165e76) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.142267 885 tablet_service.cc:1552] Processing DeleteTablet for tablet be876471f8bf453aa4467c91891cbbde with delete_type TABLET_DATA_DELETED (replaced by tablet 7d4e63f54d8443ae8322d4659a165e76) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:34.142638 1164 tablet_replica.cc:333] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:34.143167 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 6a206b37e43247b5babe3113eab2c9fb (table default.Table_028df988934346debc0f8c62b4740309 [id=e1949419f3924474bf195ac17a794497]) successfully deleted
I20251211 20:52:34.143343 1163 raft_consensus.cc:2272] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:34.143414 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e3f711f02af843adb4f5c3c750924e33 consensus state has no leader
W20251211 20:52:34.143446 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet f91487dd7e0f4c8b8483c2b83945da3b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f91487dd7e0f4c8b8483c2b83945da3b already in progress: creating tablet
W20251211 20:52:34.144194 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet e709051411354522bdb327c34dbf533a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e709051411354522bdb327c34dbf533a already in progress: creating tablet
I20251211 20:52:34.144425 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet b529b38d754f46ba8cb60b19e9705256
I20251211 20:52:34.144902 591 catalog_manager.cc:6724] ReplaceTablet: tablet b529b38d754f46ba8cb60b19e9705256 deleted and replaced by tablet 84bb3583857848459e0e865283de623d
I20251211 20:52:34.145254 885 tablet_service.cc:1552] Processing DeleteTablet for tablet b529b38d754f46ba8cb60b19e9705256 with delete_type TABLET_DATA_DELETED (replaced by tablet 84bb3583857848459e0e865283de623d) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:34.145558 1163 ts_tablet_manager.cc:1916] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.146824 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet b529b38d754f46ba8cb60b19e9705256 with delete_type TABLET_DATA_DELETED (replaced by tablet 84bb3583857848459e0e865283de623d) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:34.144198 592 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e95b0d8c507a4779a087c8933d464228 consensus state has no leader
I20251211 20:52:34.147389 1813 raft_consensus.cc:493] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:34.147555 1813 raft_consensus.cc:515] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.149258 1813 leader_election.cc:290] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:34.150338 847 leader_election.cc:343] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 9adaee28edd74df68b44b25d505fdfcb
I20251211 20:52:34.102630 739 tablet_service.cc:1552] Processing DeleteTablet for tablet dcdcff8235814b58b3683104a2ff0cbf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:34.103027 745 tablet_service.cc:1552] Processing DeleteTablet for tablet f2ae084982f44fdf8bfecdd2bc4c01b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:34.151645 1165 tablet_replica.cc:333] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:34.151870 1165 raft_consensus.cc:2243] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.152077 1165 raft_consensus.cc:2272] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.153055 1165 ts_tablet_manager.cc:1916] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.153545 1168 tablet_bootstrap.cc:492] T d5f0ba3ab16240f9822be69f941f5da2 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:34.155264 1168 tablet_bootstrap.cc:654] T d5f0ba3ab16240f9822be69f941f5da2 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.109086 710 leader_election.cc:304] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
W20251211 20:52:34.158043 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 48997.9us
I20251211 20:52:34.122776 741 tablet_service.cc:1552] Processing DeleteTablet for tablet aeff1fbabf7845dc9863f59f286ddcec with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:34.123054 711 leader_election.cc:343] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:34.158326 711 leader_election.cc:304] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:34.159622 711 leader_election.cc:343] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:34.123672 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 3dde7ed495e64909bd9784dc5eefb04c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:34.160058 1696 raft_consensus.cc:2804] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:34.160189 1696 raft_consensus.cc:493] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:34.160271 1696 raft_consensus.cc:3060] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:34.161829 1886 raft_consensus.cc:2749] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:34.124186 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 58e35de0792a464a818170886ca8d8c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:34.126618 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 605172f066d242f2abea133c08a145ca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:34.127710 757 tablet_service.cc:1552] Processing DeleteTablet for tablet c85624ad640041c29bb87416e1992a9e with delete_type TABLET_DATA_DELETED (replaced by tablet fa79b67b9eee45e2b41c4a90967606cc) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:34.128008 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 804a11c8f7414eb1a6bcff2952be9ae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:34.163236 1696 raft_consensus.cc:515] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.128902 754 tablet_service.cc:1552] Processing DeleteTablet for tablet c74427cb9d6f4a679e43eae796ff8c3f with delete_type TABLET_DATA_DELETED (replaced by tablet 4e123b96b7534f07ae48a808f47e21a1) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:34.142751 742 tablet_service.cc:1552] Processing DeleteTablet for tablet be876471f8bf453aa4467c91891cbbde with delete_type TABLET_DATA_DELETED (replaced by tablet 7d4e63f54d8443ae8322d4659a165e76) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:34.146756 751 tablet_service.cc:1552] Processing DeleteTablet for tablet b529b38d754f46ba8cb60b19e9705256 with delete_type TABLET_DATA_DELETED (replaced by tablet 84bb3583857848459e0e865283de623d) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:34.164508 1696 leader_election.cc:290] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:34.165006 847 leader_election.cc:343] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 9adaee28edd74df68b44b25d505fdfcb
I20251211 20:52:34.165169 847 leader_election.cc:304] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:34.165174 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "49a33084c0fc460eb9592224b6dfc279" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:34.165339 1044 raft_consensus.cc:3060] T 49a33084c0fc460eb9592224b6dfc279 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:34.166080 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "49a33084c0fc460eb9592224b6dfc279" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:34.166250 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet d3b383ace72748d4bf21678894815c4f
I20251211 20:52:34.166740 593 catalog_manager.cc:6724] ReplaceTablet: tablet d3b383ace72748d4bf21678894815c4f deleted and replaced by tablet 745ce63314834c8abdecaa367133a068
I20251211 20:52:34.166819 1165 ts_tablet_manager.cc:1929] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20251211 20:52:34.166815 711 leader_election.cc:343] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:34.167003 1165 log.cc:1199] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/dcdcff8235814b58b3683104a2ff0cbf
I20251211 20:52:34.167377 1165 ts_tablet_manager.cc:1950] T dcdcff8235814b58b3683104a2ff0cbf P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:34.167768 1044 raft_consensus.cc:2468] T 49a33084c0fc460eb9592224b6dfc279 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:34.168747 885 tablet_service.cc:1552] Processing DeleteTablet for tablet d3b383ace72748d4bf21678894815c4f with delete_type TABLET_DATA_DELETED (replaced by tablet 745ce63314834c8abdecaa367133a068) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:34.169245 710 leader_election.cc:304] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:34.169636 1886 raft_consensus.cc:2804] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:34.169822 1886 raft_consensus.cc:697] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:34.170086 1886 consensus_queue.cc:237] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.168550 1164 raft_consensus.cc:2243] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.171056 1890 raft_consensus.cc:2706] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Received pre-election callback for term 1 while not running: Illegal state: RaftConsensus is not running: State = Stopping
I20251211 20:52:34.171170 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet dcdcff8235814b58b3683104a2ff0cbf (table default.Table_0209a7c0449b42e9ac582543b779f5c7 [id=3aa2f2349ee9431fb9bfb0634e5768b7]) successfully deleted
I20251211 20:52:34.171473 1164 raft_consensus.cc:2272] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.172096 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet d3b383ace72748d4bf21678894815c4f with delete_type TABLET_DATA_DELETED (replaced by tablet 745ce63314834c8abdecaa367133a068) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.172464 1160 tablet_bootstrap.cc:492] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:34.172567 751 tablet_service.cc:1552] Processing DeleteTablet for tablet d3b383ace72748d4bf21678894815c4f with delete_type TABLET_DATA_DELETED (replaced by tablet 745ce63314834c8abdecaa367133a068) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:34.172616 1160 ts_tablet_manager.cc:1403] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.052s user 0.002s sys 0.001s
I20251211 20:52:34.173461 1160 raft_consensus.cc:359] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.173686 1160 raft_consensus.cc:385] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.173774 1160 raft_consensus.cc:740] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.174069 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_e2163380d2474e9a93f10e69de094195"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:34.174691 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e2163380d2474e9a93f10e69de094195 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.174765 1160 consensus_queue.cc:260] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.180696 1160 ts_tablet_manager.cc:1434] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.008s user 0.001s sys 0.002s
I20251211 20:52:34.181510 1164 ts_tablet_manager.cc:1916] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.187881 1160 tablet_bootstrap.cc:492] T 27a08a64d92342d08ab4406e8006a97b P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:34.188165 1163 ts_tablet_manager.cc:1929] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.188287 1163 log.cc:1199] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/340730a17a834a77bbc20b95aba98e2c
I20251211 20:52:34.188668 1163 ts_tablet_manager.cc:1950] T 340730a17a834a77bbc20b95aba98e2c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
W20251211 20:52:34.189517 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet f2ae084982f44fdf8bfecdd2bc4c01b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2ae084982f44fdf8bfecdd2bc4c01b9 already in progress: creating tablet
I20251211 20:52:34.189775 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 340730a17a834a77bbc20b95aba98e2c (table default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f]) successfully deleted
W20251211 20:52:34.190146 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet cc43ee61e78249bfa00b11f3a1fa21a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cc43ee61e78249bfa00b11f3a1fa21a3 already in progress: creating tablet
W20251211 20:52:34.190378 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet aeff1fbabf7845dc9863f59f286ddcec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aeff1fbabf7845dc9863f59f286ddcec already in progress: creating tablet
W20251211 20:52:34.190605 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 3dde7ed495e64909bd9784dc5eefb04c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3dde7ed495e64909bd9784dc5eefb04c already in progress: creating tablet
W20251211 20:52:34.190833 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 58e35de0792a464a818170886ca8d8c9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 58e35de0792a464a818170886ca8d8c9 already in progress: creating tablet
W20251211 20:52:34.191418 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 58e35de0792a464a818170886ca8d8c9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 58e35de0792a464a818170886ca8d8c9 already in progress: creating tablet
I20251211 20:52:34.191543 1160 tablet_bootstrap.cc:654] T 27a08a64d92342d08ab4406e8006a97b P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:34.191977 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet f2ae084982f44fdf8bfecdd2bc4c01b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2ae084982f44fdf8bfecdd2bc4c01b9 already in progress: creating tablet
W20251211 20:52:34.193375 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet aeff1fbabf7845dc9863f59f286ddcec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aeff1fbabf7845dc9863f59f286ddcec already in progress: creating tablet
W20251211 20:52:34.193940 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 3dde7ed495e64909bd9784dc5eefb04c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3dde7ed495e64909bd9784dc5eefb04c already in progress: creating tablet
W20251211 20:52:34.194473 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet c74427cb9d6f4a679e43eae796ff8c3f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c74427cb9d6f4a679e43eae796ff8c3f already in progress: creating tablet
W20251211 20:52:34.194996 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet be876471f8bf453aa4467c91891cbbde because tablet deleting was already in progress. No further retry: Already present: State transition of tablet be876471f8bf453aa4467c91891cbbde already in progress: creating tablet
W20251211 20:52:34.196367 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet b529b38d754f46ba8cb60b19e9705256 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b529b38d754f46ba8cb60b19e9705256 already in progress: creating tablet
W20251211 20:52:34.197083 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet d3b383ace72748d4bf21678894815c4f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d3b383ace72748d4bf21678894815c4f already in progress: creating tablet
I20251211 20:52:34.200908 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_608d54a047d5494d8ea7e335dd734192" } new_table_name: "default.Table_a8ebfd7d7a7f4f6cae8b6b02b99b80b0" modify_external_catalogs: true
I20251211 20:52:34.202390 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_ef6d1803b4bf4d62860e0bbdf745446a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:34.202924 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef6d1803b4bf4d62860e0bbdf745446a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.207664 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_17b787b4b49444a1ad091fc751bcbd35"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:34.208221 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_17b787b4b49444a1ad091fc751bcbd35 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.209465 595 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_20c88681d6824e06ad511ad2ccb67eb4" } new_table_name: "default.Table_aa83784f17a84101a53b3e111f31d93e" modify_external_catalogs: true
I20251211 20:52:34.210057 1165 tablet_replica.cc:333] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:34.210279 1165 raft_consensus.cc:2243] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.210785 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c0ed4563b1d44eafa92065d3ec8adb93" } new_table_name: "default.Table_328b7dd76e9e407c83b47a0f51cfcce5" modify_external_catalogs: true
I20251211 20:52:34.211934 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_8aa9c0811b4f4ac987225c8038d27553"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:34.212399 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8aa9c0811b4f4ac987225c8038d27553 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.212400 1165 raft_consensus.cc:2272] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.213488 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_01821876da7e47a1a5f2cce0f011e20c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:34.214003 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_01821876da7e47a1a5f2cce0f011e20c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.214553 1165 ts_tablet_manager.cc:1916] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:34.227883 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fd43a27db69044b68826cda87e16a086 consensus state has no leader
I20251211 20:52:34.227888 1164 ts_tablet_manager.cc:1929] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.227980 1901 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.61:40749 every 8 connection(s)
I20251211 20:52:34.228050 1164 log.cc:1199] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/9adaee28edd74df68b44b25d505fdfcb
I20251211 20:52:34.228288 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_015f1fc2445947f38320ea7699cc7388" } modify_external_catalogs: true
I20251211 20:52:34.228425 1164 ts_tablet_manager.cc:1950] T 9adaee28edd74df68b44b25d505fdfcb P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:34.228482 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_015f1fc2445947f38320ea7699cc7388" } modify_external_catalogs: true
I20251211 20:52:34.228535 1160 tablet_bootstrap.cc:492] T 27a08a64d92342d08ab4406e8006a97b P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:34.228699 1160 ts_tablet_manager.cc:1403] T 27a08a64d92342d08ab4406e8006a97b P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.041s user 0.002s sys 0.001s
I20251211 20:52:34.228778 1667 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.61:40749
I20251211 20:52:34.229387 1164 tablet_replica.cc:333] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:34.229503 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 9adaee28edd74df68b44b25d505fdfcb (table default.Table_4c974eff9ab442d499f0405abc8a4acd [id=35caba098ab545f7859c004af8568ef7]) successfully deleted
I20251211 20:52:34.229478 1160 raft_consensus.cc:359] T 27a08a64d92342d08ab4406e8006a97b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.229601 1164 raft_consensus.cc:2243] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.230186 1160 raft_consensus.cc:385] T 27a08a64d92342d08ab4406e8006a97b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.230286 1160 raft_consensus.cc:740] T 27a08a64d92342d08ab4406e8006a97b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.230525 1160 consensus_queue.cc:260] T 27a08a64d92342d08ab4406e8006a97b P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.231240 1160 ts_tablet_manager.cc:1434] T 27a08a64d92342d08ab4406e8006a97b P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251211 20:52:34.231849 1667 server_base.cc:1179] Dumped server information to /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-1/data/info.pb
I20251211 20:52:34.232620 416 external_mini_cluster.cc:1428] Started /tmp/dist-test-task5TWsQc/build/asan/bin/kudu as pid 1667
I20251211 20:52:34.233244 1160 tablet_bootstrap.cc:492] T 49a33084c0fc460eb9592224b6dfc279 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:34.235049 1160 tablet_bootstrap.cc:654] T 49a33084c0fc460eb9592224b6dfc279 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.238363 1164 raft_consensus.cc:2272] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.238993 1164 ts_tablet_manager.cc:1916] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.245182 1164 ts_tablet_manager.cc:1929] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.246908 1165 ts_tablet_manager.cc:1929] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20251211 20:52:34.247545 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bc4e7362e6d045da81bf39f4135f500e consensus state has no leader
W20251211 20:52:34.248013 595 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 32f8d104f6a948d1a97a83769ea82449 consensus state has no leader
I20251211 20:52:34.250123 593 master_service.cc:954] ReplaceTablet: received request to replace tablet d0673c37c2fd47eea8ab10d9e91b8d7a from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:34.261436 1165 log.cc:1199] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/605172f066d242f2abea133c08a145ca
I20251211 20:52:34.261857 1165 ts_tablet_manager.cc:1950] T 605172f066d242f2abea133c08a145ca P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:34.261950 1166 tablet_bootstrap.cc:492] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:34.262095 1166 ts_tablet_manager.cc:1403] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.267s user 0.005s sys 0.017s
I20251211 20:52:34.262863 1166 raft_consensus.cc:359] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:34.263034 1166 raft_consensus.cc:385] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.263077 1166 raft_consensus.cc:740] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.263319 1165 tablet_replica.cc:333] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:34.263254 1166 consensus_queue.cc:260] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:34.263531 1165 raft_consensus.cc:2243] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.263729 1165 raft_consensus.cc:2272] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.263742 1166 ts_tablet_manager.cc:1434] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:34.264043 1166 tablet_bootstrap.cc:492] T b072b759d1254859b0b8d401e3078792 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:34.265705 1166 tablet_bootstrap.cc:654] T b072b759d1254859b0b8d401e3078792 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.267306 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 605172f066d242f2abea133c08a145ca (table default.Table_40b5190e58a440e9a13d6c9221572914 [id=e2b03132f53845138e4fd9731c2d79bf]) successfully deleted
I20251211 20:52:34.270109 1165 ts_tablet_manager.cc:1916] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.276517 1168 tablet_bootstrap.cc:492] T d5f0ba3ab16240f9822be69f941f5da2 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:34.276785 1168 ts_tablet_manager.cc:1403] T d5f0ba3ab16240f9822be69f941f5da2 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.123s user 0.002s sys 0.002s
I20251211 20:52:34.277668 1168 raft_consensus.cc:359] T d5f0ba3ab16240f9822be69f941f5da2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.277930 1168 raft_consensus.cc:385] T d5f0ba3ab16240f9822be69f941f5da2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.278064 1168 raft_consensus.cc:740] T d5f0ba3ab16240f9822be69f941f5da2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.278357 1168 consensus_queue.cc:260] T d5f0ba3ab16240f9822be69f941f5da2 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.279779 1168 ts_tablet_manager.cc:1434] T d5f0ba3ab16240f9822be69f941f5da2 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:34.280154 1168 tablet_bootstrap.cc:492] T 6a2442a4ebc940a281f9ea1a394f6621 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:34.250432 1160 tablet_bootstrap.cc:492] T 49a33084c0fc460eb9592224b6dfc279 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:34.280465 1164 log.cc:1199] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/c4b76a7e84d344268567c551236ce772
I20251211 20:52:34.280587 1160 ts_tablet_manager.cc:1403] T 49a33084c0fc460eb9592224b6dfc279 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.047s user 0.003s sys 0.003s
I20251211 20:52:34.281493 1164 ts_tablet_manager.cc:1950] T c4b76a7e84d344268567c551236ce772 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:34.281883 1160 raft_consensus.cc:359] T 49a33084c0fc460eb9592224b6dfc279 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.282099 1160 raft_consensus.cc:385] T 49a33084c0fc460eb9592224b6dfc279 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.282176 1160 raft_consensus.cc:740] T 49a33084c0fc460eb9592224b6dfc279 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.282397 1160 consensus_queue.cc:260] T 49a33084c0fc460eb9592224b6dfc279 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.282940 1160 ts_tablet_manager.cc:1434] T 49a33084c0fc460eb9592224b6dfc279 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:34.283269 1160 tablet_bootstrap.cc:492] T 19d0aa259ac94f36b7f6a24d11a3d420 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:34.285305 1160 tablet_bootstrap.cc:654] T 19d0aa259ac94f36b7f6a24d11a3d420 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.286827 1890 raft_consensus.cc:493] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:34.286995 1890 raft_consensus.cc:515] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.287693 1890 leader_election.cc:290] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:34.288414 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3eac21a07ba34e1fa275e25ee9dcabb1" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:34.289608 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3eac21a07ba34e1fa275e25ee9dcabb1" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:34.289871 776 raft_consensus.cc:2393] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb0f1933d9a74fccaa4618047fc05c41 in current term 1: Already voted for candidate e715a811a82446ce8d5bb4aafd5b81c7 in this term.
I20251211 20:52:34.290524 847 leader_election.cc:304] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:34.290913 1904 sys_catalog.cc:263] Verifying existing consensus state
I20251211 20:52:34.291034 1890 raft_consensus.cc:3060] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:34.293973 1890 raft_consensus.cc:2749] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:34.298828 591 catalog_manager.cc:5654] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:34.299286 591 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 49a33084c0fc460eb9592224b6dfc279 (table default.Table_6b98cb90147a43c4aee6b39240c5edc4 [id=9c24845e4efe475fa792a51ce8a66be6]). Expected version 4 got 0
I20251211 20:52:34.299571 1164 tablet_replica.cc:333] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:34.299800 1164 raft_consensus.cc:2243] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.299965 1164 raft_consensus.cc:2272] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.301937 1168 tablet_bootstrap.cc:654] T 6a2442a4ebc940a281f9ea1a394f6621 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.302585 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 3c5819ae1abb48fe8a83af9067726d02
I20251211 20:52:34.303452 1164 ts_tablet_manager.cc:1916] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.303442 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 3c5819ae1abb48fe8a83af9067726d02 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.303961 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 3c5819ae1abb48fe8a83af9067726d02 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:34.304350 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 3c5819ae1abb48fe8a83af9067726d02 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3c5819ae1abb48fe8a83af9067726d02 already in progress: creating tablet
I20251211 20:52:34.304665 1165 ts_tablet_manager.cc:1929] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.304832 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 3c5819ae1abb48fe8a83af9067726d02 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:34.304877 1165 log.cc:1199] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/c85624ad640041c29bb87416e1992a9e
I20251211 20:52:34.304932 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet c4b76a7e84d344268567c551236ce772 (table default.Table_afd3c9ee35ad41caa1717a299443409e [id=b0ebaab8cd5a4deb9a5ea1e7bd3ccec3]) successfully deleted
I20251211 20:52:34.305446 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0e678448f89e48f0ab54479b58c50c84" } modify_external_catalogs: true
I20251211 20:52:34.305627 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0e678448f89e48f0ab54479b58c50c84" } modify_external_catalogs: true
I20251211 20:52:34.305853 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_063f94689e6547e2b871d8465153a098"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:34.306005 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_3fe66404826142609775fed9970bbb7a" } modify_external_catalogs: true
I20251211 20:52:34.306157 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_3fe66404826142609775fed9970bbb7a" } modify_external_catalogs: true
W20251211 20:52:34.306470 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_063f94689e6547e2b871d8465153a098 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.308853 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0455012740604013898f53488fd5c371" } modify_external_catalogs: true
I20251211 20:52:34.308892 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4a209d1f3ab4458bb6197e926297e60a" } modify_external_catalogs: true
I20251211 20:52:34.309031 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0455012740604013898f53488fd5c371" } modify_external_catalogs: true
I20251211 20:52:34.309073 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4a209d1f3ab4458bb6197e926297e60a" } modify_external_catalogs: true
I20251211 20:52:34.310007 1166 tablet_bootstrap.cc:492] T b072b759d1254859b0b8d401e3078792 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:34.311609 1160 tablet_bootstrap.cc:492] T 19d0aa259ac94f36b7f6a24d11a3d420 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:34.311512 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b011142d3fc34ef2a829989a9bad1114"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:34.311774 1160 ts_tablet_manager.cc:1403] T 19d0aa259ac94f36b7f6a24d11a3d420 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.029s user 0.009s sys 0.000s
W20251211 20:52:34.312067 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b011142d3fc34ef2a829989a9bad1114 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.311990 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_514c179c4a144ce398e7c9b8fbba2bce"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:34.312320 1166 ts_tablet_manager.cc:1403] T b072b759d1254859b0b8d401e3078792 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.048s user 0.009s sys 0.001s
W20251211 20:52:34.312455 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_514c179c4a144ce398e7c9b8fbba2bce in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.313066 1166 raft_consensus.cc:359] T b072b759d1254859b0b8d401e3078792 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.313513 1165 ts_tablet_manager.cc:1950] T c85624ad640041c29bb87416e1992a9e P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:34.313627 1160 raft_consensus.cc:359] T 19d0aa259ac94f36b7f6a24d11a3d420 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.313863 1166 raft_consensus.cc:385] T b072b759d1254859b0b8d401e3078792 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.313926 1166 raft_consensus.cc:740] T b072b759d1254859b0b8d401e3078792 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.314066 1160 raft_consensus.cc:385] T 19d0aa259ac94f36b7f6a24d11a3d420 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.314138 1160 raft_consensus.cc:740] T 19d0aa259ac94f36b7f6a24d11a3d420 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.314240 751 tablet_service.cc:1552] Processing DeleteTablet for tablet d5f0ba3ab16240f9822be69f941f5da2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:34.314334 1160 consensus_queue.cc:260] T 19d0aa259ac94f36b7f6a24d11a3d420 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.314713 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet c85624ad640041c29bb87416e1992a9e (table default.Table_8f7e347b9ac64793b1073dfa955ebe21 [id=a6ab64c3e0d94462b716f2d153bc3314]) successfully deleted
I20251211 20:52:34.314800 1160 ts_tablet_manager.cc:1434] T 19d0aa259ac94f36b7f6a24d11a3d420 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251211 20:52:34.314990 1166 consensus_queue.cc:260] T b072b759d1254859b0b8d401e3078792 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.315201 1165 tablet_replica.cc:333] T 804a11c8f7414eb1a6bcff2952be9ae2 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:34.315426 1165 raft_consensus.cc:2243] T 804a11c8f7414eb1a6bcff2952be9ae2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.315689 1165 raft_consensus.cc:2272] T 804a11c8f7414eb1a6bcff2952be9ae2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.316500 1165 ts_tablet_manager.cc:1916] T 804a11c8f7414eb1a6bcff2952be9ae2 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.318919 1166 ts_tablet_manager.cc:1434] T b072b759d1254859b0b8d401e3078792 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20251211 20:52:34.322275 1164 ts_tablet_manager.cc:1929] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.322665 1166 tablet_bootstrap.cc:492] T 5bd801124df84a6c96f4912993f1bdfd P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:34.326370 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet d19975cc513e414b80f478a6c5eb09de
I20251211 20:52:34.327796 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet d19975cc513e414b80f478a6c5eb09de with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.328045 1163 tablet_replica.cc:333] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:34.328212 1163 raft_consensus.cc:2243] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.328377 1163 raft_consensus.cc:2272] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.328929 1163 ts_tablet_manager.cc:1916] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.330286 1166 tablet_bootstrap.cc:654] T 5bd801124df84a6c96f4912993f1bdfd P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.330420 751 tablet_service.cc:1552] Processing DeleteTablet for tablet d19975cc513e414b80f478a6c5eb09de with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:34.337447 1166 tablet_bootstrap.cc:492] T 5bd801124df84a6c96f4912993f1bdfd P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:34.323421 1160 tablet_bootstrap.cc:492] T cf1201bba4b6421f8d7ce593f4bd892b P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:34.337760 1164 log.cc:1199] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/88066d94d5d34e0fab20da9fb6ef178e
I20251211 20:52:34.338207 1164 ts_tablet_manager.cc:1950] T 88066d94d5d34e0fab20da9fb6ef178e P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:34.339344 885 tablet_service.cc:1552] Processing DeleteTablet for tablet d19975cc513e414b80f478a6c5eb09de with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:34.339987 1160 tablet_bootstrap.cc:654] T cf1201bba4b6421f8d7ce593f4bd892b P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.340618 1164 tablet_replica.cc:333] T 5fd2a6cf80a04f20aef8ff9c84bc88db P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:34.340889 1164 raft_consensus.cc:2243] T 5fd2a6cf80a04f20aef8ff9c84bc88db P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.341125 1164 raft_consensus.cc:2272] T 5fd2a6cf80a04f20aef8ff9c84bc88db P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.341214 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 88066d94d5d34e0fab20da9fb6ef178e (table default.Table_b06f9587f78645bb9712c3b080f3c0d4 [id=ea12bdaa894f446da22635f95d97ac2a]) successfully deleted
I20251211 20:52:34.341862 1164 ts_tablet_manager.cc:1916] T 5fd2a6cf80a04f20aef8ff9c84bc88db P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.344070 1165 ts_tablet_manager.cc:1929] T 804a11c8f7414eb1a6bcff2952be9ae2 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.345494 1165 log.cc:1199] T 804a11c8f7414eb1a6bcff2952be9ae2 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/804a11c8f7414eb1a6bcff2952be9ae2
I20251211 20:52:34.345939 1165 ts_tablet_manager.cc:1950] T 804a11c8f7414eb1a6bcff2952be9ae2 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:34.346099 1904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Bootstrap starting.
I20251211 20:52:34.339416 1163 ts_tablet_manager.cc:1929] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20251211 20:52:34.346797 1163 log.cc:1199] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/d19975cc513e414b80f478a6c5eb09de
I20251211 20:52:34.347105 1163 ts_tablet_manager.cc:1950] T d19975cc513e414b80f478a6c5eb09de P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:34.337608 1166 ts_tablet_manager.cc:1403] T 5bd801124df84a6c96f4912993f1bdfd P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20251211 20:52:34.348809 1166 raft_consensus.cc:359] T 5bd801124df84a6c96f4912993f1bdfd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.348982 1166 raft_consensus.cc:385] T 5bd801124df84a6c96f4912993f1bdfd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.349041 1166 raft_consensus.cc:740] T 5bd801124df84a6c96f4912993f1bdfd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.349372 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet d19975cc513e414b80f478a6c5eb09de (table default.Table_0455012740604013898f53488fd5c371 [id=d770b26c251e41e38f9720b37852cb11]) successfully deleted
I20251211 20:52:34.349648 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 804a11c8f7414eb1a6bcff2952be9ae2 (table default.Table_415c09e71243460aaa9acfd57742bcd7 [id=7f9092a9ef3945a7851d0d9f762e161a]) successfully deleted
I20251211 20:52:34.339735 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_2282dc6429c345098eeaf08b63262e8b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:34.350472 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2282dc6429c345098eeaf08b63262e8b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.350831 588 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b7c88854437f4dfd9998ccb8246e6d0d" } new_table_name: "default.Table_12f41f708f1949598d6176d89777aa69" modify_external_catalogs: true
I20251211 20:52:34.351867 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_fa0592c7846d4e4883138ca9e597a452" } new_table_name: "default.Table_4cbc8841b276406cb4434629653a5ce3" modify_external_catalogs: true
I20251211 20:52:34.352063 1166 consensus_queue.cc:260] T 5bd801124df84a6c96f4912993f1bdfd P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.353282 1160 tablet_bootstrap.cc:492] T cf1201bba4b6421f8d7ce593f4bd892b P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:34.353436 1160 ts_tablet_manager.cc:1403] T cf1201bba4b6421f8d7ce593f4bd892b P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.030s user 0.006s sys 0.000s
I20251211 20:52:34.354308 1160 raft_consensus.cc:359] T cf1201bba4b6421f8d7ce593f4bd892b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.354521 1160 raft_consensus.cc:385] T cf1201bba4b6421f8d7ce593f4bd892b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20251211 20:52:34.354581 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet c74427cb9d6f4a679e43eae796ff8c3f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c74427cb9d6f4a679e43eae796ff8c3f already in progress: creating tablet
I20251211 20:52:34.354615 1160 raft_consensus.cc:740] T cf1201bba4b6421f8d7ce593f4bd892b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.355353 1586 raft_consensus.cc:493] T 5bd801124df84a6c96f4912993f1bdfd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:34.355487 1586 raft_consensus.cc:515] T 5bd801124df84a6c96f4912993f1bdfd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.356106 1586 leader_election.cc:290] T 5bd801124df84a6c96f4912993f1bdfd P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:34.356133 1165 tablet_replica.cc:333] T d5f0ba3ab16240f9822be69f941f5da2 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:34.356348 1165 raft_consensus.cc:2243] T d5f0ba3ab16240f9822be69f941f5da2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.358179 1165 raft_consensus.cc:2272] T d5f0ba3ab16240f9822be69f941f5da2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.358244 1166 ts_tablet_manager.cc:1434] T 5bd801124df84a6c96f4912993f1bdfd P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20251211 20:52:34.358750 1166 tablet_bootstrap.cc:492] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:34.359436 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5bd801124df84a6c96f4912993f1bdfd" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:34.359838 1165 ts_tablet_manager.cc:1916] T d5f0ba3ab16240f9822be69f941f5da2 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.360350 1166 tablet_bootstrap.cc:654] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:34.361093 982 leader_election.cc:343] T 5bd801124df84a6c96f4912993f1bdfd P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:34.358043 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5bd801124df84a6c96f4912993f1bdfd" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:34.354833 1160 consensus_queue.cc:260] T cf1201bba4b6421f8d7ce593f4bd892b P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.362160 1160 ts_tablet_manager.cc:1434] T cf1201bba4b6421f8d7ce593f4bd892b P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20251211 20:52:34.362427 1160 tablet_bootstrap.cc:492] T f93006227cc6489e91f3593e11789b54 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:34.362845 983 leader_election.cc:343] T 5bd801124df84a6c96f4912993f1bdfd P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:34.362944 983 leader_election.cc:304] T 5bd801124df84a6c96f4912993f1bdfd P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:34.363030 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet be876471f8bf453aa4467c91891cbbde because tablet deleting was already in progress. No further retry: Already present: State transition of tablet be876471f8bf453aa4467c91891cbbde already in progress: creating tablet
W20251211 20:52:34.363297 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet b529b38d754f46ba8cb60b19e9705256 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b529b38d754f46ba8cb60b19e9705256 already in progress: creating tablet
I20251211 20:52:34.363376 1586 raft_consensus.cc:2749] T 5bd801124df84a6c96f4912993f1bdfd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:34.363529 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet d3b383ace72748d4bf21678894815c4f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d3b383ace72748d4bf21678894815c4f already in progress: creating tablet
W20251211 20:52:34.363777 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 3c5819ae1abb48fe8a83af9067726d02 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3c5819ae1abb48fe8a83af9067726d02 already in progress: creating tablet
I20251211 20:52:34.365818 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_198aadd702614de39863c677cd5d885b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:34.368165 1160 tablet_bootstrap.cc:654] T f93006227cc6489e91f3593e11789b54 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.370962 1164 ts_tablet_manager.cc:1929] T 5fd2a6cf80a04f20aef8ff9c84bc88db P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.371069 1164 log.cc:1199] T 5fd2a6cf80a04f20aef8ff9c84bc88db P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/5fd2a6cf80a04f20aef8ff9c84bc88db
I20251211 20:52:34.371672 1164 ts_tablet_manager.cc:1950] T 5fd2a6cf80a04f20aef8ff9c84bc88db P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:34.374179 1166 tablet_bootstrap.cc:492] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:34.374323 1166 ts_tablet_manager.cc:1403] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.003s
I20251211 20:52:34.375169 1166 raft_consensus.cc:359] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.377154 1166 raft_consensus.cc:385] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.377218 1166 raft_consensus.cc:740] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.378087 1166 consensus_queue.cc:260] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:34.380009 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_198aadd702614de39863c677cd5d885b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.382560 1166 ts_tablet_manager.cc:1434] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
W20251211 20:52:34.385653 588 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 40722aef4e194ff094d271a079bda9d9 consensus state has no leader
I20251211 20:52:34.388032 1166 tablet_bootstrap.cc:492] T 3cd7a465249246c28f07b8fb2e42675f P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:34.388451 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ab4e7a9062134e489d6f9abe6ff26fb4 consensus state has no leader
I20251211 20:52:34.389006 1165 ts_tablet_manager.cc:1929] T d5f0ba3ab16240f9822be69f941f5da2 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.389163 1165 log.cc:1199] T d5f0ba3ab16240f9822be69f941f5da2 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/d5f0ba3ab16240f9822be69f941f5da2
I20251211 20:52:34.389508 1165 ts_tablet_manager.cc:1950] T d5f0ba3ab16240f9822be69f941f5da2 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:34.393287 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet d5f0ba3ab16240f9822be69f941f5da2 (table default.Table_d73d156809724a399ad27846bfcc460b [id=baf6ca043ec048b894cf9716fad105b2]) successfully deleted
W20251211 20:52:34.394745 581 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 8feee01fc0aa4b32be926ff2b617b894 (table default.Table_265379fa17b74d0f954a2a2c4de113c8 [id=18f8d722faaa4293a112525ff0c96216], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 8feee01fc0aa4b32be926ff2b617b894 consensus state has no leader
I20251211 20:52:34.396314 1166 tablet_bootstrap.cc:654] T 3cd7a465249246c28f07b8fb2e42675f P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.409193 1890 raft_consensus.cc:493] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:34.409353 1890 raft_consensus.cc:515] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.410037 1890 leader_election.cc:290] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:34.410915 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ae1abef61473443b9922eb13f2ae92fa" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:34.411113 1044 raft_consensus.cc:2468] T ae1abef61473443b9922eb13f2ae92fa P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 0.
I20251211 20:52:34.411195 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ae1abef61473443b9922eb13f2ae92fa" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:34.411382 776 raft_consensus.cc:2468] T ae1abef61473443b9922eb13f2ae92fa P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 0.
I20251211 20:52:34.411859 847 leader_election.cc:304] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters:
I20251211 20:52:34.412480 1890 raft_consensus.cc:2804] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:34.412569 1890 raft_consensus.cc:493] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:34.412628 1890 raft_consensus.cc:3060] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:34.413815 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 5fd2a6cf80a04f20aef8ff9c84bc88db (table default.Table_370d3069ceee4b8e8856bd74fe3a1815 [id=10fd0368198c447c95b692b1f3e6a023]) successfully deleted
I20251211 20:52:34.414155 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 6d4e6528352f43b6a1b47d7a4b6b3689
I20251211 20:52:34.415094 1890 raft_consensus.cc:515] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.415256 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 6d4e6528352f43b6a1b47d7a4b6b3689 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:34.415768 1164 tablet_replica.cc:333] T c910246b3d5d4e7e8a71d9f18e94d24c P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:34.415899 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 6d4e6528352f43b6a1b47d7a4b6b3689 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.415941 1164 raft_consensus.cc:2243] T c910246b3d5d4e7e8a71d9f18e94d24c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.416070 1164 raft_consensus.cc:2272] T c910246b3d5d4e7e8a71d9f18e94d24c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:34.416527 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 6d4e6528352f43b6a1b47d7a4b6b3689 because the tablet was not found. No further retry: Not found: Tablet not found: 6d4e6528352f43b6a1b47d7a4b6b3689
W20251211 20:52:34.416987 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet cc43ee61e78249bfa00b11f3a1fa21a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cc43ee61e78249bfa00b11f3a1fa21a3 already in progress: creating tablet
W20251211 20:52:34.417187 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet f2ae084982f44fdf8bfecdd2bc4c01b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2ae084982f44fdf8bfecdd2bc4c01b9 already in progress: creating tablet
W20251211 20:52:34.417402 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet aeff1fbabf7845dc9863f59f286ddcec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aeff1fbabf7845dc9863f59f286ddcec already in progress: creating tablet
W20251211 20:52:34.417586 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet c74427cb9d6f4a679e43eae796ff8c3f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c74427cb9d6f4a679e43eae796ff8c3f already in progress: creating tablet
W20251211 20:52:34.417780 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 58e35de0792a464a818170886ca8d8c9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 58e35de0792a464a818170886ca8d8c9 already in progress: creating tablet
W20251211 20:52:34.417943 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 3dde7ed495e64909bd9784dc5eefb04c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3dde7ed495e64909bd9784dc5eefb04c already in progress: creating tablet
I20251211 20:52:34.418256 1165 tablet_replica.cc:333] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:34.418355 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 6d4e6528352f43b6a1b47d7a4b6b3689 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:34.418454 1165 raft_consensus.cc:2243] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.418663 1165 raft_consensus.cc:2272] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.419236 1165 ts_tablet_manager.cc:1916] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.422235 1164 ts_tablet_manager.cc:1916] T c910246b3d5d4e7e8a71d9f18e94d24c P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.423780 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ae1abef61473443b9922eb13f2ae92fa" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:34.423983 776 raft_consensus.cc:3060] T ae1abef61473443b9922eb13f2ae92fa P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:34.425578 1165 ts_tablet_manager.cc:1929] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20251211 20:52:34.425789 1165 log.cc:1199] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/d19975cc513e414b80f478a6c5eb09de
I20251211 20:52:34.425794 596 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b072b759d1254859b0b8d401e3078792 (table default.Table_b56de1e85a1b42ea81308593884dadb8 [id=a137f0ddf52145a99c62444602ab3fdb]). Expected version 4 got 0
I20251211 20:52:34.425962 592 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 49a33084c0fc460eb9592224b6dfc279 (table default.Table_6b98cb90147a43c4aee6b39240c5edc4 [id=9c24845e4efe475fa792a51ce8a66be6]). Expected version 4 got 0
I20251211 20:52:34.426283 1165 ts_tablet_manager.cc:1950] T d19975cc513e414b80f478a6c5eb09de P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:34.426484 776 raft_consensus.cc:2468] T ae1abef61473443b9922eb13f2ae92fa P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 1.
I20251211 20:52:34.427269 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet d19975cc513e414b80f478a6c5eb09de (table default.Table_0455012740604013898f53488fd5c371 [id=d770b26c251e41e38f9720b37852cb11]) successfully deleted
I20251211 20:52:34.429127 665 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 6d4e6528352f43b6a1b47d7a4b6b3689
I20251211 20:52:34.429570 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_7dc91b2b86fc4c95b162a34345fc8443" } new_table_name: "default.Table_faa83de4f1eb4c8fb577bcaaa63687d7" modify_external_catalogs: true
I20251211 20:52:34.430140 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 27a08a64d92342d08ab4406e8006a97b with delete_type TABLET_DATA_DELETED (replaced by tablet bfb52935296443c58190d4d06e2e6e9c) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:34.430359 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 19d0aa259ac94f36b7f6a24d11a3d420 with delete_type TABLET_DATA_DELETED (replaced by tablet b9e5ad8030a44e03acfe73124b156ac8) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:34.431555 1886 consensus_queue.cc:1048] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:34.432518 596 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b072b759d1254859b0b8d401e3078792 (table default.Table_b56de1e85a1b42ea81308593884dadb8 [id=a137f0ddf52145a99c62444602ab3fdb]). Expected version 4 got 0
I20251211 20:52:34.432711 596 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 60d966c0397c4a4ea3718445e87e9703 (table default.Table_f51ac924827c4dbea700109489db0963 [id=cae3f6884694487e82fc81328e9527d1]). Expected version 1 got 0
I20251211 20:52:34.432884 596 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 5bd801124df84a6c96f4912993f1bdfd (table default.Table_9aa05c766d874348a9e0eb2c016058d3 [id=44f2f2bf53af4b038ced294f20db9e48]). Expected version 1 got 0
I20251211 20:52:34.436655 1586 raft_consensus.cc:493] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:34.436795 1586 raft_consensus.cc:515] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.437358 1586 leader_election.cc:290] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:34.437824 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "60d966c0397c4a4ea3718445e87e9703" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:34.438109 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "60d966c0397c4a4ea3718445e87e9703" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:34.438319 982 leader_election.cc:343] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:34.438552 1890 leader_election.cc:290] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Requested vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:34.438678 983 leader_election.cc:343] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:34.438781 983 leader_election.cc:304] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:34.439215 1586 raft_consensus.cc:2749] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:34.439220 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ae1abef61473443b9922eb13f2ae92fa" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:34.439615 1044 raft_consensus.cc:3060] T ae1abef61473443b9922eb13f2ae92fa P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:34.440759 581 catalog_manager.cc:4952] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 6d4e6528352f43b6a1b47d7a4b6b3689 because the tablet was not found. No further retry: Not found: Tablet not found: 6d4e6528352f43b6a1b47d7a4b6b3689
I20251211 20:52:34.441740 1164 ts_tablet_manager.cc:1929] T c910246b3d5d4e7e8a71d9f18e94d24c P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.441876 1164 log.cc:1199] T c910246b3d5d4e7e8a71d9f18e94d24c P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/c910246b3d5d4e7e8a71d9f18e94d24c
I20251211 20:52:34.441992 1044 raft_consensus.cc:2468] T ae1abef61473443b9922eb13f2ae92fa P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 1.
I20251211 20:52:34.442150 1164 ts_tablet_manager.cc:1950] T c910246b3d5d4e7e8a71d9f18e94d24c P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:34.442730 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet c910246b3d5d4e7e8a71d9f18e94d24c (table default.Table_c2300bb8bfe44c689969a2b233d69353 [id=996745d9420c4589a52eca6bad10d6a9]) successfully deleted
I20251211 20:52:34.443176 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_11d5dfcc3a7d4db59119bf9c9666c50a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:34.444162 847 leader_election.cc:304] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41; no voters:
W20251211 20:52:34.444413 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_11d5dfcc3a7d4db59119bf9c9666c50a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:34.446028 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet be876471f8bf453aa4467c91891cbbde because tablet deleting was already in progress. No further retry: Already present: State transition of tablet be876471f8bf453aa4467c91891cbbde already in progress: creating tablet
I20251211 20:52:34.447489 1164 tablet_replica.cc:333] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:34.447690 1164 raft_consensus.cc:2243] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [term 2 LEADER]: Raft consensus shutting down.
I20251211 20:52:34.448058 1164 raft_consensus.cc:2272] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.448514 1164 ts_tablet_manager.cc:1916] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.448858 1890 raft_consensus.cc:2804] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election won for term 1
W20251211 20:52:34.449221 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet b529b38d754f46ba8cb60b19e9705256 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b529b38d754f46ba8cb60b19e9705256 already in progress: creating tablet
I20251211 20:52:34.449312 1890 raft_consensus.cc:697] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41 [term 1 LEADER]: Becoming Leader. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Running, Role: LEADER
W20251211 20:52:34.449483 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet d3b383ace72748d4bf21678894815c4f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d3b383ace72748d4bf21678894815c4f already in progress: creating tablet
I20251211 20:52:34.449551 1890 consensus_queue.cc:237] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:34.449700 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 3c5819ae1abb48fe8a83af9067726d02 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3c5819ae1abb48fe8a83af9067726d02 already in progress: creating tablet
I20251211 20:52:34.452046 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f06fbcca762d4b93b6dd39878d6d58fb" } new_table_name: "default.Table_907c596bdddc4edca25d14cbacebb23f" modify_external_catalogs: true
I20251211 20:52:34.452330 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_d71f88120b9d4408a1b04988573d137d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:34.452826 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d71f88120b9d4408a1b04988573d137d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.452862 588 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8747ca05a949428ba103bb1e2088f0f2" } new_table_name: "default.Table_ca2877f9e5004eada6f91e1d7bc95c10" modify_external_catalogs: true
I20251211 20:52:34.457762 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_24b29712550340f2bea51b04f527c4f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:34.458590 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_24b29712550340f2bea51b04f527c4f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.461215 590 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 2630ad57845a4c31b20c8a104dd27335 (table default.Table_4d70f35e05524680b3d72b2e488adbd5 [id=61b2de16fee74487823292913de1b685]). Expected version 3 got 0
I20251211 20:52:34.461516 1096 heartbeater.cc:344] Connected to a master server at 127.0.104.61:40749
I20251211 20:52:34.462162 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_3a9e5e76c5324906b05c61141dfb7034"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:34.462924 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3a9e5e76c5324906b05c61141dfb7034 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.466622 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 6d4e6528352f43b6a1b47d7a4b6b3689 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:34.466979 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 6d4e6528352f43b6a1b47d7a4b6b3689 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:34.467669 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 6d4e6528352f43b6a1b47d7a4b6b3689 because the tablet was not found. No further retry: Not found: Tablet not found: 6d4e6528352f43b6a1b47d7a4b6b3689
I20251211 20:52:34.470600 751 tablet_service.cc:1505] Processing CreateTablet for tablet a3e05f60fd5540d38230ffab3bf49307 (DEFAULT_TABLE table=default.Table_5a933aeef8dc44b18bb0cf7ddfe412f2 [id=3496dd8ce5184ac79c516278d403cbcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.471014 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3e05f60fd5540d38230ffab3bf49307. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.471626 1166 tablet_bootstrap.cc:492] T 3cd7a465249246c28f07b8fb2e42675f P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:34.471797 1166 ts_tablet_manager.cc:1403] T 3cd7a465249246c28f07b8fb2e42675f P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.084s user 0.004s sys 0.004s
I20251211 20:52:34.472543 1166 raft_consensus.cc:359] T 3cd7a465249246c28f07b8fb2e42675f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.472702 1166 raft_consensus.cc:385] T 3cd7a465249246c28f07b8fb2e42675f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.472743 1166 raft_consensus.cc:740] T 3cd7a465249246c28f07b8fb2e42675f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.472916 1166 consensus_queue.cc:260] T 3cd7a465249246c28f07b8fb2e42675f P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.473346 1166 ts_tablet_manager.cc:1434] T 3cd7a465249246c28f07b8fb2e42675f P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251211 20:52:34.476018 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 6d4e6528352f43b6a1b47d7a4b6b3689 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:34.476475 885 tablet_service.cc:1505] Processing CreateTablet for tablet a3e05f60fd5540d38230ffab3bf49307 (DEFAULT_TABLE table=default.Table_5a933aeef8dc44b18bb0cf7ddfe412f2 [id=3496dd8ce5184ac79c516278d403cbcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.476884 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3e05f60fd5540d38230ffab3bf49307. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.480472 564 master_runner.cc:186] Error getting master registration for 127.0.104.61:40749: OK, instance_id { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" instance_seqno: 1765486354040642 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20251211 20:52:34.480780 564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20251211 20:52:34.482596 1870 consensus_queue.cc:1048] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:34.485114 1164 ts_tablet_manager.cc:1929] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20251211 20:52:34.485308 1164 log.cc:1199] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/d19975cc513e414b80f478a6c5eb09de
I20251211 20:52:34.485731 1164 ts_tablet_manager.cc:1950] T d19975cc513e414b80f478a6c5eb09de P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:34.486364 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet d19975cc513e414b80f478a6c5eb09de (table default.Table_0455012740604013898f53488fd5c371 [id=d770b26c251e41e38f9720b37852cb11]) successfully deleted
I20251211 20:52:34.490326 1168 tablet_bootstrap.cc:492] T 6a2442a4ebc940a281f9ea1a394f6621 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:34.490486 1168 ts_tablet_manager.cc:1403] T 6a2442a4ebc940a281f9ea1a394f6621 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.210s user 0.004s sys 0.025s
I20251211 20:52:34.491242 1168 raft_consensus.cc:359] T 6a2442a4ebc940a281f9ea1a394f6621 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.491416 1168 raft_consensus.cc:385] T 6a2442a4ebc940a281f9ea1a394f6621 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.491482 1168 raft_consensus.cc:740] T 6a2442a4ebc940a281f9ea1a394f6621 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.491726 1168 consensus_queue.cc:260] T 6a2442a4ebc940a281f9ea1a394f6621 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:34.491983 582 catalog_manager.cc:4952] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 6d4e6528352f43b6a1b47d7a4b6b3689 because the tablet was not found. No further retry: Not found: Tablet not found: 6d4e6528352f43b6a1b47d7a4b6b3689
I20251211 20:52:34.492262 1168 ts_tablet_manager.cc:1434] T 6a2442a4ebc940a281f9ea1a394f6621 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:34.492444 1164 tablet_replica.cc:333] T 27a08a64d92342d08ab4406e8006a97b P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:34.492661 1164 raft_consensus.cc:2243] T 27a08a64d92342d08ab4406e8006a97b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.492880 1164 raft_consensus.cc:2272] T 27a08a64d92342d08ab4406e8006a97b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.492926 1168 tablet_bootstrap.cc:492] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:34.493723 581 catalog_manager.cc:4952] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 6d4e6528352f43b6a1b47d7a4b6b3689 because the tablet was not found. No further retry: Not found: Tablet not found: 6d4e6528352f43b6a1b47d7a4b6b3689
I20251211 20:52:34.493767 1164 ts_tablet_manager.cc:1916] T 27a08a64d92342d08ab4406e8006a97b P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.495033 1160 tablet_bootstrap.cc:492] T f93006227cc6489e91f3593e11789b54 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:34.495170 1160 ts_tablet_manager.cc:1403] T f93006227cc6489e91f3593e11789b54 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.133s user 0.001s sys 0.020s
I20251211 20:52:34.497265 1168 tablet_bootstrap.cc:654] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.497476 1160 raft_consensus.cc:359] T f93006227cc6489e91f3593e11789b54 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.497635 1160 raft_consensus.cc:385] T f93006227cc6489e91f3593e11789b54 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.497680 1160 raft_consensus.cc:740] T f93006227cc6489e91f3593e11789b54 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.500036 1166 tablet_bootstrap.cc:492] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:34.502071 1096 heartbeater.cc:344] Connected to a master server at 127.0.104.61:40749
I20251211 20:52:34.502741 1166 tablet_bootstrap.cc:654] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.504659 1160 consensus_queue.cc:260] T f93006227cc6489e91f3593e11789b54 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.505339 1160 ts_tablet_manager.cc:1434] T f93006227cc6489e91f3593e11789b54 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20251211 20:52:34.505632 1160 tablet_bootstrap.cc:492] T bb5c5523e175406d9f6f9cd2736a2045 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:34.507191 1160 tablet_bootstrap.cc:654] T bb5c5523e175406d9f6f9cd2736a2045 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.510483 1164 ts_tablet_manager.cc:1929] T 27a08a64d92342d08ab4406e8006a97b P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.510695 1164 log.cc:1199] T 27a08a64d92342d08ab4406e8006a97b P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/27a08a64d92342d08ab4406e8006a97b
I20251211 20:52:34.510985 1164 ts_tablet_manager.cc:1950] T 27a08a64d92342d08ab4406e8006a97b P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:34.511394 1164 tablet_replica.cc:333] T 19d0aa259ac94f36b7f6a24d11a3d420 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:34.511930 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 27a08a64d92342d08ab4406e8006a97b (table default.Table_3721b5725e8c4297a392d3986a081128 [id=383392d835da4c50848068d83261cecc]) successfully deleted
I20251211 20:52:34.513257 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 7d1cc9809ab843e693a179a35c53ce25
I20251211 20:52:34.514499 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 7d1cc9809ab843e693a179a35c53ce25 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:34.514678 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 7d1cc9809ab843e693a179a35c53ce25 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.514986 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet e1aeae8c18b6499e9dc1accabdc3f633
W20251211 20:52:34.515560 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 7d1cc9809ab843e693a179a35c53ce25 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d1cc9809ab843e693a179a35c53ce25 already in progress: creating tablet
I20251211 20:52:34.516327 742 tablet_service.cc:1552] Processing DeleteTablet for tablet e1aeae8c18b6499e9dc1accabdc3f633 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:34.517132 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet e1aeae8c18b6499e9dc1accabdc3f633 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e1aeae8c18b6499e9dc1accabdc3f633 already in progress: creating tablet
I20251211 20:52:34.517627 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet e1aeae8c18b6499e9dc1accabdc3f633 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.518029 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet d0673c37c2fd47eea8ab10d9e91b8d7a
I20251211 20:52:34.518262 1166 tablet_bootstrap.cc:492] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
W20251211 20:52:34.518317 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet e1aeae8c18b6499e9dc1accabdc3f633 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e1aeae8c18b6499e9dc1accabdc3f633 already in progress: creating tablet
I20251211 20:52:34.518517 593 catalog_manager.cc:6724] ReplaceTablet: tablet d0673c37c2fd47eea8ab10d9e91b8d7a deleted and replaced by tablet 86c5228210954839ae7bf5bf0b5aa027
I20251211 20:52:34.518615 1166 ts_tablet_manager.cc:1403] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.003s
I20251211 20:52:34.518898 742 tablet_service.cc:1552] Processing DeleteTablet for tablet d0673c37c2fd47eea8ab10d9e91b8d7a with delete_type TABLET_DATA_DELETED (replaced by tablet 86c5228210954839ae7bf5bf0b5aa027) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:34.519603 891 tablet_service.cc:1552] Processing DeleteTablet for tablet d0673c37c2fd47eea8ab10d9e91b8d7a with delete_type TABLET_DATA_DELETED (replaced by tablet 86c5228210954839ae7bf5bf0b5aa027) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:34.519917 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet d0673c37c2fd47eea8ab10d9e91b8d7a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d0673c37c2fd47eea8ab10d9e91b8d7a already in progress: creating tablet
I20251211 20:52:34.519905 1166 raft_consensus.cc:359] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.520241 1166 raft_consensus.cc:385] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.520334 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet d0673c37c2fd47eea8ab10d9e91b8d7a with delete_type TABLET_DATA_DELETED (replaced by tablet 86c5228210954839ae7bf5bf0b5aa027) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.520511 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 7d1cc9809ab843e693a179a35c53ce25 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:34.521008 1166 raft_consensus.cc:740] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
W20251211 20:52:34.521087 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 7d1cc9809ab843e693a179a35c53ce25 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d1cc9809ab843e693a179a35c53ce25 already in progress: creating tablet
W20251211 20:52:34.521337 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet d0673c37c2fd47eea8ab10d9e91b8d7a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d0673c37c2fd47eea8ab10d9e91b8d7a already in progress: creating tablet
I20251211 20:52:34.521358 1166 consensus_queue.cc:260] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:34.521819 592 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 07e4336171714da8adc21977ce33ce50 consensus state has no leader
I20251211 20:52:34.522382 592 master_service.cc:954] ReplaceTablet: received request to replace tablet d30376baf75b4761971f91da3493a142 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:34.523166 592 master_service.cc:954] ReplaceTablet: received request to replace tablet 50f9f29c064c4f938a0f20f302ab5e8a from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:34.523731 1010 tablet_service.cc:1505] Processing CreateTablet for tablet a3e05f60fd5540d38230ffab3bf49307 (DEFAULT_TABLE table=default.Table_5a933aeef8dc44b18bb0cf7ddfe412f2 [id=3496dd8ce5184ac79c516278d403cbcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.524102 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3e05f60fd5540d38230ffab3bf49307. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.524029 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_434ed373bf494a40b55b1f275d648d28"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:34.524366 891 tablet_service.cc:1505] Processing CreateTablet for tablet e218051cdc404af0b0335fa4c373222d (DEFAULT_TABLE table=default.Table_015ac77359ed47db98e074fa602a0141 [id=5db43d4090b947ffa3bdad62184f1db9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.524730 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e218051cdc404af0b0335fa4c373222d. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:34.524745 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_434ed373bf494a40b55b1f275d648d28 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.527098 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ddc346d7126a4b579fa1f29b6ef6cb87" } modify_external_catalogs: true
I20251211 20:52:34.527269 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ddc346d7126a4b579fa1f29b6ef6cb87" } modify_external_catalogs: true
I20251211 20:52:34.528172 1164 raft_consensus.cc:2243] T 19d0aa259ac94f36b7f6a24d11a3d420 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.528366 1164 raft_consensus.cc:2272] T 19d0aa259ac94f36b7f6a24d11a3d420 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.530297 1164 ts_tablet_manager.cc:1916] T 19d0aa259ac94f36b7f6a24d11a3d420 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.531877 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 3cd7a465249246c28f07b8fb2e42675f with delete_type TABLET_DATA_DELETED (replaced by tablet 0763a475527546bb83cee8fb823893ac) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.532125 1163 tablet_replica.cc:333] T 3cd7a465249246c28f07b8fb2e42675f P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:34.532318 1163 raft_consensus.cc:2243] T 3cd7a465249246c28f07b8fb2e42675f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.532470 1163 raft_consensus.cc:2272] T 3cd7a465249246c28f07b8fb2e42675f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.533351 1163 ts_tablet_manager.cc:1916] T 3cd7a465249246c28f07b8fb2e42675f P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.538894 1586 raft_consensus.cc:493] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:34.539052 1586 raft_consensus.cc:515] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.539350 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 97ada0641165444a998db26796793507 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.540664 824 heartbeater.cc:344] Connected to a master server at 127.0.104.61:40749
I20251211 20:52:34.543602 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_254d66657cb14d0f8bbf713f794806f6" } modify_external_catalogs: true
I20251211 20:52:34.543825 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_254d66657cb14d0f8bbf713f794806f6" } modify_external_catalogs: true
I20251211 20:52:34.544420 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_5b7fdd64f8f245a69b384dd22aedba9e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:34.544998 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5b7fdd64f8f245a69b384dd22aedba9e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.545084 1586 leader_election.cc:290] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:34.545733 885 tablet_service.cc:1552] Processing DeleteTablet for tablet e1aeae8c18b6499e9dc1accabdc3f633 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:34.547202 1813 raft_consensus.cc:493] T dc5de12701714fcd81b9dba6973a28b5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:34.547341 1813 raft_consensus.cc:515] T dc5de12701714fcd81b9dba6973a28b5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.547935 1813 leader_election.cc:290] T dc5de12701714fcd81b9dba6973a28b5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:34.548581 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dc5de12701714fcd81b9dba6973a28b5" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:34.548890 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dc5de12701714fcd81b9dba6973a28b5" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:34.549114 1046 raft_consensus.cc:2393] T dc5de12701714fcd81b9dba6973a28b5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb0f1933d9a74fccaa4618047fc05c41 in current term 1: Already voted for candidate 6f8d784fd1cd492bbe3809403ec34919 in this term.
I20251211 20:52:34.549520 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "97ada0641165444a998db26796793507" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:34.549381 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "97ada0641165444a998db26796793507" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:34.550915 983 leader_election.cc:343] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:34.551262 982 leader_election.cc:343] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:34.551367 982 leader_election.cc:304] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:34.551505 1164 ts_tablet_manager.cc:1929] T 19d0aa259ac94f36b7f6a24d11a3d420 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.551618 1164 log.cc:1199] T 19d0aa259ac94f36b7f6a24d11a3d420 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/19d0aa259ac94f36b7f6a24d11a3d420
I20251211 20:52:34.552024 847 leader_election.cc:304] T dc5de12701714fcd81b9dba6973a28b5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:34.552071 1164 ts_tablet_manager.cc:1950] T 19d0aa259ac94f36b7f6a24d11a3d420 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:34.552381 1813 raft_consensus.cc:3060] T dc5de12701714fcd81b9dba6973a28b5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:34.552623 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 19d0aa259ac94f36b7f6a24d11a3d420 (table default.Table_ef1c4ca915474d569c91296213346105 [id=9dd03042ff0948b99b39b899255848fe]) successfully deleted
W20251211 20:52:34.553208 582 catalog_manager.cc:4952] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 6d4e6528352f43b6a1b47d7a4b6b3689 because the tablet was not found. No further retry: Not found: Tablet not found: 6d4e6528352f43b6a1b47d7a4b6b3689
W20251211 20:52:34.553810 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 7d1cc9809ab843e693a179a35c53ce25 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d1cc9809ab843e693a179a35c53ce25 already in progress: creating tablet
I20251211 20:52:34.553982 1163 ts_tablet_manager.cc:1929] T 3cd7a465249246c28f07b8fb2e42675f P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20251211 20:52:34.554312 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet d0673c37c2fd47eea8ab10d9e91b8d7a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d0673c37c2fd47eea8ab10d9e91b8d7a already in progress: creating tablet
I20251211 20:52:34.554358 1163 log.cc:1199] T 3cd7a465249246c28f07b8fb2e42675f P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/3cd7a465249246c28f07b8fb2e42675f
I20251211 20:52:34.554664 1813 raft_consensus.cc:2749] T dc5de12701714fcd81b9dba6973a28b5 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:34.554847 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet e1aeae8c18b6499e9dc1accabdc3f633 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e1aeae8c18b6499e9dc1accabdc3f633 already in progress: creating tablet
I20251211 20:52:34.554837 1166 ts_tablet_manager.cc:1434] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.036s user 0.002s sys 0.000s
I20251211 20:52:34.555164 1586 raft_consensus.cc:2749] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:34.555338 1886 consensus_queue.cc:1048] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:34.557152 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a2442a4ebc940a281f9ea1a394f6621 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:34.557479 1165 tablet_replica.cc:333] T 6a2442a4ebc940a281f9ea1a394f6621 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:34.557673 1165 raft_consensus.cc:2243] T 6a2442a4ebc940a281f9ea1a394f6621 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.557822 1165 raft_consensus.cc:2272] T 6a2442a4ebc940a281f9ea1a394f6621 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.558454 1165 ts_tablet_manager.cc:1916] T 6a2442a4ebc940a281f9ea1a394f6621 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.559787 1163 ts_tablet_manager.cc:1950] T 3cd7a465249246c28f07b8fb2e42675f P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:34.560364 1166 tablet_bootstrap.cc:492] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:34.561656 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 3cd7a465249246c28f07b8fb2e42675f (table default.Table_227fbf051d5643528315ca00194583bc [id=1515561ab78d46f6b5aa37a20d90d3f5]) successfully deleted
I20251211 20:52:34.562357 1163 tablet_replica.cc:333] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:34.564514 1163 raft_consensus.cc:2243] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.565285 1165 ts_tablet_manager.cc:1929] T 6a2442a4ebc940a281f9ea1a394f6621 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.565251 1163 raft_consensus.cc:2272] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.566162 1165 log.cc:1199] T 6a2442a4ebc940a281f9ea1a394f6621 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/6a2442a4ebc940a281f9ea1a394f6621
I20251211 20:52:34.566529 1165 ts_tablet_manager.cc:1950] T 6a2442a4ebc940a281f9ea1a394f6621 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:34.567822 1163 ts_tablet_manager.cc:1916] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.568290 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 6a2442a4ebc940a281f9ea1a394f6621 (table default.Table_cbe68d84efef4745822f6aef5a723f11 [id=f9c1230a7d71435cac705d4675a9908d]) successfully deleted
I20251211 20:52:34.571180 1166 tablet_bootstrap.cc:654] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.572026 1046 raft_consensus.cc:1275] T 49a33084c0fc460eb9592224b6dfc279 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Refusing update from remote peer 6f8d784fd1cd492bbe3809403ec34919: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251211 20:52:34.572656 1807 consensus_queue.cc:1048] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:34.574381 913 raft_consensus.cc:3060] T 49a33084c0fc460eb9592224b6dfc279 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:34.579682 1166 tablet_bootstrap.cc:492] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:34.579957 1166 ts_tablet_manager.cc:1403] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.001s
I20251211 20:52:34.581552 961 heartbeater.cc:344] Connected to a master server at 127.0.104.61:40749
I20251211 20:52:34.581698 913 raft_consensus.cc:1275] T 49a33084c0fc460eb9592224b6dfc279 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Refusing update from remote peer 6f8d784fd1cd492bbe3809403ec34919: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251211 20:52:34.581697 1166 raft_consensus.cc:359] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.581962 1166 raft_consensus.cc:385] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.582044 1166 raft_consensus.cc:740] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.582437 1166 consensus_queue.cc:260] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.583464 1166 ts_tablet_manager.cc:1434] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:34.584021 1010 tablet_service.cc:1505] Processing CreateTablet for tablet e218051cdc404af0b0335fa4c373222d (DEFAULT_TABLE table=default.Table_015ac77359ed47db98e074fa602a0141 [id=5db43d4090b947ffa3bdad62184f1db9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.584043 751 tablet_service.cc:1505] Processing CreateTablet for tablet e218051cdc404af0b0335fa4c373222d (DEFAULT_TABLE table=default.Table_015ac77359ed47db98e074fa602a0141 [id=5db43d4090b947ffa3bdad62184f1db9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.584151 891 tablet_service.cc:1505] Processing CreateTablet for tablet 857512c55c644c36bf6cacf046b008eb (DEFAULT_TABLE table=default.Table_91f5eb149cce4e40a34b953dac528702 [id=1bb8d23d94f44ee88ef0aa71624480db]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.584384 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e218051cdc404af0b0335fa4c373222d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.584391 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e218051cdc404af0b0335fa4c373222d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.584525 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 857512c55c644c36bf6cacf046b008eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.587224 1886 consensus_queue.cc:1048] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:34.592886 1166 tablet_bootstrap.cc:492] T fb2d30147cd74d1893783be1e1694d1f P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:34.598235 1163 ts_tablet_manager.cc:1929] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.598740 1163 log.cc:1199] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/97ada0641165444a998db26796793507
I20251211 20:52:34.598711 1166 tablet_bootstrap.cc:654] T fb2d30147cd74d1893783be1e1694d1f P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.599959 1163 ts_tablet_manager.cc:1950] T 97ada0641165444a998db26796793507 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:34.601567 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 97ada0641165444a998db26796793507 (table default.Table_a66a87006b684bbd87431e2aad969dcf [id=021234dec4df4bcb9ec0fee4cb1f887e]) successfully deleted
W20251211 20:52:34.605695 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4cf0cde801bf44449e539d2799b895a7 consensus state has no leader
I20251211 20:52:34.609844 594 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 2630ad57845a4c31b20c8a104dd27335 (table default.Table_4d70f35e05524680b3d72b2e488adbd5 [id=61b2de16fee74487823292913de1b685]). Expected version 3 got 0
I20251211 20:52:34.612020 594 catalog_manager.cc:5654] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2). New cstate: current_term: 1 leader_uuid: "eb0f1933d9a74fccaa4618047fc05c41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:34.613312 1166 tablet_bootstrap.cc:492] T fb2d30147cd74d1893783be1e1694d1f P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:34.613600 1166 ts_tablet_manager.cc:1403] T fb2d30147cd74d1893783be1e1694d1f P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.006s
I20251211 20:52:34.621656 1166 raft_consensus.cc:359] T fb2d30147cd74d1893783be1e1694d1f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.622159 1166 raft_consensus.cc:385] T fb2d30147cd74d1893783be1e1694d1f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.622313 1166 raft_consensus.cc:740] T fb2d30147cd74d1893783be1e1694d1f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.622781 1166 consensus_queue.cc:260] T fb2d30147cd74d1893783be1e1694d1f P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.623970 1166 ts_tablet_manager.cc:1434] T fb2d30147cd74d1893783be1e1694d1f P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20251211 20:52:34.624567 1166 tablet_bootstrap.cc:492] T a9b2aa21ab124b28aebc33adba0583d2 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:34.627693 1166 tablet_bootstrap.cc:654] T a9b2aa21ab124b28aebc33adba0583d2 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.637276 1166 tablet_bootstrap.cc:492] T a9b2aa21ab124b28aebc33adba0583d2 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:34.637723 1166 ts_tablet_manager.cc:1403] T a9b2aa21ab124b28aebc33adba0583d2 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.008s
I20251211 20:52:34.639115 1166 raft_consensus.cc:359] T a9b2aa21ab124b28aebc33adba0583d2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.639521 1166 raft_consensus.cc:385] T a9b2aa21ab124b28aebc33adba0583d2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.639729 1166 raft_consensus.cc:740] T a9b2aa21ab124b28aebc33adba0583d2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.640231 1166 consensus_queue.cc:260] T a9b2aa21ab124b28aebc33adba0583d2 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.641235 1166 ts_tablet_manager.cc:1434] T a9b2aa21ab124b28aebc33adba0583d2 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:34.641800 1166 tablet_bootstrap.cc:492] T 171d5b17dfee49159925b476b08787a4 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:34.644629 1166 tablet_bootstrap.cc:654] T 171d5b17dfee49159925b476b08787a4 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.645423 1168 tablet_bootstrap.cc:492] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:34.645676 1168 ts_tablet_manager.cc:1403] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.153s user 0.039s sys 0.042s
I20251211 20:52:34.652698 1166 tablet_bootstrap.cc:492] T 171d5b17dfee49159925b476b08787a4 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:34.652930 1166 ts_tablet_manager.cc:1403] T 171d5b17dfee49159925b476b08787a4 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.007s
I20251211 20:52:34.653741 1166 raft_consensus.cc:359] T 171d5b17dfee49159925b476b08787a4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.653995 1166 raft_consensus.cc:385] T 171d5b17dfee49159925b476b08787a4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.654096 1166 raft_consensus.cc:740] T 171d5b17dfee49159925b476b08787a4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.654371 1166 consensus_queue.cc:260] T 171d5b17dfee49159925b476b08787a4 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.654894 1166 ts_tablet_manager.cc:1434] T 171d5b17dfee49159925b476b08787a4 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:34.655215 1166 tablet_bootstrap.cc:492] T 33addf3d158941098f240cfcdec45d28 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:34.657461 1166 tablet_bootstrap.cc:654] T 33addf3d158941098f240cfcdec45d28 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.664222 1168 raft_consensus.cc:359] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.665014 1168 raft_consensus.cc:385] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.665280 1168 raft_consensus.cc:740] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.665597 1166 tablet_bootstrap.cc:492] T 33addf3d158941098f240cfcdec45d28 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:34.665773 1166 ts_tablet_manager.cc:1403] T 33addf3d158941098f240cfcdec45d28 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.003s
I20251211 20:52:34.666081 1168 consensus_queue.cc:260] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.667846 1166 raft_consensus.cc:359] T 33addf3d158941098f240cfcdec45d28 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.668059 1166 raft_consensus.cc:385] T 33addf3d158941098f240cfcdec45d28 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.667968 1168 ts_tablet_manager.cc:1434] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.022s user 0.001s sys 0.001s
I20251211 20:52:34.668152 1166 raft_consensus.cc:740] T 33addf3d158941098f240cfcdec45d28 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.668769 1166 consensus_queue.cc:260] T 33addf3d158941098f240cfcdec45d28 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.669267 1166 ts_tablet_manager.cc:1434] T 33addf3d158941098f240cfcdec45d28 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251211 20:52:34.669265 1168 tablet_bootstrap.cc:492] T f7f52d2547534ef99255c5d3f59410c8 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:34.669359 824 heartbeater.cc:344] Connected to a master server at 127.0.104.61:40749
I20251211 20:52:34.669687 1166 tablet_bootstrap.cc:492] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:34.671346 1166 tablet_bootstrap.cc:654] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.671947 589 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 2630ad57845a4c31b20c8a104dd27335 (table default.Table_4d70f35e05524680b3d72b2e488adbd5 [id=61b2de16fee74487823292913de1b685]). Expected version 3 got 0
I20251211 20:52:34.675103 589 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 1843dc4548134f29969d46fbc760f5d7 (table default.Table_99917cb4c70840e19004ad381746d77e [id=6abb187bfd214e91b2e11267065a2841]). Expected version 1 got 0
I20251211 20:52:34.675885 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 97ada0641165444a998db26796793507 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.676338 1168 tablet_bootstrap.cc:654] T f7f52d2547534ef99255c5d3f59410c8 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:34.676741 581 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 97ada0641165444a998db26796793507 because the tablet was not found. No further retry: Not found: Tablet not found: 97ada0641165444a998db26796793507
I20251211 20:52:34.676816 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_c676a5b83a30404b9e0e9b189701845a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:34.677605 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c676a5b83a30404b9e0e9b189701845a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.679277 1937 tablet.cc:1722] T 49a33084c0fc460eb9592224b6dfc279 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20251211 20:52:34.680402 1938 tablet.cc:1722] T 49a33084c0fc460eb9592224b6dfc279 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20251211 20:52:34.692174 1870 raft_consensus.cc:493] T b072b759d1254859b0b8d401e3078792 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:34.692322 1870 raft_consensus.cc:515] T b072b759d1254859b0b8d401e3078792 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.692917 1870 leader_election.cc:290] T b072b759d1254859b0b8d401e3078792 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:34.693674 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b072b759d1254859b0b8d401e3078792" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:34.694314 983 leader_election.cc:343] T b072b759d1254859b0b8d401e3078792 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:34.695232 1807 consensus_queue.cc:1048] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:34.695389 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b072b759d1254859b0b8d401e3078792" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:34.695919 982 leader_election.cc:343] T b072b759d1254859b0b8d401e3078792 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:34.696055 982 leader_election.cc:304] T b072b759d1254859b0b8d401e3078792 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:34.696326 751 tablet_service.cc:1505] Processing CreateTablet for tablet 857512c55c644c36bf6cacf046b008eb (DEFAULT_TABLE table=default.Table_91f5eb149cce4e40a34b953dac528702 [id=1bb8d23d94f44ee88ef0aa71624480db]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.696444 1870 raft_consensus.cc:2749] T b072b759d1254859b0b8d401e3078792 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:34.696801 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 857512c55c644c36bf6cacf046b008eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.700958 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 857512c55c644c36bf6cacf046b008eb (DEFAULT_TABLE table=default.Table_91f5eb149cce4e40a34b953dac528702 [id=1bb8d23d94f44ee88ef0aa71624480db]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.701319 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 857512c55c644c36bf6cacf046b008eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.701236 885 tablet_service.cc:1505] Processing CreateTablet for tablet 2d0fa31361b74760ae7394a11f6fcafa (DEFAULT_TABLE table=default.Table_1ccc66e4393445059e9653a319360528 [id=c57f12c04ee54729b215809907e7f20e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.701606 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d0fa31361b74760ae7394a11f6fcafa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.704663 883 tablet_service.cc:1505] Processing CreateTablet for tablet 6d4e6528352f43b6a1b47d7a4b6b3689 (DEFAULT_TABLE table=default.Table_3fe66404826142609775fed9970bbb7a [id=1e5cd33d082b41f89ceb56a63f55e5e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.705024 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d4e6528352f43b6a1b47d7a4b6b3689. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.705339 751 tablet_service.cc:1505] Processing CreateTablet for tablet 2d0fa31361b74760ae7394a11f6fcafa (DEFAULT_TABLE table=default.Table_1ccc66e4393445059e9653a319360528 [id=c57f12c04ee54729b215809907e7f20e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.705420 756 tablet_service.cc:1505] Processing CreateTablet for tablet 46986e9537d54515b8de7a3c5dcaedf0 (DEFAULT_TABLE table=default.Table_c993b3973aa64eabb16fe02f9514b161 [id=7f24fb8df80943cb836588beda1de7cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.705804 754 tablet_service.cc:1505] Processing CreateTablet for tablet 617db82d7747484a91e17219eedfe971 (DEFAULT_TABLE table=default.Table_e345037da95a4c16b6e8b650f0ba3911 [id=efb341043b8748dd883699a699425bbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.706276 757 tablet_service.cc:1505] Processing CreateTablet for tablet a0618ed6fb164b9b9bcde43149b60c2d (DEFAULT_TABLE table=default.Table_9fb97b3359de4806bf0557739a5da4fd [id=17c25e0db2f9436688a7338067e09aac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.705493 742 tablet_service.cc:1505] Processing CreateTablet for tablet 6d4e6528352f43b6a1b47d7a4b6b3689 (DEFAULT_TABLE table=default.Table_3fe66404826142609775fed9970bbb7a [id=1e5cd33d082b41f89ceb56a63f55e5e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.707417 1013 tablet_service.cc:1505] Processing CreateTablet for tablet 2d0fa31361b74760ae7394a11f6fcafa (DEFAULT_TABLE table=default.Table_1ccc66e4393445059e9653a319360528 [id=c57f12c04ee54729b215809907e7f20e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.707800 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d0fa31361b74760ae7394a11f6fcafa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.707904 879 tablet_service.cc:1505] Processing CreateTablet for tablet 46986e9537d54515b8de7a3c5dcaedf0 (DEFAULT_TABLE table=default.Table_c993b3973aa64eabb16fe02f9514b161 [id=7f24fb8df80943cb836588beda1de7cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.708228 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46986e9537d54515b8de7a3c5dcaedf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.711884 889 tablet_service.cc:1505] Processing CreateTablet for tablet a0618ed6fb164b9b9bcde43149b60c2d (DEFAULT_TABLE table=default.Table_9fb97b3359de4806bf0557739a5da4fd [id=17c25e0db2f9436688a7338067e09aac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.711890 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 6d4e6528352f43b6a1b47d7a4b6b3689 (DEFAULT_TABLE table=default.Table_3fe66404826142609775fed9970bbb7a [id=1e5cd33d082b41f89ceb56a63f55e5e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.712215 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0618ed6fb164b9b9bcde43149b60c2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.712244 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d4e6528352f43b6a1b47d7a4b6b3689. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.712864 1939 tablet.cc:1722] T 49a33084c0fc460eb9592224b6dfc279 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20251211 20:52:34.715888 1011 tablet_service.cc:1505] Processing CreateTablet for tablet 617db82d7747484a91e17219eedfe971 (DEFAULT_TABLE table=default.Table_e345037da95a4c16b6e8b650f0ba3911 [id=efb341043b8748dd883699a699425bbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.716223 1011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 617db82d7747484a91e17219eedfe971. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.716291 888 tablet_service.cc:1505] Processing CreateTablet for tablet 617db82d7747484a91e17219eedfe971 (DEFAULT_TABLE table=default.Table_e345037da95a4c16b6e8b650f0ba3911 [id=efb341043b8748dd883699a699425bbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.716593 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 617db82d7747484a91e17219eedfe971. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.719837 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 46986e9537d54515b8de7a3c5dcaedf0 (DEFAULT_TABLE table=default.Table_c993b3973aa64eabb16fe02f9514b161 [id=7f24fb8df80943cb836588beda1de7cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.720156 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46986e9537d54515b8de7a3c5dcaedf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.705816 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46986e9537d54515b8de7a3c5dcaedf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.725437 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d0fa31361b74760ae7394a11f6fcafa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.727818 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0618ed6fb164b9b9bcde43149b60c2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.730530 1586 consensus_queue.cc:1048] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251211 20:52:34.732705 588 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 48820386e41b47129a13bdd5768f021b consensus state has no leader
I20251211 20:52:34.733949 588 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 1843dc4548134f29969d46fbc760f5d7 (table default.Table_99917cb4c70840e19004ad381746d77e [id=6abb187bfd214e91b2e11267065a2841]). Expected version 1 got 0
I20251211 20:52:34.734913 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d4e6528352f43b6a1b47d7a4b6b3689. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.738055 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 617db82d7747484a91e17219eedfe971. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.741230 588 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet c8e17645a26947ab8c01f9b093fa422c (table default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5]). Expected version 2 got 0
I20251211 20:52:34.741411 588 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 49a33084c0fc460eb9592224b6dfc279 (table default.Table_6b98cb90147a43c4aee6b39240c5edc4 [id=9c24845e4efe475fa792a51ce8a66be6]). Expected version 4 got 0
I20251211 20:52:34.741559 588 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 2630ad57845a4c31b20c8a104dd27335 (table default.Table_4d70f35e05524680b3d72b2e488adbd5 [id=61b2de16fee74487823292913de1b685]). Expected version 3 got 0
I20251211 20:52:34.745301 1012 tablet_service.cc:1552] Processing DeleteTablet for tablet 33addf3d158941098f240cfcdec45d28 with delete_type TABLET_DATA_DELETED (replaced by tablet 7be44df39a8f4d6da793479d425fa684) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.745573 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet a9b2aa21ab124b28aebc33adba0583d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.745760 1028 tablet_service.cc:1552] Processing DeleteTablet for tablet 171d5b17dfee49159925b476b08787a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.745926 1019 tablet_service.cc:1552] Processing DeleteTablet for tablet fb2d30147cd74d1893783be1e1694d1f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.746632 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b0ac4ccf9e224d32af03dcaa1b97330a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:34.747234 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b0ac4ccf9e224d32af03dcaa1b97330a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.747737 588 master_service.cc:954] ReplaceTablet: received request to replace tablet 71dc8d20d93243e289330d7b8c635511 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:34.748279 588 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b40488a7667a4d79a917e877f21cee86" } new_table_name: "default.Table_b008891c4e894b02aa0db32d1740371a" modify_external_catalogs: true
I20251211 20:52:34.748420 597 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet efb6e326080d4ae1bc7ed360e72349a3
I20251211 20:52:34.749281 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_2a25c4a0088d463684aa33e76ad0dd66" } new_table_name: "default.Table_e4656c3f3f974d68af3f207e635d316d" modify_external_catalogs: true
I20251211 20:52:34.750332 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 50f9f29c064c4f938a0f20f302ab5e8a
I20251211 20:52:34.750778 592 catalog_manager.cc:6724] ReplaceTablet: tablet 50f9f29c064c4f938a0f20f302ab5e8a deleted and replaced by tablet da81bf8854a24954ba4548185812295d
I20251211 20:52:34.751072 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_f2dc337cf8ba44408562bb0710ec4527"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:34.751461 1019 tablet_service.cc:1552] Processing DeleteTablet for tablet 50f9f29c064c4f938a0f20f302ab5e8a with delete_type TABLET_DATA_DELETED (replaced by tablet da81bf8854a24954ba4548185812295d) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:34.751567 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f2dc337cf8ba44408562bb0710ec4527 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.754805 1163 tablet_replica.cc:333] T 33addf3d158941098f240cfcdec45d28 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:34.754920 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 50f9f29c064c4f938a0f20f302ab5e8a with delete_type TABLET_DATA_DELETED (replaced by tablet da81bf8854a24954ba4548185812295d) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:34.754999 1163 raft_consensus.cc:2243] T 33addf3d158941098f240cfcdec45d28 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.755146 1163 raft_consensus.cc:2272] T 33addf3d158941098f240cfcdec45d28 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:34.755568 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 50f9f29c064c4f938a0f20f302ab5e8a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 50f9f29c064c4f938a0f20f302ab5e8a already in progress: creating tablet
I20251211 20:52:34.755493 1807 consensus_queue.cc:1048] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:34.756217 1163 ts_tablet_manager.cc:1916] T 33addf3d158941098f240cfcdec45d28 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.764683 886 tablet_service.cc:1505] Processing CreateTablet for tablet 76cfb26762d54593a150ec98125c6592 (DEFAULT_TABLE table=default.Table_25aba30fb89c47c5b77403382971cf65 [id=2169ef888528456ca4508567da12812e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.765091 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76cfb26762d54593a150ec98125c6592. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.773041 1163 ts_tablet_manager.cc:1929] T 33addf3d158941098f240cfcdec45d28 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.773531 879 tablet_service.cc:1505] Processing CreateTablet for tablet 8ed276a0fd3b4d7a8d26b8541e35fc60 (DEFAULT_TABLE table=default.Table_b7695e75618444e5a8e1ae1e2fd02d5b [id=5f737bc407a14f83a660f02af3be8e39]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.774027 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ed276a0fd3b4d7a8d26b8541e35fc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.775525 757 tablet_service.cc:1505] Processing CreateTablet for tablet 76cfb26762d54593a150ec98125c6592 (DEFAULT_TABLE table=default.Table_25aba30fb89c47c5b77403382971cf65 [id=2169ef888528456ca4508567da12812e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.776786 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76cfb26762d54593a150ec98125c6592. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.779170 1163 log.cc:1199] T 33addf3d158941098f240cfcdec45d28 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/33addf3d158941098f240cfcdec45d28
I20251211 20:52:34.776417 1010 tablet_service.cc:1505] Processing CreateTablet for tablet a0618ed6fb164b9b9bcde43149b60c2d (DEFAULT_TABLE table=default.Table_9fb97b3359de4806bf0557739a5da4fd [id=17c25e0db2f9436688a7338067e09aac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.779954 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 50f9f29c064c4f938a0f20f302ab5e8a with delete_type TABLET_DATA_DELETED (replaced by tablet da81bf8854a24954ba4548185812295d) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:34.780032 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0618ed6fb164b9b9bcde43149b60c2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.781309 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 6a526a6d1bc84cefad2c5d897223292e
I20251211 20:52:34.781632 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_5da0e9ddf96549c88878f7136d0ba415"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:34.782009 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a076c34f6d2c4e38b6a1078d322e151c" } modify_external_catalogs: true
W20251211 20:52:34.782161 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5da0e9ddf96549c88878f7136d0ba415 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.782177 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a076c34f6d2c4e38b6a1078d322e151c" } modify_external_catalogs: true
I20251211 20:52:34.776057 1013 tablet_service.cc:1505] Processing CreateTablet for tablet 76cfb26762d54593a150ec98125c6592 (DEFAULT_TABLE table=default.Table_25aba30fb89c47c5b77403382971cf65 [id=2169ef888528456ca4508567da12812e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.784448 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76cfb26762d54593a150ec98125c6592. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.784466 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a526a6d1bc84cefad2c5d897223292e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:34.785245 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 6a526a6d1bc84cefad2c5d897223292e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a526a6d1bc84cefad2c5d897223292e already in progress: creating tablet
I20251211 20:52:34.785585 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a526a6d1bc84cefad2c5d897223292e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:34.786414 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 50f9f29c064c4f938a0f20f302ab5e8a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 50f9f29c064c4f938a0f20f302ab5e8a already in progress: creating tablet
W20251211 20:52:34.786736 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 6a526a6d1bc84cefad2c5d897223292e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a526a6d1bc84cefad2c5d897223292e already in progress: creating tablet
I20251211 20:52:34.787739 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a526a6d1bc84cefad2c5d897223292e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:34.788024 1163 ts_tablet_manager.cc:1950] T 33addf3d158941098f240cfcdec45d28 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:34.788103 590 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3eac21a07ba34e1fa275e25ee9dcabb1 (table default.Table_d571120dedcb48cd9d4b75314be3a654 [id=9700f9fe6e3044748e34954bef8a0038]). Expected version 2 got 0
I20251211 20:52:34.792659 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 33addf3d158941098f240cfcdec45d28 (table default.Table_f6a10556ff104db5bd5ecc9fc380e228 [id=f75813add54e46189ff1bbed82d83ba7]) successfully deleted
I20251211 20:52:34.793097 1163 tablet_replica.cc:333] T a9b2aa21ab124b28aebc33adba0583d2 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:34.793390 1163 raft_consensus.cc:2243] T a9b2aa21ab124b28aebc33adba0583d2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.795778 1586 raft_consensus.cc:493] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:34.796324 1586 raft_consensus.cc:515] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:34.798018 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3fc173c396ff4af5a8589cff18165e85" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:34.798506 982 leader_election.cc:343] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:34.796672 1163 raft_consensus.cc:2272] T a9b2aa21ab124b28aebc33adba0583d2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.800240 1166 tablet_bootstrap.cc:492] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:34.800824 756 tablet_service.cc:1505] Processing CreateTablet for tablet 8ed276a0fd3b4d7a8d26b8541e35fc60 (DEFAULT_TABLE table=default.Table_b7695e75618444e5a8e1ae1e2fd02d5b [id=5f737bc407a14f83a660f02af3be8e39]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.801218 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ed276a0fd3b4d7a8d26b8541e35fc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.804852 594 master_service.cc:954] ReplaceTablet: received request to replace tablet e95b0d8c507a4779a087c8933d464228 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:34.808804 751 tablet_service.cc:1505] Processing CreateTablet for tablet 0add14ccd2af42aeb02fad7a28ed4520 (DEFAULT_TABLE table=default.Table_68f5e21962ce48b89834785bf8dd332f [id=df1df827d2194032b668c7579bc1dcda]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.809188 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0add14ccd2af42aeb02fad7a28ed4520. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.813799 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0d7c51baaa1541b98af0c9b474cf8d66" } new_table_name: "default.Table_909bbe6da1a648d68199fb27424e4342" modify_external_catalogs: true
I20251211 20:52:34.817792 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3fc173c396ff4af5a8589cff18165e85" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:34.818401 983 leader_election.cc:343] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:34.818598 983 leader_election.cc:304] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:34.818226 885 tablet_service.cc:1505] Processing CreateTablet for tablet 0add14ccd2af42aeb02fad7a28ed4520 (DEFAULT_TABLE table=default.Table_68f5e21962ce48b89834785bf8dd332f [id=df1df827d2194032b668c7579bc1dcda]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.801008 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 8ed276a0fd3b4d7a8d26b8541e35fc60 (DEFAULT_TABLE table=default.Table_b7695e75618444e5a8e1ae1e2fd02d5b [id=5f737bc407a14f83a660f02af3be8e39]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.818930 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0add14ccd2af42aeb02fad7a28ed4520. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.819257 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ed276a0fd3b4d7a8d26b8541e35fc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.819201 891 tablet_service.cc:1505] Processing CreateTablet for tablet 6cfa15533fad4aec84f8d9038227faf6 (DEFAULT_TABLE table=default.Table_a076c34f6d2c4e38b6a1078d322e151c [id=7d7aaa95a5924f8b835ed6485c2ec83f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.819613 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cfa15533fad4aec84f8d9038227faf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.824051 1160 tablet_bootstrap.cc:492] T bb5c5523e175406d9f6f9cd2736a2045 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:34.824185 1160 ts_tablet_manager.cc:1403] T bb5c5523e175406d9f6f9cd2736a2045 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.319s user 0.006s sys 0.005s
I20251211 20:52:34.824282 1166 ts_tablet_manager.cc:1403] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.149s user 0.015s sys 0.009s
I20251211 20:52:34.825009 1166 raft_consensus.cc:359] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.825028 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f51ac924827c4dbea700109489db0963" } modify_external_catalogs: true
I20251211 20:52:34.825196 1166 raft_consensus.cc:385] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.825224 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f51ac924827c4dbea700109489db0963" } modify_external_catalogs: true
I20251211 20:52:34.825796 1166 raft_consensus.cc:740] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.826087 1166 consensus_queue.cc:260] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.801560 1163 ts_tablet_manager.cc:1916] T a9b2aa21ab124b28aebc33adba0583d2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.803943 1586 leader_election.cc:290] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:34.815877 1011 tablet_service.cc:1505] Processing CreateTablet for tablet 0add14ccd2af42aeb02fad7a28ed4520 (DEFAULT_TABLE table=default.Table_68f5e21962ce48b89834785bf8dd332f [id=df1df827d2194032b668c7579bc1dcda]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.828038 1011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0add14ccd2af42aeb02fad7a28ed4520. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.829125 756 tablet_service.cc:1505] Processing CreateTablet for tablet 6cfa15533fad4aec84f8d9038227faf6 (DEFAULT_TABLE table=default.Table_a076c34f6d2c4e38b6a1078d322e151c [id=7d7aaa95a5924f8b835ed6485c2ec83f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.829540 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cfa15533fad4aec84f8d9038227faf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.831859 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 6cfa15533fad4aec84f8d9038227faf6 (DEFAULT_TABLE table=default.Table_a076c34f6d2c4e38b6a1078d322e151c [id=7d7aaa95a5924f8b835ed6485c2ec83f]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:34.832240 588 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8229b704e7464b30b7d2902a938ce33b consensus state has no leader
I20251211 20:52:34.832518 1160 raft_consensus.cc:359] T bb5c5523e175406d9f6f9cd2736a2045 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.832676 1160 raft_consensus.cc:385] T bb5c5523e175406d9f6f9cd2736a2045 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.832723 1160 raft_consensus.cc:740] T bb5c5523e175406d9f6f9cd2736a2045 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.832908 1160 consensus_queue.cc:260] T bb5c5523e175406d9f6f9cd2736a2045 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.833467 1160 ts_tablet_manager.cc:1434] T bb5c5523e175406d9f6f9cd2736a2045 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20251211 20:52:34.833895 1160 tablet_bootstrap.cc:492] T d30376baf75b4761971f91da3493a142 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:34.834744 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f5ee1b1b44c645f28737720bc385a577" } modify_external_catalogs: true
I20251211 20:52:34.834837 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cfa15533fad4aec84f8d9038227faf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.834915 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f5ee1b1b44c645f28737720bc385a577" } modify_external_catalogs: true
I20251211 20:52:34.837998 888 tablet_service.cc:1505] Processing CreateTablet for tablet 7bc03ed8c35740f58fe48ddbaa120ea0 (DEFAULT_TABLE table=default.Table_bf0900ef3fb8447c834a43e509caa105 [id=861b00629ceb42588a1d1be34153b792]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.838346 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bc03ed8c35740f58fe48ddbaa120ea0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.841485 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 7bc03ed8c35740f58fe48ddbaa120ea0 (DEFAULT_TABLE table=default.Table_bf0900ef3fb8447c834a43e509caa105 [id=861b00629ceb42588a1d1be34153b792]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.841710 879 tablet_service.cc:1505] Processing CreateTablet for tablet 0f8808bfd68841e4ae30818018040cf9 (DEFAULT_TABLE table=default.Table_bc3ec8261d83495488df9d7b7481de50 [id=7754da45ff5f4174adfbd7cefb26c683]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.841902 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bc03ed8c35740f58fe48ddbaa120ea0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.842089 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f8808bfd68841e4ae30818018040cf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.842936 1160 tablet_bootstrap.cc:654] T d30376baf75b4761971f91da3493a142 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.852985 597 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 3eac21a07ba34e1fa275e25ee9dcabb1 (table default.Table_d571120dedcb48cd9d4b75314be3a654 [id=9700f9fe6e3044748e34954bef8a0038]). Expected version 2 got 0
I20251211 20:52:34.853219 597 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 2630ad57845a4c31b20c8a104dd27335 (table default.Table_4d70f35e05524680b3d72b2e488adbd5 [id=61b2de16fee74487823292913de1b685]). Expected version 3 got 0
I20251211 20:52:34.853410 597 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet dc5de12701714fcd81b9dba6973a28b5 (table default.Table_f572100667d148e8a69e7930596f0f52 [id=cb97ccf262fd4c41a180267fd0678cec]). Expected version 2 got 0
I20251211 20:52:34.854343 888 tablet_service.cc:1505] Processing CreateTablet for tablet 5f2c14c7a972414e8328e0b388bde0a4 (DEFAULT_TABLE table=default.Table_7287bc8f28a64682b352f9adae2ec224 [id=c2883f35f01248219793761e8f0984aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.854717 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f2c14c7a972414e8328e0b388bde0a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.856093 754 tablet_service.cc:1505] Processing CreateTablet for tablet 0f8808bfd68841e4ae30818018040cf9 (DEFAULT_TABLE table=default.Table_bc3ec8261d83495488df9d7b7481de50 [id=7754da45ff5f4174adfbd7cefb26c683]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.856554 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f8808bfd68841e4ae30818018040cf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.860452 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 0f8808bfd68841e4ae30818018040cf9 (DEFAULT_TABLE table=default.Table_bc3ec8261d83495488df9d7b7481de50 [id=7754da45ff5f4174adfbd7cefb26c683]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.860991 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f8808bfd68841e4ae30818018040cf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.862623 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_6cce646a13f6405c96fd61f193add5fa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:34.863202 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6cce646a13f6405c96fd61f193add5fa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.863704 1870 raft_consensus.cc:2749] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:34.864117 742 tablet_service.cc:1505] Processing CreateTablet for tablet 5f2c14c7a972414e8328e0b388bde0a4 (DEFAULT_TABLE table=default.Table_7287bc8f28a64682b352f9adae2ec224 [id=c2883f35f01248219793761e8f0984aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.864518 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f2c14c7a972414e8328e0b388bde0a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.867466 1166 ts_tablet_manager.cc:1434] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.043s user 0.001s sys 0.001s
I20251211 20:52:34.867811 1166 tablet_bootstrap.cc:492] T 648f43e95e034f2188a20e1b837f2818 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:34.887037 1166 tablet_bootstrap.cc:654] T 648f43e95e034f2188a20e1b837f2818 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.888275 1028 tablet_service.cc:1505] Processing CreateTablet for tablet 5f2c14c7a972414e8328e0b388bde0a4 (DEFAULT_TABLE table=default.Table_7287bc8f28a64682b352f9adae2ec224 [id=c2883f35f01248219793761e8f0984aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.888691 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f2c14c7a972414e8328e0b388bde0a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.891690 1163 ts_tablet_manager.cc:1929] T a9b2aa21ab124b28aebc33adba0583d2 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.891847 1163 log.cc:1199] T a9b2aa21ab124b28aebc33adba0583d2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/a9b2aa21ab124b28aebc33adba0583d2
I20251211 20:52:34.891865 1168 tablet_bootstrap.cc:492] T f7f52d2547534ef99255c5d3f59410c8 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:34.892005 1168 ts_tablet_manager.cc:1403] T f7f52d2547534ef99255c5d3f59410c8 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.223s user 0.010s sys 0.018s
I20251211 20:52:34.892163 1163 ts_tablet_manager.cc:1950] T a9b2aa21ab124b28aebc33adba0583d2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:34.892751 1168 raft_consensus.cc:359] T f7f52d2547534ef99255c5d3f59410c8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.892916 1168 raft_consensus.cc:385] T f7f52d2547534ef99255c5d3f59410c8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.892958 1168 raft_consensus.cc:740] T f7f52d2547534ef99255c5d3f59410c8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.893142 1168 consensus_queue.cc:260] T f7f52d2547534ef99255c5d3f59410c8 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.893335 1166 tablet_bootstrap.cc:492] T 648f43e95e034f2188a20e1b837f2818 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:34.893471 1166 ts_tablet_manager.cc:1403] T 648f43e95e034f2188a20e1b837f2818 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.000s
I20251211 20:52:34.893564 1168 ts_tablet_manager.cc:1434] T f7f52d2547534ef99255c5d3f59410c8 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251211 20:52:34.894269 1166 raft_consensus.cc:359] T 648f43e95e034f2188a20e1b837f2818 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.894438 1166 raft_consensus.cc:385] T 648f43e95e034f2188a20e1b837f2818 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.894531 1166 raft_consensus.cc:740] T 648f43e95e034f2188a20e1b837f2818 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.895573 1163 tablet_replica.cc:333] T 171d5b17dfee49159925b476b08787a4 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:34.895704 1168 tablet_bootstrap.cc:492] T 05385f20e88c46108e09a78814811209 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:34.895995 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet a9b2aa21ab124b28aebc33adba0583d2 (table default.Table_b70392ad5fb6451482100395256494bd [id=eafa1350ab384c8cba766017840d59a6]) successfully deleted
I20251211 20:52:34.896101 1163 raft_consensus.cc:2243] T 171d5b17dfee49159925b476b08787a4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.896270 1163 raft_consensus.cc:2272] T 171d5b17dfee49159925b476b08787a4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.895804 1166 consensus_queue.cc:260] T 648f43e95e034f2188a20e1b837f2818 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.896767 1166 ts_tablet_manager.cc:1434] T 648f43e95e034f2188a20e1b837f2818 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:34.897264 1168 tablet_bootstrap.cc:654] T 05385f20e88c46108e09a78814811209 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.903473 1166 tablet_bootstrap.cc:492] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:34.905750 591 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_6b98cb90147a43c4aee6b39240c5edc4 [id=9c24845e4efe475fa792a51ce8a66be6] alter complete (version 4)
I20251211 20:52:34.911908 1163 ts_tablet_manager.cc:1916] T 171d5b17dfee49159925b476b08787a4 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.916096 751 tablet_service.cc:1505] Processing CreateTablet for tablet 7bc03ed8c35740f58fe48ddbaa120ea0 (DEFAULT_TABLE table=default.Table_bf0900ef3fb8447c834a43e509caa105 [id=861b00629ceb42588a1d1be34153b792]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.916615 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bc03ed8c35740f58fe48ddbaa120ea0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.920178 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f572100667d148e8a69e7930596f0f52" } new_table_name: "default.Table_d465fa4868dd4c4db1036653f8b6f463" modify_external_catalogs: true
I20251211 20:52:34.921653 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_87f092378445464ab48c155fae469f29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:34.922225 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_87f092378445464ab48c155fae469f29 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:34.924976 597 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet dc5de12701714fcd81b9dba6973a28b5 (table default.Table_f572100667d148e8a69e7930596f0f52 [id=cb97ccf262fd4c41a180267fd0678cec]). Expected version 2 got 0
I20251211 20:52:34.926896 1163 ts_tablet_manager.cc:1929] T 171d5b17dfee49159925b476b08787a4 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.927186 1166 tablet_bootstrap.cc:654] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.929574 891 tablet_service.cc:1505] Processing CreateTablet for tablet a3a6e2d4b060449f957eb8d6cacee843 (DEFAULT_TABLE table=default.Table_d812d6f6c90842e4821d33226a54ec68 [id=276b9b5c545849dcb1ed38324b7ad540]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.929960 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3a6e2d4b060449f957eb8d6cacee843. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.930253 1168 tablet_bootstrap.cc:492] T 05385f20e88c46108e09a78814811209 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:34.930378 1168 ts_tablet_manager.cc:1403] T 05385f20e88c46108e09a78814811209 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.035s user 0.005s sys 0.003s
I20251211 20:52:34.931072 1168 raft_consensus.cc:359] T 05385f20e88c46108e09a78814811209 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.931237 1168 raft_consensus.cc:385] T 05385f20e88c46108e09a78814811209 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:34.931280 1168 raft_consensus.cc:740] T 05385f20e88c46108e09a78814811209 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:34.931444 1168 consensus_queue.cc:260] T 05385f20e88c46108e09a78814811209 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.931915 1168 ts_tablet_manager.cc:1434] T 05385f20e88c46108e09a78814811209 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251211 20:52:34.932200 1168 tablet_bootstrap.cc:492] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:34.933745 1168 tablet_bootstrap.cc:654] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:34.939152 1937 tablet.cc:1722] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:34.941538 1163 log.cc:1199] T 171d5b17dfee49159925b476b08787a4 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/171d5b17dfee49159925b476b08787a4
I20251211 20:52:34.943535 911 raft_consensus.cc:3060] T cf1201bba4b6421f8d7ce593f4bd892b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:34.943539 1938 tablet.cc:1722] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:34.944166 1163 ts_tablet_manager.cc:1950] T 171d5b17dfee49159925b476b08787a4 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:34.946866 1807 consensus_queue.cc:1048] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:34.949002 1096 heartbeater.cc:344] Connected to a master server at 127.0.104.61:40749
I20251211 20:52:34.949460 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 171d5b17dfee49159925b476b08787a4 (table default.Table_75c7933bd82449a1a24a7481815ee23b [id=4f5722552e534a00874e4113c0dfbaf3]) successfully deleted
I20251211 20:52:34.950626 1163 tablet_replica.cc:333] T fb2d30147cd74d1893783be1e1694d1f P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:34.952186 1163 raft_consensus.cc:2243] T fb2d30147cd74d1893783be1e1694d1f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:34.953336 1163 raft_consensus.cc:2272] T fb2d30147cd74d1893783be1e1694d1f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:34.957654 1166 tablet_bootstrap.cc:492] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:34.960140 1163 ts_tablet_manager.cc:1916] T fb2d30147cd74d1893783be1e1694d1f P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:34.974231 1807 consensus_queue.cc:1048] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:34.976536 754 tablet_service.cc:1505] Processing CreateTablet for tablet a3a6e2d4b060449f957eb8d6cacee843 (DEFAULT_TABLE table=default.Table_d812d6f6c90842e4821d33226a54ec68 [id=276b9b5c545849dcb1ed38324b7ad540]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.978667 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3a6e2d4b060449f957eb8d6cacee843. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.980141 595 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet dc5de12701714fcd81b9dba6973a28b5 (table default.Table_f572100667d148e8a69e7930596f0f52 [id=cb97ccf262fd4c41a180267fd0678cec]). Expected version 2 got 0
I20251211 20:52:34.980391 595 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 3eac21a07ba34e1fa275e25ee9dcabb1 (table default.Table_d571120dedcb48cd9d4b75314be3a654 [id=9700f9fe6e3044748e34954bef8a0038]). Expected version 2 got 0
I20251211 20:52:34.985805 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 6cfa15533fad4aec84f8d9038227faf6
I20251211 20:52:34.986151 1807 raft_consensus.cc:493] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:34.986284 1807 raft_consensus.cc:515] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:34.986830 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 6cfa15533fad4aec84f8d9038227faf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:34.987322 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1843dc4548134f29969d46fbc760f5d7" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:34.987322 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1843dc4548134f29969d46fbc760f5d7" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:34.987521 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 6cfa15533fad4aec84f8d9038227faf6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6cfa15533fad4aec84f8d9038227faf6 already in progress: creating tablet
I20251211 20:52:34.987556 1045 raft_consensus.cc:2468] T 1843dc4548134f29969d46fbc760f5d7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:34.986873 1807 leader_election.cc:290] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:34.987869 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 6cfa15533fad4aec84f8d9038227faf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:34.987910 711 leader_election.cc:343] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:34.988435 710 leader_election.cc:304] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:34.988942 1807 raft_consensus.cc:2804] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:34.989012 1807 raft_consensus.cc:493] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:34.989060 1807 raft_consensus.cc:3060] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:34.989989 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 6cfa15533fad4aec84f8d9038227faf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:34.990693 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 6cfa15533fad4aec84f8d9038227faf6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6cfa15533fad4aec84f8d9038227faf6 already in progress: creating tablet
I20251211 20:52:34.993858 756 tablet_service.cc:1505] Processing CreateTablet for tablet dfeb47dfaa6445309e28715b1b8db2ae (DEFAULT_TABLE table=default.Table_bd0dcfadc59f4ffa81d0004ae868ed3e [id=c07b39a3869a4f3781ebccf30c95d5e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.994333 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfeb47dfaa6445309e28715b1b8db2ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.994438 891 tablet_service.cc:1505] Processing CreateTablet for tablet dfeb47dfaa6445309e28715b1b8db2ae (DEFAULT_TABLE table=default.Table_bd0dcfadc59f4ffa81d0004ae868ed3e [id=c07b39a3869a4f3781ebccf30c95d5e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.994813 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfeb47dfaa6445309e28715b1b8db2ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.997157 1020 tablet_service.cc:1505] Processing CreateTablet for tablet a3a6e2d4b060449f957eb8d6cacee843 (DEFAULT_TABLE table=default.Table_d812d6f6c90842e4821d33226a54ec68 [id=276b9b5c545849dcb1ed38324b7ad540]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:34.997414 1163 ts_tablet_manager.cc:1929] T fb2d30147cd74d1893783be1e1694d1f P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:34.997567 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3a6e2d4b060449f957eb8d6cacee843. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:34.997589 1163 log.cc:1199] T fb2d30147cd74d1893783be1e1694d1f P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/fb2d30147cd74d1893783be1e1694d1f
I20251211 20:52:34.997977 1163 ts_tablet_manager.cc:1950] T fb2d30147cd74d1893783be1e1694d1f P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:34.999109 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet fb2d30147cd74d1893783be1e1694d1f (table default.Table_59fd4cd22d6b4065bc70dc11849fa628 [id=c5b5110e2f1f4eb29d233b6a9eef4737]) successfully deleted
W20251211 20:52:34.999907 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 50f9f29c064c4f938a0f20f302ab5e8a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 50f9f29c064c4f938a0f20f302ab5e8a already in progress: creating tablet
W20251211 20:52:35.000499 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 6a526a6d1bc84cefad2c5d897223292e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a526a6d1bc84cefad2c5d897223292e already in progress: creating tablet
W20251211 20:52:35.001113 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 6cfa15533fad4aec84f8d9038227faf6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6cfa15533fad4aec84f8d9038227faf6 already in progress: creating tablet
I20251211 20:52:35.002609 596 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet dc5de12701714fcd81b9dba6973a28b5 (table default.Table_f572100667d148e8a69e7930596f0f52 [id=cb97ccf262fd4c41a180267fd0678cec]). Expected version 2 got 0
I20251211 20:52:35.002908 1939 tablet.cc:1722] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:35.004163 1807 raft_consensus.cc:515] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.005230 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1843dc4548134f29969d46fbc760f5d7" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:35.005743 1807 leader_election.cc:290] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:35.005941 711 leader_election.cc:343] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.005990 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1843dc4548134f29969d46fbc760f5d7" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:35.006155 1044 raft_consensus.cc:3060] T 1843dc4548134f29969d46fbc760f5d7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:35.008709 1044 raft_consensus.cc:2468] T 1843dc4548134f29969d46fbc760f5d7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:35.009186 710 leader_election.cc:304] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:35.010716 1886 raft_consensus.cc:2804] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:35.010861 1886 raft_consensus.cc:697] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:35.011106 1886 consensus_queue.cc:237] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.012176 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet e95b0d8c507a4779a087c8933d464228
I20251211 20:52:35.012615 594 catalog_manager.cc:6724] ReplaceTablet: tablet e95b0d8c507a4779a087c8933d464228 deleted and replaced by tablet 9c5c937cc7db4ed18fe03f4a8a99a66a
I20251211 20:52:35.014137 757 tablet_service.cc:1552] Processing DeleteTablet for tablet e95b0d8c507a4779a087c8933d464228 with delete_type TABLET_DATA_DELETED (replaced by tablet 9c5c937cc7db4ed18fe03f4a8a99a66a) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:35.014843 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet e95b0d8c507a4779a087c8933d464228 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e95b0d8c507a4779a087c8933d464228 already in progress: creating tablet
I20251211 20:52:35.014901 891 tablet_service.cc:1552] Processing DeleteTablet for tablet e95b0d8c507a4779a087c8933d464228 with delete_type TABLET_DATA_DELETED (replaced by tablet 9c5c937cc7db4ed18fe03f4a8a99a66a) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:35.015820 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet e95b0d8c507a4779a087c8933d464228 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e95b0d8c507a4779a087c8933d464228 already in progress: creating tablet
I20251211 20:52:34.957921 1166 ts_tablet_manager.cc:1403] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.055s user 0.004s sys 0.004s
I20251211 20:52:35.017206 757 tablet_service.cc:1505] Processing CreateTablet for tablet 33fd09ea06ee47908b85911e4ee79708 (DEFAULT_TABLE table=default.Table_f3c10706c1294f978865836c1f29549f [id=30620a01c27f456bac05fa7c02a2cb4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.017616 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33fd09ea06ee47908b85911e4ee79708. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.018198 1166 raft_consensus.cc:359] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.018556 1166 raft_consensus.cc:385] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.018639 1166 raft_consensus.cc:740] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.018997 1166 consensus_queue.cc:260] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.020020 1166 ts_tablet_manager.cc:1434] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:35.020031 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b06a07477d3146c288f85dad5e264daa" } new_table_name: "default.Table_ca767602e99a4dd9a13c30726a3465fa" modify_external_catalogs: true
I20251211 20:52:35.020512 1166 tablet_bootstrap.cc:492] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:35.021344 1028 tablet_service.cc:1505] Processing CreateTablet for tablet dfeb47dfaa6445309e28715b1b8db2ae (DEFAULT_TABLE table=default.Table_bd0dcfadc59f4ffa81d0004ae868ed3e [id=c07b39a3869a4f3781ebccf30c95d5e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.021709 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfeb47dfaa6445309e28715b1b8db2ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.021947 1166 tablet_bootstrap.cc:654] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:35.024497 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 834b84a557b04e78b5d2c8c84520a644 consensus state has no leader
I20251211 20:52:35.028584 888 tablet_service.cc:1552] Processing DeleteTablet for tablet f93006227cc6489e91f3593e11789b54 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:35.028961 1164 tablet_replica.cc:333] T f93006227cc6489e91f3593e11789b54 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:35.029134 1164 raft_consensus.cc:2243] T f93006227cc6489e91f3593e11789b54 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:35.029281 1164 raft_consensus.cc:2272] T f93006227cc6489e91f3593e11789b54 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:35.029808 1164 ts_tablet_manager.cc:1916] T f93006227cc6489e91f3593e11789b54 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:35.029735 1013 tablet_service.cc:1505] Processing CreateTablet for tablet 33fd09ea06ee47908b85911e4ee79708 (DEFAULT_TABLE table=default.Table_f3c10706c1294f978865836c1f29549f [id=30620a01c27f456bac05fa7c02a2cb4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.030164 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33fd09ea06ee47908b85911e4ee79708. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.035775 1164 ts_tablet_manager.cc:1929] T f93006227cc6489e91f3593e11789b54 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:35.035952 1164 log.cc:1199] T f93006227cc6489e91f3593e11789b54 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/f93006227cc6489e91f3593e11789b54
I20251211 20:52:35.036298 1164 ts_tablet_manager.cc:1950] T f93006227cc6489e91f3593e11789b54 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:35.036875 1019 tablet_service.cc:1552] Processing DeleteTablet for tablet e95b0d8c507a4779a087c8933d464228 with delete_type TABLET_DATA_DELETED (replaced by tablet 9c5c937cc7db4ed18fe03f4a8a99a66a) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:35.036921 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet f93006227cc6489e91f3593e11789b54 (table default.Table_4cd9b395cfa94302b298a3df4971d631 [id=42b8ff20f93f4d3fa10fe7ff92dc598d]) successfully deleted
W20251211 20:52:35.037699 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet e95b0d8c507a4779a087c8933d464228 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e95b0d8c507a4779a087c8933d464228 already in progress: creating tablet
I20251211 20:52:35.037616 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_f2eaa62388894072a390cc4c167f3385"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:35.038029 891 tablet_service.cc:1505] Processing CreateTablet for tablet 33fd09ea06ee47908b85911e4ee79708 (DEFAULT_TABLE table=default.Table_f3c10706c1294f978865836c1f29549f [id=30620a01c27f456bac05fa7c02a2cb4d]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:35.038347 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f2eaa62388894072a390cc4c167f3385 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:35.038381 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33fd09ea06ee47908b85911e4ee79708. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.053274 1939 tablet.cc:1722] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:35.054777 1938 tablet.cc:1722] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:35.067293 961 heartbeater.cc:344] Connected to a master server at 127.0.104.61:40749
I20251211 20:52:35.067467 1586 raft_consensus.cc:493] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.067621 1586 raft_consensus.cc:515] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.068162 1586 leader_election.cc:290] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:35.069391 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68be21a25c6743f5b86d518d98fbc35e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:35.071529 982 leader_election.cc:343] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.071940 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68be21a25c6743f5b86d518d98fbc35e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:35.072508 983 leader_election.cc:343] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.072607 983 leader_election.cc:304] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:35.073045 1586 raft_consensus.cc:2749] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:35.074712 1937 tablet.cc:1722] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:35.075361 1166 tablet_bootstrap.cc:492] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:35.076171 1166 ts_tablet_manager.cc:1403] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.056s user 0.003s sys 0.000s
I20251211 20:52:35.081494 1160 tablet_bootstrap.cc:492] T d30376baf75b4761971f91da3493a142 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:35.081755 1160 ts_tablet_manager.cc:1403] T d30376baf75b4761971f91da3493a142 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.248s user 0.003s sys 0.008s
I20251211 20:52:35.082592 1160 raft_consensus.cc:359] T d30376baf75b4761971f91da3493a142 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.082841 1160 raft_consensus.cc:385] T d30376baf75b4761971f91da3493a142 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.082943 1160 raft_consensus.cc:740] T d30376baf75b4761971f91da3493a142 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.083215 1160 consensus_queue.cc:260] T d30376baf75b4761971f91da3493a142 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.082937 1166 raft_consensus.cc:359] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.083866 1160 ts_tablet_manager.cc:1434] T d30376baf75b4761971f91da3493a142 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:35.084157 1160 tablet_bootstrap.cc:492] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:35.083935 1166 raft_consensus.cc:385] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.084328 1166 raft_consensus.cc:740] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.087042 1160 tablet_bootstrap.cc:654] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.087532 1166 consensus_queue.cc:260] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.095945 1166 ts_tablet_manager.cc:1434] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.019s user 0.001s sys 0.001s
I20251211 20:52:35.098615 1166 tablet_bootstrap.cc:492] T e28d66ce4711430cb09a9650e99fbe9e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:35.102205 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 60d966c0397c4a4ea3718445e87e9703
I20251211 20:52:35.103210 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 60d966c0397c4a4ea3718445e87e9703 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:35.103936 1168 tablet_bootstrap.cc:492] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:35.104085 1168 ts_tablet_manager.cc:1403] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.172s user 0.016s sys 0.017s
I20251211 20:52:35.104307 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 60d966c0397c4a4ea3718445e87e9703 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:35.105229 1168 raft_consensus.cc:359] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.105566 1168 raft_consensus.cc:385] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.105500 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_f520281d794144b5b124b2a218ef3fea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:35.105681 1168 raft_consensus.cc:740] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.105914 1168 consensus_queue.cc:260] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:35.106160 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f520281d794144b5b124b2a218ef3fea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:35.106364 1168 ts_tablet_manager.cc:1434] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:35.113653 1168 tablet_bootstrap.cc:492] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:35.111145 1166 tablet_bootstrap.cc:654] T e28d66ce4711430cb09a9650e99fbe9e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:35.114212 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 60d966c0397c4a4ea3718445e87e9703 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60d966c0397c4a4ea3718445e87e9703 already in progress: creating tablet
I20251211 20:52:35.115255 1168 tablet_bootstrap.cc:654] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.111274 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 60d966c0397c4a4ea3718445e87e9703 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:35.118261 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 2e29df291d8d40daba83e1d9f97f902b
I20251211 20:52:35.118479 1163 tablet_replica.cc:333] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:35.118652 1163 raft_consensus.cc:2243] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:35.118808 1163 raft_consensus.cc:2272] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:35.119318 1163 ts_tablet_manager.cc:1916] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:35.122287 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8f7e347b9ac64793b1073dfa955ebe21" } modify_external_catalogs: true
I20251211 20:52:35.122510 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8f7e347b9ac64793b1073dfa955ebe21" } modify_external_catalogs: true
W20251211 20:52:35.125226 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 60d966c0397c4a4ea3718445e87e9703 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60d966c0397c4a4ea3718445e87e9703 already in progress: creating tablet
I20251211 20:52:35.125629 1019 tablet_service.cc:1552] Processing DeleteTablet for tablet 2e29df291d8d40daba83e1d9f97f902b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:35.127285 1163 ts_tablet_manager.cc:1929] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:35.127454 1163 log.cc:1199] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/60d966c0397c4a4ea3718445e87e9703
I20251211 20:52:35.127831 1163 ts_tablet_manager.cc:1950] T 60d966c0397c4a4ea3718445e87e9703 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:35.128419 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 60d966c0397c4a4ea3718445e87e9703 (table default.Table_f51ac924827c4dbea700109489db0963 [id=cae3f6884694487e82fc81328e9527d1]) successfully deleted
W20251211 20:52:35.129086 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 2e29df291d8d40daba83e1d9f97f902b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2e29df291d8d40daba83e1d9f97f902b already in progress: creating tablet
I20251211 20:52:35.129482 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 2e29df291d8d40daba83e1d9f97f902b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:35.130136 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 2e29df291d8d40daba83e1d9f97f902b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2e29df291d8d40daba83e1d9f97f902b already in progress: creating tablet
I20251211 20:52:35.130488 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 2e29df291d8d40daba83e1d9f97f902b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:35.131114 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 2e29df291d8d40daba83e1d9f97f902b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2e29df291d8d40daba83e1d9f97f902b already in progress: creating tablet
I20251211 20:52:35.134871 1160 tablet_bootstrap.cc:492] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:35.135066 1160 ts_tablet_manager.cc:1403] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.051s user 0.002s sys 0.002s
I20251211 20:52:35.135914 1160 raft_consensus.cc:359] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.136152 1160 raft_consensus.cc:385] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.136245 1160 raft_consensus.cc:740] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.136479 1160 consensus_queue.cc:260] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.136994 1160 ts_tablet_manager.cc:1434] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:35.137320 1160 tablet_bootstrap.cc:492] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:35.139416 1160 tablet_bootstrap.cc:654] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.139612 1166 tablet_bootstrap.cc:492] T e28d66ce4711430cb09a9650e99fbe9e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:35.139834 1166 ts_tablet_manager.cc:1403] T e28d66ce4711430cb09a9650e99fbe9e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.042s user 0.005s sys 0.000s
I20251211 20:52:35.142418 1166 raft_consensus.cc:359] T e28d66ce4711430cb09a9650e99fbe9e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.143055 1166 raft_consensus.cc:385] T e28d66ce4711430cb09a9650e99fbe9e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.143306 1166 raft_consensus.cc:740] T e28d66ce4711430cb09a9650e99fbe9e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.146024 1168 tablet_bootstrap.cc:492] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:35.146175 1168 ts_tablet_manager.cc:1403] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.033s user 0.001s sys 0.008s
I20251211 20:52:35.146931 1168 raft_consensus.cc:359] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.147106 1168 raft_consensus.cc:385] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.147164 1168 raft_consensus.cc:740] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.147431 1168 consensus_queue.cc:260] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.147929 1168 ts_tablet_manager.cc:1434] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251211 20:52:35.148308 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_a05c8b486c1940c29db8cbb1ea1dfb17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:35.148358 1166 consensus_queue.cc:260] T e28d66ce4711430cb09a9650e99fbe9e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:35.148885 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a05c8b486c1940c29db8cbb1ea1dfb17 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:35.149745 1168 tablet_bootstrap.cc:492] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:35.151703 1168 tablet_bootstrap.cc:654] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.151226 1166 ts_tablet_manager.cc:1434] T e28d66ce4711430cb09a9650e99fbe9e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20251211 20:52:35.154106 1166 tablet_bootstrap.cc:492] T 1a77e3e744ff42ccb68897704b4c2f1d P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:35.156503 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_7730b7f512954bcab4c83c9f272d9a59"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:35.157080 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7730b7f512954bcab4c83c9f272d9a59 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:35.159044 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8044d887ac4b408283fbb97b19ac6af6" } modify_external_catalogs: true
I20251211 20:52:35.159173 1019 tablet_service.cc:1552] Processing DeleteTablet for tablet 68be21a25c6743f5b86d518d98fbc35e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:35.159217 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8044d887ac4b408283fbb97b19ac6af6" } modify_external_catalogs: true
I20251211 20:52:35.159421 1163 tablet_replica.cc:333] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:35.159641 1163 raft_consensus.cc:2243] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:35.159811 1163 raft_consensus.cc:2272] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:35.160909 1163 ts_tablet_manager.cc:1916] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:35.161074 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 822abda3c85848728205b71847ef7029 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:35.162030 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_fb5ab0d4096b4494b7ca5a55deb992c2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:35.162549 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fb5ab0d4096b4494b7ca5a55deb992c2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:35.164247 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_7e3bb905c8ec4e96809e6b7d4e44a3b7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:35.164731 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e3bb905c8ec4e96809e6b7d4e44a3b7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:35.165323 756 tablet_service.cc:1552] Processing DeleteTablet for tablet f7f52d2547534ef99255c5d3f59410c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:35.165573 1165 tablet_replica.cc:333] T f7f52d2547534ef99255c5d3f59410c8 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:35.165733 1165 raft_consensus.cc:2243] T f7f52d2547534ef99255c5d3f59410c8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:35.165886 1165 raft_consensus.cc:2272] T f7f52d2547534ef99255c5d3f59410c8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:35.167718 1168 tablet_bootstrap.cc:492] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:35.167853 1168 ts_tablet_manager.cc:1403] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.004s
I20251211 20:52:35.169318 1166 tablet_bootstrap.cc:654] T 1a77e3e744ff42ccb68897704b4c2f1d P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.169943 1163 ts_tablet_manager.cc:1929] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:35.170039 1163 log.cc:1199] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/68be21a25c6743f5b86d518d98fbc35e
I20251211 20:52:35.170300 1163 ts_tablet_manager.cc:1950] T 68be21a25c6743f5b86d518d98fbc35e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:35.171300 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 68be21a25c6743f5b86d518d98fbc35e (table default.Table_afd3c9ee35ad41caa1717a299443409e [id=b0ebaab8cd5a4deb9a5ea1e7bd3ccec3]) successfully deleted
I20251211 20:52:35.173013 1165 ts_tablet_manager.cc:1916] T f7f52d2547534ef99255c5d3f59410c8 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:35.173015 1168 raft_consensus.cc:359] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.173238 1168 raft_consensus.cc:385] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.173280 1168 raft_consensus.cc:740] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.173478 1168 consensus_queue.cc:260] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.173952 1168 ts_tablet_manager.cc:1434] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20251211 20:52:35.174225 1168 tablet_bootstrap.cc:492] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:35.175868 1168 tablet_bootstrap.cc:654] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:35.178830 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 449921ea74f04a9c88eb9e094e8226c8 consensus state has no leader
I20251211 20:52:35.178989 1937 tablet.cc:1722] T dc5de12701714fcd81b9dba6973a28b5 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:35.179744 891 tablet_service.cc:1552] Processing DeleteTablet for tablet bb5c5523e175406d9f6f9cd2736a2045 with delete_type TABLET_DATA_DELETED (replaced by tablet cce44b80c97948e1af7ac68e77cccfd6) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:35.180068 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 648f43e95e034f2188a20e1b837f2818 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:35.180721 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_bc3ec8261d83495488df9d7b7481de50" } new_table_name: "default.Table_964232e162ed4d26ba69b353b2f2065e" modify_external_catalogs: true
I20251211 20:52:35.181917 1165 ts_tablet_manager.cc:1929] T f7f52d2547534ef99255c5d3f59410c8 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:35.182019 1165 log.cc:1199] T f7f52d2547534ef99255c5d3f59410c8 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/f7f52d2547534ef99255c5d3f59410c8
I20251211 20:52:35.182262 1165 ts_tablet_manager.cc:1950] T f7f52d2547534ef99255c5d3f59410c8 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:35.184173 1938 tablet.cc:1722] T dc5de12701714fcd81b9dba6973a28b5 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:35.184377 1163 tablet_replica.cc:333] T 648f43e95e034f2188a20e1b837f2818 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:35.184551 1163 raft_consensus.cc:2243] T 648f43e95e034f2188a20e1b837f2818 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:35.184689 1163 raft_consensus.cc:2272] T 648f43e95e034f2188a20e1b837f2818 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:35.188385 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet f7f52d2547534ef99255c5d3f59410c8 (table default.Table_b3e8496af4d340218ce858c81796a29c [id=ecb6d90c41224c2d8f771a78922eaf05]) successfully deleted
I20251211 20:52:35.188846 1939 tablet.cc:1722] T dc5de12701714fcd81b9dba6973a28b5 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:35.189253 1163 ts_tablet_manager.cc:1916] T 648f43e95e034f2188a20e1b837f2818 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:35.200126 1163 ts_tablet_manager.cc:1929] T 648f43e95e034f2188a20e1b837f2818 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:35.200495 1163 log.cc:1199] T 648f43e95e034f2188a20e1b837f2818 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/648f43e95e034f2188a20e1b837f2818
I20251211 20:52:35.201427 1163 ts_tablet_manager.cc:1950] T 648f43e95e034f2188a20e1b837f2818 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:35.202713 1164 tablet_replica.cc:333] T bb5c5523e175406d9f6f9cd2736a2045 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:35.202939 1164 raft_consensus.cc:2243] T bb5c5523e175406d9f6f9cd2736a2045 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:35.203095 1164 raft_consensus.cc:2272] T bb5c5523e175406d9f6f9cd2736a2045 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:35.204123 1164 ts_tablet_manager.cc:1916] T bb5c5523e175406d9f6f9cd2736a2045 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:35.204439 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 648f43e95e034f2188a20e1b837f2818 (table default.Table_fc80da3fac43460dabeea7e43bf098df [id=75a5a167f3f84710afbf197ea1eff3cc]) successfully deleted
I20251211 20:52:35.209609 1168 tablet_bootstrap.cc:492] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:35.209764 1168 ts_tablet_manager.cc:1403] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.036s user 0.006s sys 0.001s
I20251211 20:52:35.210381 1164 ts_tablet_manager.cc:1929] T bb5c5523e175406d9f6f9cd2736a2045 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:35.210556 1164 log.cc:1199] T bb5c5523e175406d9f6f9cd2736a2045 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/bb5c5523e175406d9f6f9cd2736a2045
I20251211 20:52:35.210589 1168 raft_consensus.cc:359] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.210762 1168 raft_consensus.cc:385] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.210827 1168 raft_consensus.cc:740] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.211061 1164 ts_tablet_manager.cc:1950] T bb5c5523e175406d9f6f9cd2736a2045 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:35.211041 1168 consensus_queue.cc:260] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.211514 1168 ts_tablet_manager.cc:1434] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:35.211802 1168 tablet_bootstrap.cc:492] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:35.212260 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet bb5c5523e175406d9f6f9cd2736a2045 (table default.Table_32ebd46bd3cc460986098c79c9b67ca7 [id=1bd6b7930f2447659da365b7c06b1640]) successfully deleted
I20251211 20:52:35.213433 1168 tablet_bootstrap.cc:654] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.221349 1168 tablet_bootstrap.cc:492] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:35.221582 1168 ts_tablet_manager.cc:1403] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.003s
I20251211 20:52:35.222452 1168 raft_consensus.cc:359] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.223295 1168 raft_consensus.cc:385] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.223508 1168 raft_consensus.cc:740] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.225515 1168 consensus_queue.cc:260] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.226393 1168 ts_tablet_manager.cc:1434] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251211 20:52:35.226816 1168 tablet_bootstrap.cc:492] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:35.242635 1160 tablet_bootstrap.cc:492] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:35.242899 1160 ts_tablet_manager.cc:1403] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.106s user 0.009s sys 0.000s
I20251211 20:52:35.243330 1168 tablet_bootstrap.cc:654] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.243888 1160 raft_consensus.cc:359] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.244135 1160 raft_consensus.cc:385] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.244238 1160 raft_consensus.cc:740] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.244547 1160 consensus_queue.cc:260] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.248513 1160 ts_tablet_manager.cc:1434] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251211 20:52:35.252126 1160 tablet_bootstrap.cc:492] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:35.255831 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 83067b139f504bc191a515feed0ec771
I20251211 20:52:35.256908 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 83067b139f504bc191a515feed0ec771 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:35.257297 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 83067b139f504bc191a515feed0ec771 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:35.258035 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 83067b139f504bc191a515feed0ec771 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 83067b139f504bc191a515feed0ec771 already in progress: creating tablet
W20251211 20:52:35.258294 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 83067b139f504bc191a515feed0ec771 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 83067b139f504bc191a515feed0ec771 already in progress: creating tablet
I20251211 20:52:35.259560 1168 tablet_bootstrap.cc:492] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:35.259763 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e891143fbbcd41509b6372a316eaac2c" } new_table_name: "default.Table_d231bb862aa24617b1f92793007bf755" modify_external_catalogs: true
I20251211 20:52:35.259868 1168 ts_tablet_manager.cc:1403] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.033s user 0.001s sys 0.003s
I20251211 20:52:35.260336 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 83067b139f504bc191a515feed0ec771 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:35.260989 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 83067b139f504bc191a515feed0ec771 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 83067b139f504bc191a515feed0ec771 already in progress: creating tablet
I20251211 20:52:35.261237 1168 raft_consensus.cc:359] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.261513 1168 raft_consensus.cc:385] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.261600 1168 raft_consensus.cc:740] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.261972 1168 consensus_queue.cc:260] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.262544 1160 tablet_bootstrap.cc:654] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.262851 1168 ts_tablet_manager.cc:1434] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:35.263461 1168 tablet_bootstrap.cc:492] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:35.266945 1168 tablet_bootstrap.cc:654] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.275310 1168 tablet_bootstrap.cc:492] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:35.275619 1168 ts_tablet_manager.cc:1403] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.001s
I20251211 20:52:35.277748 1168 raft_consensus.cc:359] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.278105 1168 raft_consensus.cc:385] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.278216 1168 raft_consensus.cc:740] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.278566 1168 consensus_queue.cc:260] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.279556 1168 ts_tablet_manager.cc:1434] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20251211 20:52:35.280517 1168 tablet_bootstrap.cc:492] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:35.278972 1904 log.cc:826] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Log is configured to *not* fsync() on all Append() calls
I20251211 20:52:35.283706 1168 tablet_bootstrap.cc:654] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.288605 1168 tablet_bootstrap.cc:492] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:35.288870 1168 ts_tablet_manager.cc:1403] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20251211 20:52:35.290585 1168 raft_consensus.cc:359] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.291087 1168 raft_consensus.cc:385] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.291219 1168 raft_consensus.cc:740] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.291810 1168 consensus_queue.cc:260] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.292829 1168 ts_tablet_manager.cc:1434] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20251211 20:52:35.293710 1160 tablet_bootstrap.cc:492] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:35.293905 1160 ts_tablet_manager.cc:1403] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.042s user 0.005s sys 0.002s
I20251211 20:52:35.294687 1160 raft_consensus.cc:359] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.294854 1168 tablet_bootstrap.cc:492] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:35.295537 1160 raft_consensus.cc:385] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.296690 1160 raft_consensus.cc:740] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.296980 1160 consensus_queue.cc:260] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.297528 1160 ts_tablet_manager.cc:1434] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251211 20:52:35.297806 1160 tablet_bootstrap.cc:492] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:35.299417 1160 tablet_bootstrap.cc:654] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.302078 1168 tablet_bootstrap.cc:654] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.328668 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_72c2c33862a149418a3c31c07caf8a37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:35.329103 1166 tablet_bootstrap.cc:492] T 1a77e3e744ff42ccb68897704b4c2f1d P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
W20251211 20:52:35.329339 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_72c2c33862a149418a3c31c07caf8a37 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:35.329366 1166 ts_tablet_manager.cc:1403] T 1a77e3e744ff42ccb68897704b4c2f1d P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.176s user 0.054s sys 0.043s
I20251211 20:52:35.330737 1166 raft_consensus.cc:359] T 1a77e3e744ff42ccb68897704b4c2f1d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.331048 1166 raft_consensus.cc:385] T 1a77e3e744ff42ccb68897704b4c2f1d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.331125 1166 raft_consensus.cc:740] T 1a77e3e744ff42ccb68897704b4c2f1d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.331466 1166 consensus_queue.cc:260] T 1a77e3e744ff42ccb68897704b4c2f1d P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.332334 1166 ts_tablet_manager.cc:1434] T 1a77e3e744ff42ccb68897704b4c2f1d P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:35.332885 1166 tablet_bootstrap.cc:492] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:35.333103 1160 tablet_bootstrap.cc:492] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:35.333288 1160 ts_tablet_manager.cc:1403] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.036s user 0.003s sys 0.001s
I20251211 20:52:35.334077 1160 raft_consensus.cc:359] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.334324 1160 raft_consensus.cc:385] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.334422 1160 raft_consensus.cc:740] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.334758 1160 consensus_queue.cc:260] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.335268 1160 ts_tablet_manager.cc:1434] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:35.335521 1166 tablet_bootstrap.cc:654] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.339888 1160 tablet_bootstrap.cc:492] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:35.340781 591 master_service.cc:954] ReplaceTablet: received request to replace tablet 95866de91f9d437d80f40b7962253610 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:35.342247 1160 tablet_bootstrap.cc:654] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.350657 591 master_service.cc:954] ReplaceTablet: received request to replace tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:35.351188 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_c0c0556e591f4b1eb2f215439d385a47"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:35.351823 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c0c0556e591f4b1eb2f215439d385a47 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:35.358281 1166 tablet_bootstrap.cc:492] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:35.358559 1166 ts_tablet_manager.cc:1403] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.026s user 0.009s sys 0.012s
I20251211 20:52:35.359781 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 8b1ce6d2a12a4c7ea044b349412b4bf5
I20251211 20:52:35.360426 597 master_service.cc:954] ReplaceTablet: received request to replace tablet 4df68d1b6a91458e9f212ae81aa25438 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:35.360008 1166 raft_consensus.cc:359] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.360607 1166 raft_consensus.cc:385] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.360654 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b1ce6d2a12a4c7ea044b349412b4bf5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:35.360692 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b1ce6d2a12a4c7ea044b349412b4bf5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:35.360728 1166 raft_consensus.cc:740] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.361053 1166 consensus_queue.cc:260] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.361198 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b1ce6d2a12a4c7ea044b349412b4bf5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:35.361335 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 8b1ce6d2a12a4c7ea044b349412b4bf5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b1ce6d2a12a4c7ea044b349412b4bf5 already in progress: creating tablet
I20251211 20:52:35.361896 1166 ts_tablet_manager.cc:1434] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251211 20:52:35.362303 1166 tablet_bootstrap.cc:492] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:35.362371 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 8b1ce6d2a12a4c7ea044b349412b4bf5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b1ce6d2a12a4c7ea044b349412b4bf5 already in progress: creating tablet
I20251211 20:52:35.362831 1096 heartbeater.cc:344] Connected to a master server at 127.0.104.61:40749
I20251211 20:52:35.365293 1160 tablet_bootstrap.cc:492] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:35.365482 1160 ts_tablet_manager.cc:1403] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.002s
I20251211 20:52:35.365531 1166 tablet_bootstrap.cc:654] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.366439 1160 raft_consensus.cc:359] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.366690 1160 raft_consensus.cc:385] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.366798 1160 raft_consensus.cc:740] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.367125 1160 consensus_queue.cc:260] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.367717 1160 ts_tablet_manager.cc:1434] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:35.371871 1160 tablet_bootstrap.cc:492] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:35.371837 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 8b1ce6d2a12a4c7ea044b349412b4bf5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b1ce6d2a12a4c7ea044b349412b4bf5 already in progress: creating tablet
I20251211 20:52:35.373672 1160 tablet_bootstrap.cc:654] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.374326 824 heartbeater.cc:344] Connected to a master server at 127.0.104.61:40749
I20251211 20:52:35.378521 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_06f4a9c6bfcd4fac94df56806a29ec03" } modify_external_catalogs: true
I20251211 20:52:35.378757 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_06f4a9c6bfcd4fac94df56806a29ec03" } modify_external_catalogs: true
I20251211 20:52:35.389328 1160 tablet_bootstrap.cc:492] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:35.389485 1160 ts_tablet_manager.cc:1403] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.000s
I20251211 20:52:35.390328 1160 raft_consensus.cc:359] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.391227 1160 raft_consensus.cc:385] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.391294 1160 raft_consensus.cc:740] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.391498 1160 consensus_queue.cc:260] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.392022 1160 ts_tablet_manager.cc:1434] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:35.392292 1160 tablet_bootstrap.cc:492] T 1843dc4548134f29969d46fbc760f5d7 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:35.394243 1160 tablet_bootstrap.cc:654] T 1843dc4548134f29969d46fbc760f5d7 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.394119 595 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_7ecc6c3df6214115bf3e0aa5d1ffbb48" } new_table_name: "default.Table_690a1a1fd5d848818451819f38fc1470" modify_external_catalogs: true
W20251211 20:52:35.400408 581 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 449921ea74f04a9c88eb9e094e8226c8 (table default.Table_ca767602e99a4dd9a13c30726a3465fa [id=5fdca585035347b183d092e5a3858dc7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 449921ea74f04a9c88eb9e094e8226c8 consensus state has no leader
I20251211 20:52:35.408846 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8d1a1564105844ed8c61257caa9e03e9" } modify_external_catalogs: true
I20251211 20:52:35.409051 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8d1a1564105844ed8c61257caa9e03e9" } modify_external_catalogs: true
I20251211 20:52:35.413218 1696 raft_consensus.cc:493] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.413228 1166 tablet_bootstrap.cc:492] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:35.413386 1696 raft_consensus.cc:515] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.413539 1166 ts_tablet_manager.cc:1403] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.051s user 0.012s sys 0.030s
I20251211 20:52:35.414052 1696 leader_election.cc:290] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:35.414930 1166 raft_consensus.cc:359] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:35.415145 710 leader_election.cc:343] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 3c578d81e8dd41aaa0f707c4e3fed225
I20251211 20:52:35.415215 1166 raft_consensus.cc:385] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.415318 1166 raft_consensus.cc:740] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.415683 1166 consensus_queue.cc:260] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:35.416489 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0f8808bfd68841e4ae30818018040cf9 consensus state has no leader
I20251211 20:52:35.416484 1166 ts_tablet_manager.cc:1434] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:35.416797 589 master_service.cc:954] ReplaceTablet: received request to replace tablet 97ada0641165444a998db26796793507 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:35.417017 1166 tablet_bootstrap.cc:492] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:35.419633 1166 tablet_bootstrap.cc:654] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.421842 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_55b2abeacec041418b6359c21e87b3d8" } modify_external_catalogs: true
I20251211 20:52:35.422070 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_55b2abeacec041418b6359c21e87b3d8" } modify_external_catalogs: true
I20251211 20:52:35.430677 1166 tablet_bootstrap.cc:492] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:35.430917 1166 ts_tablet_manager.cc:1403] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20251211 20:52:35.431468 1813 raft_consensus.cc:493] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.431617 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3c578d81e8dd41aaa0f707c4e3fed225" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:35.431641 1813 raft_consensus.cc:515] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.432147 1166 raft_consensus.cc:359] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.432405 1166 raft_consensus.cc:385] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.432487 1166 raft_consensus.cc:740] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.432618 1813 leader_election.cc:290] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:35.432798 1166 consensus_queue.cc:260] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.433591 1166 ts_tablet_manager.cc:1434] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:35.434096 1166 tablet_bootstrap.cc:492] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:35.434155 847 leader_election.cc:343] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 6a2442a4ebc940a281f9ea1a394f6621
W20251211 20:52:35.434850 711 leader_election.cc:343] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.434968 711 leader_election.cc:304] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:35.435179 1696 raft_consensus.cc:2749] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:35.436250 847 leader_election.cc:343] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 6a2442a4ebc940a281f9ea1a394f6621
I20251211 20:52:35.436349 847 leader_election.cc:304] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:35.436715 1166 tablet_bootstrap.cc:654] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.437245 1813 raft_consensus.cc:2749] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:35.441782 1586 raft_consensus.cc:493] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.441943 1586 raft_consensus.cc:515] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.442574 1586 leader_election.cc:290] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:35.443426 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6c64fd8dd49b4e5d91c1ffb0fc2559e0" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:35.444583 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6c64fd8dd49b4e5d91c1ffb0fc2559e0" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:35.445214 983 leader_election.cc:343] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:35.445497 982 leader_election.cc:343] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.445766 982 leader_election.cc:304] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:35.446265 1586 raft_consensus.cc:2749] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:35.448655 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1c4b2c22bd4c456eb77af45576de466f consensus state has no leader
I20251211 20:52:35.451999 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_88717d63417a4513b501264b4cf6f5be"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:35.455299 1813 raft_consensus.cc:493] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.455448 1813 raft_consensus.cc:515] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.456125 1813 leader_election.cc:290] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:35.459981 1696 raft_consensus.cc:493] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.460155 1696 raft_consensus.cc:515] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.460784 1696 leader_election.cc:290] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:35.461283 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d762b4afb5ff447dac3468393826205a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:35.461794 711 leader_election.cc:343] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:35.462718 710 leader_election.cc:343] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: d762b4afb5ff447dac3468393826205a
I20251211 20:52:35.462829 710 leader_election.cc:304] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:35.462924 1586 raft_consensus.cc:493] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.463083 1586 raft_consensus.cc:515] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.463683 1586 leader_election.cc:290] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:35.464114 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8e17645a26947ab8c01f9b093fa422c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:35.464214 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8e17645a26947ab8c01f9b093fa422c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:35.464654 982 leader_election.cc:343] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:35.464825 983 leader_election.cc:343] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.464867 1696 raft_consensus.cc:2749] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:35.464912 983 leader_election.cc:304] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:35.465227 1586 raft_consensus.cc:2749] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:35.465245 847 leader_election.cc:343] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 804a11c8f7414eb1a6bcff2952be9ae2
W20251211 20:52:35.465509 847 leader_election.cc:343] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 804a11c8f7414eb1a6bcff2952be9ae2
I20251211 20:52:35.465656 847 leader_election.cc:304] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:35.465757 1166 tablet_bootstrap.cc:492] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:35.465988 1166 ts_tablet_manager.cc:1403] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.032s user 0.008s sys 0.018s
I20251211 20:52:35.466044 1813 raft_consensus.cc:2749] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:35.466620 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ca2877f9e5004eada6f91e1d7bc95c10" } new_table_name: "default.Table_6402a801d48e4a53827b1b62b6cdaba1" modify_external_catalogs: true
I20251211 20:52:35.467339 1166 raft_consensus.cc:359] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.467631 1166 raft_consensus.cc:385] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.467717 1166 raft_consensus.cc:740] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.468026 1166 consensus_queue.cc:260] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.470327 1166 ts_tablet_manager.cc:1434] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:35.470798 1166 tablet_bootstrap.cc:492] T 8229b704e7464b30b7d2902a938ce33b P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:35.475148 1166 tablet_bootstrap.cc:654] T 8229b704e7464b30b7d2902a938ce33b P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.477342 1696 raft_consensus.cc:493] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.477492 1696 raft_consensus.cc:515] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.478111 1696 leader_election.cc:290] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:35.478868 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "55c40b2464f8444ca6c9fa0400f7f7f2" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:35.479437 711 leader_election.cc:343] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:35.480211 710 leader_election.cc:343] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 55c40b2464f8444ca6c9fa0400f7f7f2
I20251211 20:52:35.480324 710 leader_election.cc:304] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:35.480788 1696 raft_consensus.cc:2749] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:35.485738 1813 consensus_queue.cc:1048] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:35.487300 1166 tablet_bootstrap.cc:492] T 8229b704e7464b30b7d2902a938ce33b P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:35.487489 1166 ts_tablet_manager.cc:1403] T 8229b704e7464b30b7d2902a938ce33b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.002s
I20251211 20:52:35.488442 1166 raft_consensus.cc:359] T 8229b704e7464b30b7d2902a938ce33b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.488649 1166 raft_consensus.cc:385] T 8229b704e7464b30b7d2902a938ce33b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.488716 1166 raft_consensus.cc:740] T 8229b704e7464b30b7d2902a938ce33b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.488957 1166 consensus_queue.cc:260] T 8229b704e7464b30b7d2902a938ce33b P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.489447 1166 ts_tablet_manager.cc:1434] T 8229b704e7464b30b7d2902a938ce33b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:35.489750 1166 tablet_bootstrap.cc:492] T c7a2610f20df4c1ca81eae2fc7dba6bb P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:35.490432 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_e4c970184c9c4300b80e462a20527759"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:35.491482 1166 tablet_bootstrap.cc:654] T c7a2610f20df4c1ca81eae2fc7dba6bb P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.493431 1168 tablet_bootstrap.cc:492] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:35.493677 1168 ts_tablet_manager.cc:1403] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.199s user 0.033s sys 0.024s
I20251211 20:52:35.495002 1168 raft_consensus.cc:359] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.495350 1168 raft_consensus.cc:385] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.495434 1168 raft_consensus.cc:740] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.495888 1168 consensus_queue.cc:260] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.497165 1160 tablet_bootstrap.cc:492] T 1843dc4548134f29969d46fbc760f5d7 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:35.497371 1160 ts_tablet_manager.cc:1403] T 1843dc4548134f29969d46fbc760f5d7 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.105s user 0.016s sys 0.002s
I20251211 20:52:35.501147 1160 raft_consensus.cc:359] T 1843dc4548134f29969d46fbc760f5d7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.501856 1160 raft_consensus.cc:385] T 1843dc4548134f29969d46fbc760f5d7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.501984 1160 raft_consensus.cc:740] T 1843dc4548134f29969d46fbc760f5d7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.502259 1160 consensus_queue.cc:260] T 1843dc4548134f29969d46fbc760f5d7 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.502573 1168 ts_tablet_manager.cc:1434] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20251211 20:52:35.504596 1168 tablet_bootstrap.cc:492] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:35.507943 1160 ts_tablet_manager.cc:1434] T 1843dc4548134f29969d46fbc760f5d7 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20251211 20:52:35.513988 1160 tablet_bootstrap.cc:492] T ef4fada332cf4b5c9da26ba5f908b4c6 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:35.515645 1160 tablet_bootstrap.cc:654] T ef4fada332cf4b5c9da26ba5f908b4c6 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.517953 1890 raft_consensus.cc:493] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.518100 1890 raft_consensus.cc:515] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.518674 1890 leader_election.cc:290] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:35.521301 847 leader_election.cc:343] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: c85624ad640041c29bb87416e1992a9e
W20251211 20:52:35.521970 847 leader_election.cc:343] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: c85624ad640041c29bb87416e1992a9e
I20251211 20:52:35.522073 847 leader_election.cc:304] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:35.522606 1890 raft_consensus.cc:2749] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:35.522763 1890 consensus_queue.cc:1048] T ae1abef61473443b9922eb13f2ae92fa P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:35.523343 1586 raft_consensus.cc:493] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.523468 1586 raft_consensus.cc:515] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.524049 1586 leader_election.cc:290] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:35.524600 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "773cf53745a64f62817113b33e3182e7" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:35.525264 983 leader_election.cc:343] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.525938 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "773cf53745a64f62817113b33e3182e7" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:35.526629 982 leader_election.cc:343] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.526728 982 leader_election.cc:304] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:35.527338 1586 raft_consensus.cc:2749] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:35.533502 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_88717d63417a4513b501264b4cf6f5be in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:35.534852 1168 tablet_bootstrap.cc:654] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:35.533514 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e4c970184c9c4300b80e462a20527759 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:35.546092 595 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f3c10706c1294f978865836c1f29549f" } new_table_name: "default.Table_3c3cd48c8fe5411e96cdf811d492925d" modify_external_catalogs: true
W20251211 20:52:35.546463 592 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 48820386e41b47129a13bdd5768f021b consensus state has no leader
I20251211 20:52:35.546880 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_d87f970fdabb4d10871f55bb8193bd4d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:35.555155 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c27589191ac249f4a9460f5bbae2611b" } modify_external_catalogs: true
I20251211 20:52:35.555353 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c27589191ac249f4a9460f5bbae2611b" } modify_external_catalogs: true
W20251211 20:52:35.562239 595 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 33fd09ea06ee47908b85911e4ee79708 consensus state has no leader
I20251211 20:52:35.562714 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_bd71cd88b0354c88aa03799a4362f1b1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:35.568064 1168 tablet_bootstrap.cc:492] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:35.568217 1168 ts_tablet_manager.cc:1403] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.064s user 0.003s sys 0.000s
I20251211 20:52:35.569572 1168 raft_consensus.cc:359] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.569897 1168 raft_consensus.cc:385] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.570010 1168 raft_consensus.cc:740] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.570256 1168 consensus_queue.cc:260] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.570869 1168 ts_tablet_manager.cc:1434] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
W20251211 20:52:35.572301 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd71cd88b0354c88aa03799a4362f1b1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:35.572419 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d87f970fdabb4d10871f55bb8193bd4d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:35.579743 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_fde09a02a1664bc5bad79b2e0036b42a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:35.586803 1168 tablet_bootstrap.cc:492] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:35.584128 592 master_service.cc:954] ReplaceTablet: received request to replace tablet 2207af105bc3471ca930d35a91d38e70 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:35.586697 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_d04571b1151e4e89bfbf3148cbcfd269"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:35.596386 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d04571b1151e4e89bfbf3148cbcfd269 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:35.599507 1994 raft_consensus.cc:493] T 1a77e3e744ff42ccb68897704b4c2f1d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251211 20:52:35.603093 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fde09a02a1664bc5bad79b2e0036b42a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:35.603806 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_dffb2ba445c0475e96e343895b21cb51" } modify_external_catalogs: true
I20251211 20:52:35.604079 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_dffb2ba445c0475e96e343895b21cb51" } modify_external_catalogs: true
I20251211 20:52:35.611966 1994 raft_consensus.cc:515] T 1a77e3e744ff42ccb68897704b4c2f1d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.613183 1994 leader_election.cc:290] T 1a77e3e744ff42ccb68897704b4c2f1d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:35.613663 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a77e3e744ff42ccb68897704b4c2f1d" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:35.618510 1168 tablet_bootstrap.cc:654] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.618597 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_4abd8d4223574788846f1682e0e522ff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:35.620013 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4abd8d4223574788846f1682e0e522ff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:35.621968 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a77e3e744ff42ccb68897704b4c2f1d" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:35.614298 983 leader_election.cc:343] T 1a77e3e744ff42ccb68897704b4c2f1d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:35.631774 982 leader_election.cc:343] T 1a77e3e744ff42ccb68897704b4c2f1d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.632045 982 leader_election.cc:304] T 1a77e3e744ff42ccb68897704b4c2f1d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:35.632377 1995 raft_consensus.cc:2749] T 1a77e3e744ff42ccb68897704b4c2f1d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:35.621027 1166 tablet_bootstrap.cc:492] T c7a2610f20df4c1ca81eae2fc7dba6bb P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:35.627081 1586 raft_consensus.cc:493] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.636454 1166 ts_tablet_manager.cc:1403] T c7a2610f20df4c1ca81eae2fc7dba6bb P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.147s user 0.040s sys 0.048s
I20251211 20:52:35.635411 1586 raft_consensus.cc:515] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.639755 1586 leader_election.cc:290] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:35.640470 1586 raft_consensus.cc:493] T 8229b704e7464b30b7d2902a938ce33b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.641202 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0f046ea2637e4a5583b7969e19a605d5" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:35.641821 983 leader_election.cc:343] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.641310 1586 raft_consensus.cc:515] T 8229b704e7464b30b7d2902a938ce33b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.641573 1166 raft_consensus.cc:359] T c7a2610f20df4c1ca81eae2fc7dba6bb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.642184 1166 raft_consensus.cc:385] T c7a2610f20df4c1ca81eae2fc7dba6bb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.642751 1166 raft_consensus.cc:740] T c7a2610f20df4c1ca81eae2fc7dba6bb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.643416 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0f046ea2637e4a5583b7969e19a605d5" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:35.645200 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8229b704e7464b30b7d2902a938ce33b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:35.645287 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8229b704e7464b30b7d2902a938ce33b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:35.645694 983 leader_election.cc:343] T 8229b704e7464b30b7d2902a938ce33b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.645870 1995 raft_consensus.cc:493] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.643967 1166 consensus_queue.cc:260] T c7a2610f20df4c1ca81eae2fc7dba6bb P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.646090 1995 raft_consensus.cc:515] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.645956 1586 leader_election.cc:290] T 8229b704e7464b30b7d2902a938ce33b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:35.647046 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6a526a6d1bc84cefad2c5d897223292e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:35.646817 1586 raft_consensus.cc:493] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.647348 594 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 0f046ea2637e4a5583b7969e19a605d5 (table default.Table_dffb2ba445c0475e96e343895b21cb51 [id=e96fd7db496847a5b1e0b79546a336d0]). Expected version 2 got 0
W20251211 20:52:35.647531 983 leader_election.cc:343] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.647769 1995 leader_election.cc:290] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:35.647357 1586 raft_consensus.cc:515] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.651888 590 catalog_manager.cc:5654] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:35.652310 590 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 1843dc4548134f29969d46fbc760f5d7 (table default.Table_99917cb4c70840e19004ad381746d77e [id=6abb187bfd214e91b2e11267065a2841]). Expected version 1 got 0
I20251211 20:52:35.652930 1168 tablet_bootstrap.cc:492] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:35.653055 1168 ts_tablet_manager.cc:1403] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.066s user 0.003s sys 0.000s
I20251211 20:52:35.653842 1168 raft_consensus.cc:359] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.653993 1168 raft_consensus.cc:385] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.654038 1168 raft_consensus.cc:740] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.654217 1168 consensus_queue.cc:260] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.654647 1168 ts_tablet_manager.cc:1434] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:35.654906 1168 tablet_bootstrap.cc:492] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:35.655714 588 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_4d70f35e05524680b3d72b2e488adbd5 [id=61b2de16fee74487823292913de1b685] alter complete (version 3)
I20251211 20:52:35.658347 891 tablet_service.cc:1505] Processing CreateTablet for tablet 7023afec769c4ca3b2242a225004d482 (DEFAULT_TABLE table=default.Table_5da0e9ddf96549c88878f7136d0ba415 [id=3811f8235b2f44cca283401302d613d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.658743 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7023afec769c4ca3b2242a225004d482. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:35.660260 982 leader_election.cc:343] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.660475 982 leader_election.cc:304] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:35.660856 982 leader_election.cc:343] T 8229b704e7464b30b7d2902a938ce33b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.660945 982 leader_election.cc:304] T 8229b704e7464b30b7d2902a938ce33b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:35.661144 1994 raft_consensus.cc:2749] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:35.661343 1994 raft_consensus.cc:2749] T 8229b704e7464b30b7d2902a938ce33b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:35.661655 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6a526a6d1bc84cefad2c5d897223292e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:35.660100 1166 ts_tablet_manager.cc:1434] T c7a2610f20df4c1ca81eae2fc7dba6bb P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.023s user 0.003s sys 0.000s
W20251211 20:52:35.662142 982 leader_election.cc:343] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.662230 982 leader_election.cc:304] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:35.662590 2000 raft_consensus.cc:2749] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:35.664628 754 tablet_service.cc:1505] Processing CreateTablet for tablet 7023afec769c4ca3b2242a225004d482 (DEFAULT_TABLE table=default.Table_5da0e9ddf96549c88878f7136d0ba415 [id=3811f8235b2f44cca283401302d613d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.665025 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7023afec769c4ca3b2242a225004d482. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.664999 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "87863daf77304a399318cdf56da9f9bb" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:35.665158 1168 tablet_bootstrap.cc:654] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.666998 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "87863daf77304a399318cdf56da9f9bb" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:35.665472 982 leader_election.cc:343] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.665722 1166 tablet_bootstrap.cc:492] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:35.666644 1586 leader_election.cc:290] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
W20251211 20:52:35.668324 983 leader_election.cc:343] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.668789 983 leader_election.cc:304] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:35.669222 1586 raft_consensus.cc:2749] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:35.679979 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_97f780d6dd39409dbb5e5ff7e4fd61fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:35.680589 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_97f780d6dd39409dbb5e5ff7e4fd61fd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:35.684340 1166 tablet_bootstrap.cc:654] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.687249 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 7023afec769c4ca3b2242a225004d482 (DEFAULT_TABLE table=default.Table_5da0e9ddf96549c88878f7136d0ba415 [id=3811f8235b2f44cca283401302d613d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.687695 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7023afec769c4ca3b2242a225004d482. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.693261 888 tablet_service.cc:1505] Processing CreateTablet for tablet 14f367adb60744c8a593d3b9d4711823 (DEFAULT_TABLE table=default.Table_f2dc337cf8ba44408562bb0710ec4527 [id=69b0d7547eef4822a3cf7d4909f8265b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.693668 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14f367adb60744c8a593d3b9d4711823. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.693651 756 tablet_service.cc:1505] Processing CreateTablet for tablet 14f367adb60744c8a593d3b9d4711823 (DEFAULT_TABLE table=default.Table_f2dc337cf8ba44408562bb0710ec4527 [id=69b0d7547eef4822a3cf7d4909f8265b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.694046 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14f367adb60744c8a593d3b9d4711823. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.695864 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 14f367adb60744c8a593d3b9d4711823 (DEFAULT_TABLE table=default.Table_f2dc337cf8ba44408562bb0710ec4527 [id=69b0d7547eef4822a3cf7d4909f8265b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.695873 879 tablet_service.cc:1505] Processing CreateTablet for tablet a37c273c9a5742eaae616e5c5f87f9d7 (DEFAULT_TABLE table=default.Table_b0ac4ccf9e224d32af03dcaa1b97330a [id=10bf2a53ba594b5e8b8a37d2ba0235b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.696214 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14f367adb60744c8a593d3b9d4711823. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.696264 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a37c273c9a5742eaae616e5c5f87f9d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.699937 751 tablet_service.cc:1505] Processing CreateTablet for tablet a37c273c9a5742eaae616e5c5f87f9d7 (DEFAULT_TABLE table=default.Table_b0ac4ccf9e224d32af03dcaa1b97330a [id=10bf2a53ba594b5e8b8a37d2ba0235b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.700320 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a37c273c9a5742eaae616e5c5f87f9d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.703941 1028 tablet_service.cc:1505] Processing CreateTablet for tablet a37c273c9a5742eaae616e5c5f87f9d7 (DEFAULT_TABLE table=default.Table_b0ac4ccf9e224d32af03dcaa1b97330a [id=10bf2a53ba594b5e8b8a37d2ba0235b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.704324 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a37c273c9a5742eaae616e5c5f87f9d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.708523 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet bb4004212441473483a209d23c0800ce
I20251211 20:52:35.709499 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet c549c2bfebd64b87a4fc8ce8210e14f6
I20251211 20:52:35.711290 885 tablet_service.cc:1552] Processing DeleteTablet for tablet c549c2bfebd64b87a4fc8ce8210e14f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:35.711459 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 2207af105bc3471ca930d35a91d38e70
I20251211 20:52:35.711972 592 catalog_manager.cc:6724] ReplaceTablet: tablet 2207af105bc3471ca930d35a91d38e70 deleted and replaced by tablet 8675e2a18f0f4a0e95a87005cf82d8b4
W20251211 20:52:35.712623 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet c549c2bfebd64b87a4fc8ce8210e14f6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c549c2bfebd64b87a4fc8ce8210e14f6 already in progress: creating tablet
I20251211 20:52:35.713274 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 2207af105bc3471ca930d35a91d38e70 with delete_type TABLET_DATA_DELETED (replaced by tablet 8675e2a18f0f4a0e95a87005cf82d8b4) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:35.714026 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 2207af105bc3471ca930d35a91d38e70 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2207af105bc3471ca930d35a91d38e70 already in progress: creating tablet
I20251211 20:52:35.717722 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 2207af105bc3471ca930d35a91d38e70 with delete_type TABLET_DATA_DELETED (replaced by tablet 8675e2a18f0f4a0e95a87005cf82d8b4) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:35.719556 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 2207af105bc3471ca930d35a91d38e70 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2207af105bc3471ca930d35a91d38e70 already in progress: creating tablet
I20251211 20:52:35.720700 757 tablet_service.cc:1552] Processing DeleteTablet for tablet c549c2bfebd64b87a4fc8ce8210e14f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:35.721315 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 2207af105bc3471ca930d35a91d38e70 with delete_type TABLET_DATA_DELETED (replaced by tablet 8675e2a18f0f4a0e95a87005cf82d8b4) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:35.721869 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet c549c2bfebd64b87a4fc8ce8210e14f6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c549c2bfebd64b87a4fc8ce8210e14f6 already in progress: creating tablet
W20251211 20:52:35.722102 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 2207af105bc3471ca930d35a91d38e70 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2207af105bc3471ca930d35a91d38e70 already in progress: creating tablet
I20251211 20:52:35.722409 751 tablet_service.cc:1552] Processing DeleteTablet for tablet bb4004212441473483a209d23c0800ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:35.722992 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet bb4004212441473483a209d23c0800ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb4004212441473483a209d23c0800ce already in progress: creating tablet
I20251211 20:52:35.728109 1696 raft_consensus.cc:493] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.728286 1696 raft_consensus.cc:515] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.728937 1696 leader_election.cc:290] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:35.729466 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8139acb0ae324d0f973ce877edc1cd04" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:35.730001 711 leader_election.cc:343] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:35.730204 710 leader_election.cc:343] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 8139acb0ae324d0f973ce877edc1cd04
I20251211 20:52:35.730342 710 leader_election.cc:304] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:35.730674 1166 tablet_bootstrap.cc:492] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:35.730662 1696 raft_consensus.cc:2749] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:35.730813 1166 ts_tablet_manager.cc:1403] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.067s user 0.005s sys 0.000s
I20251211 20:52:35.731504 1166 raft_consensus.cc:359] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.731694 1166 raft_consensus.cc:385] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.731738 1166 raft_consensus.cc:740] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.731756 1019 tablet_service.cc:1552] Processing DeleteTablet for tablet bb4004212441473483a209d23c0800ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:35.731917 1166 consensus_queue.cc:260] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.732353 1166 ts_tablet_manager.cc:1434] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251211 20:52:35.732437 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet c549c2bfebd64b87a4fc8ce8210e14f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:35.732626 1166 tablet_bootstrap.cc:492] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:35.733382 891 tablet_service.cc:1552] Processing DeleteTablet for tablet bb4004212441473483a209d23c0800ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:35.734099 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet bb4004212441473483a209d23c0800ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb4004212441473483a209d23c0800ce already in progress: creating tablet
I20251211 20:52:35.734701 2000 raft_consensus.cc:493] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.734819 2000 raft_consensus.cc:515] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.735313 2000 leader_election.cc:290] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:35.736495 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7bd218b3ff340b2a589c9517b49a649" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:35.737165 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7bd218b3ff340b2a589c9517b49a649" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:35.737732 983 leader_election.cc:343] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:35.738124 982 leader_election.cc:343] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.738211 982 leader_election.cc:304] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:35.738619 1994 raft_consensus.cc:2749] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:35.738695 879 tablet_service.cc:1505] Processing CreateTablet for tablet 81fe26e81790448fb436929c03e3c284 (DEFAULT_TABLE table=default.Table_24b29712550340f2bea51b04f527c4f3 [id=d8a0839df7524fb7b95bfc05fd4ada25]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.738749 1166 tablet_bootstrap.cc:654] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.739053 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81fe26e81790448fb436929c03e3c284. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:35.739619 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet bb4004212441473483a209d23c0800ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb4004212441473483a209d23c0800ce already in progress: creating tablet
W20251211 20:52:35.739863 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet c549c2bfebd64b87a4fc8ce8210e14f6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c549c2bfebd64b87a4fc8ce8210e14f6 already in progress: creating tablet
I20251211 20:52:35.741897 754 tablet_service.cc:1505] Processing CreateTablet for tablet b81a4cef27164824b52695039744db15 (DEFAULT_TABLE table=default.Table_5b7fdd64f8f245a69b384dd22aedba9e [id=11352566d1744193a40a8bb605aa3912]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.742352 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b81a4cef27164824b52695039744db15. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.743175 1168 tablet_bootstrap.cc:492] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:35.743328 1168 ts_tablet_manager.cc:1403] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.088s user 0.019s sys 0.002s
I20251211 20:52:35.744170 1168 raft_consensus.cc:359] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.744251 1020 tablet_service.cc:1505] Processing CreateTablet for tablet b81a4cef27164824b52695039744db15 (DEFAULT_TABLE table=default.Table_5b7fdd64f8f245a69b384dd22aedba9e [id=11352566d1744193a40a8bb605aa3912]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.744359 1168 raft_consensus.cc:385] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.744446 1168 raft_consensus.cc:740] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.744727 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b81a4cef27164824b52695039744db15. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.744660 1168 consensus_queue.cc:260] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.745179 1168 ts_tablet_manager.cc:1434] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251211 20:52:35.745646 1168 tablet_bootstrap.cc:492] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:35.747272 1168 tablet_bootstrap.cc:654] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.745642 891 tablet_service.cc:1505] Processing CreateTablet for tablet b81a4cef27164824b52695039744db15 (DEFAULT_TABLE table=default.Table_5b7fdd64f8f245a69b384dd22aedba9e [id=11352566d1744193a40a8bb605aa3912]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.748631 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b81a4cef27164824b52695039744db15. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.766294 2008 raft_consensus.cc:493] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.766461 2008 raft_consensus.cc:515] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.767091 2008 leader_election.cc:290] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:35.768242 847 leader_election.cc:343] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: d5f0ba3ab16240f9822be69f941f5da2
W20251211 20:52:35.768591 847 leader_election.cc:343] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: d5f0ba3ab16240f9822be69f941f5da2
I20251211 20:52:35.768690 847 leader_election.cc:304] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:35.770903 1989 raft_consensus.cc:2749] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:35.772567 961 heartbeater.cc:344] Connected to a master server at 127.0.104.61:40749
I20251211 20:52:35.774667 754 tablet_service.cc:1505] Processing CreateTablet for tablet 81fe26e81790448fb436929c03e3c284 (DEFAULT_TABLE table=default.Table_24b29712550340f2bea51b04f527c4f3 [id=d8a0839df7524fb7b95bfc05fd4ada25]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.775214 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81fe26e81790448fb436929c03e3c284. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.775355 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 81fe26e81790448fb436929c03e3c284 (DEFAULT_TABLE table=default.Table_24b29712550340f2bea51b04f527c4f3 [id=d8a0839df7524fb7b95bfc05fd4ada25]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.775840 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81fe26e81790448fb436929c03e3c284. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.777264 1168 tablet_bootstrap.cc:492] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:35.777451 1168 ts_tablet_manager.cc:1403] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.032s user 0.000s sys 0.008s
I20251211 20:52:35.778214 1168 raft_consensus.cc:359] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.778409 1168 raft_consensus.cc:385] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.778494 1168 raft_consensus.cc:740] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.778766 1168 consensus_queue.cc:260] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.791922 1168 ts_tablet_manager.cc:1434] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.014s user 0.000s sys 0.002s
I20251211 20:52:35.796701 1166 tablet_bootstrap.cc:492] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:35.798158 1166 ts_tablet_manager.cc:1403] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.065s user 0.003s sys 0.005s
I20251211 20:52:35.801391 1160 tablet_bootstrap.cc:492] T ef4fada332cf4b5c9da26ba5f908b4c6 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:35.801559 1160 ts_tablet_manager.cc:1403] T ef4fada332cf4b5c9da26ba5f908b4c6 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.288s user 0.000s sys 0.015s
I20251211 20:52:35.802268 1160 raft_consensus.cc:359] T ef4fada332cf4b5c9da26ba5f908b4c6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.802438 1160 raft_consensus.cc:385] T ef4fada332cf4b5c9da26ba5f908b4c6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.802480 1160 raft_consensus.cc:740] T ef4fada332cf4b5c9da26ba5f908b4c6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.802703 1160 consensus_queue.cc:260] T ef4fada332cf4b5c9da26ba5f908b4c6 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.803184 1160 ts_tablet_manager.cc:1434] T ef4fada332cf4b5c9da26ba5f908b4c6 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251211 20:52:35.803370 1028 tablet_service.cc:1505] Processing CreateTablet for tablet 4cae3a6362be4834acd9b06123694a2b (DEFAULT_TABLE table=default.Table_d71f88120b9d4408a1b04988573d137d [id=6dae16bd96534a0c89619742c262e081]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.803490 1160 tablet_bootstrap.cc:492] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:35.803867 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cae3a6362be4834acd9b06123694a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.803133 1166 raft_consensus.cc:359] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.805035 1166 raft_consensus.cc:385] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.805424 1166 raft_consensus.cc:740] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.806321 1160 tablet_bootstrap.cc:654] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.807195 1890 raft_consensus.cc:493] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.806627 1166 consensus_queue.cc:260] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.807323 1890 raft_consensus.cc:515] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.807934 1890 leader_election.cc:290] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:35.809168 879 tablet_service.cc:1505] Processing CreateTablet for tablet 4cae3a6362be4834acd9b06123694a2b (DEFAULT_TABLE table=default.Table_d71f88120b9d4408a1b04988573d137d [id=6dae16bd96534a0c89619742c262e081]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.809635 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cae3a6362be4834acd9b06123694a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.809414 1166 ts_tablet_manager.cc:1434] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
W20251211 20:52:35.810315 847 leader_election.cc:343] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: dcdcff8235814b58b3683104a2ff0cbf
I20251211 20:52:35.811120 1166 tablet_bootstrap.cc:492] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:35.812162 1160 tablet_bootstrap.cc:492] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:35.812290 1160 ts_tablet_manager.cc:1403] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.001s
I20251211 20:52:35.812942 1160 raft_consensus.cc:359] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.813099 1160 raft_consensus.cc:385] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.813149 1160 raft_consensus.cc:740] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.813335 1160 consensus_queue.cc:260] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.813776 1160 ts_tablet_manager.cc:1434] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251211 20:52:35.814059 1160 tablet_bootstrap.cc:492] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:35.815433 1160 tablet_bootstrap.cc:654] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.815922 751 tablet_service.cc:1505] Processing CreateTablet for tablet 4cae3a6362be4834acd9b06123694a2b (DEFAULT_TABLE table=default.Table_d71f88120b9d4408a1b04988573d137d [id=6dae16bd96534a0c89619742c262e081]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.816380 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cae3a6362be4834acd9b06123694a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.819207 1168 tablet_bootstrap.cc:492] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:35.820657 847 leader_election.cc:343] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: dcdcff8235814b58b3683104a2ff0cbf
I20251211 20:52:35.820773 847 leader_election.cc:304] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:35.821345 1813 raft_consensus.cc:2749] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:35.821919 1168 tablet_bootstrap.cc:654] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.822103 1166 tablet_bootstrap.cc:654] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.827309 1696 raft_consensus.cc:493] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.827468 1696 raft_consensus.cc:515] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.827983 2013 tablet.cc:1722] T 01ada50100bc48e39d1bc67082da750e P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 6
I20251211 20:52:35.828207 1696 leader_election.cc:290] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:35.829532 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "40722aef4e194ff094d271a079bda9d9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:35.830129 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "40722aef4e194ff094d271a079bda9d9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:35.830309 1044 raft_consensus.cc:2468] T 40722aef4e194ff094d271a079bda9d9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
W20251211 20:52:35.830771 711 leader_election.cc:343] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.831156 710 leader_election.cc:304] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:35.831557 1696 raft_consensus.cc:2804] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:35.831488 2009 tablet.cc:1722] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:35.831681 1696 raft_consensus.cc:493] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:35.831763 1696 raft_consensus.cc:3060] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:35.834532 1696 raft_consensus.cc:515] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.839419 1166 tablet_bootstrap.cc:492] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:35.839697 1166 ts_tablet_manager.cc:1403] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.029s user 0.005s sys 0.001s
I20251211 20:52:35.840991 1166 raft_consensus.cc:359] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.841260 1166 raft_consensus.cc:385] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.841377 1166 raft_consensus.cc:740] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.841677 1166 consensus_queue.cc:260] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.842829 1166 ts_tablet_manager.cc:1434] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:35.843814 1166 tablet_bootstrap.cc:492] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:35.846611 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "40722aef4e194ff094d271a079bda9d9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:35.847179 711 leader_election.cc:343] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.848069 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "40722aef4e194ff094d271a079bda9d9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:35.848232 1044 raft_consensus.cc:3060] T 40722aef4e194ff094d271a079bda9d9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:35.849547 1166 tablet_bootstrap.cc:654] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.850862 1044 raft_consensus.cc:2468] T 40722aef4e194ff094d271a079bda9d9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:35.851334 710 leader_election.cc:304] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:35.852352 1696 leader_election.cc:290] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:35.854255 1696 raft_consensus.cc:2804] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:35.854686 1696 raft_consensus.cc:697] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:35.855121 1696 consensus_queue.cc:237] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.858001 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 0f046ea2637e4a5583b7969e19a605d5
I20251211 20:52:35.859282 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 0f046ea2637e4a5583b7969e19a605d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:35.860146 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 0f046ea2637e4a5583b7969e19a605d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0f046ea2637e4a5583b7969e19a605d5 already in progress: creating tablet
I20251211 20:52:35.860729 588 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_e4656c3f3f974d68af3f207e635d316d [id=f78e1da479af4d23a552af1c4fc55f81] alter complete (version 3)
I20251211 20:52:35.860872 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 0f046ea2637e4a5583b7969e19a605d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:35.861462 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 0f046ea2637e4a5583b7969e19a605d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0f046ea2637e4a5583b7969e19a605d5 already in progress: creating tablet
I20251211 20:52:35.861621 588 master_service.cc:954] ReplaceTablet: received request to replace tablet 95866de91f9d437d80f40b7962253610 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:35.862393 888 tablet_service.cc:1552] Processing DeleteTablet for tablet dcdcff8235814b58b3683104a2ff0cbf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:35.862605 1164 tablet_replica.cc:333] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:35.862762 1164 raft_consensus.cc:2243] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:35.862893 1164 raft_consensus.cc:2272] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:35.863039 885 tablet_service.cc:1552] Processing DeleteTablet for tablet d30376baf75b4761971f91da3493a142 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:24 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:35.863420 1164 ts_tablet_manager.cc:1916] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:35.865711 1166 tablet_bootstrap.cc:492] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:35.865916 1166 ts_tablet_manager.cc:1403] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.022s user 0.005s sys 0.003s
I20251211 20:52:35.866681 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet e28d66ce4711430cb09a9650e99fbe9e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:35.866897 1163 tablet_replica.cc:333] T e28d66ce4711430cb09a9650e99fbe9e P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:35.867051 1163 raft_consensus.cc:2243] T e28d66ce4711430cb09a9650e99fbe9e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:35.867174 1163 raft_consensus.cc:2272] T e28d66ce4711430cb09a9650e99fbe9e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:35.867151 1166 raft_consensus.cc:359] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.867374 1166 raft_consensus.cc:385] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.867484 1166 raft_consensus.cc:740] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.867709 1163 ts_tablet_manager.cc:1916] T e28d66ce4711430cb09a9650e99fbe9e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:35.867872 1166 consensus_queue.cc:260] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.869904 1164 ts_tablet_manager.cc:1929] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:35.869997 1164 log.cc:1199] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/dcdcff8235814b58b3683104a2ff0cbf
I20251211 20:52:35.870246 1164 ts_tablet_manager.cc:1950] T dcdcff8235814b58b3683104a2ff0cbf P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:35.870813 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet dcdcff8235814b58b3683104a2ff0cbf (table default.Table_0209a7c0449b42e9ac582543b779f5c7 [id=3aa2f2349ee9431fb9bfb0634e5768b7]) successfully deleted
I20251211 20:52:35.871142 1164 tablet_replica.cc:333] T d30376baf75b4761971f91da3493a142 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:35.871141 1166 ts_tablet_manager.cc:1434] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251211 20:52:35.871309 1164 raft_consensus.cc:2243] T d30376baf75b4761971f91da3493a142 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:35.871440 1164 raft_consensus.cc:2272] T d30376baf75b4761971f91da3493a142 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:35.871789 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 8139acb0ae324d0f973ce877edc1cd04 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:35.871896 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 05385f20e88c46108e09a78814811209 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:35.872100 1165 tablet_replica.cc:333] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:35.872048 1166 tablet_bootstrap.cc:492] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:35.872309 1165 raft_consensus.cc:2243] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:35.872464 1165 raft_consensus.cc:2272] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:35.872998 1165 ts_tablet_manager.cc:1916] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:35.875169 1163 ts_tablet_manager.cc:1929] T e28d66ce4711430cb09a9650e99fbe9e P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:35.875392 1163 log.cc:1199] T e28d66ce4711430cb09a9650e99fbe9e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/e28d66ce4711430cb09a9650e99fbe9e
I20251211 20:52:35.875892 1163 ts_tablet_manager.cc:1950] T e28d66ce4711430cb09a9650e99fbe9e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:35.876477 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet e28d66ce4711430cb09a9650e99fbe9e (table default.Table_8b8ed719985f42128e56bc26105e7a9c [id=401822f4521f46eeb815147f643a6fcf]) successfully deleted
I20251211 20:52:35.876861 1028 tablet_service.cc:1552] Processing DeleteTablet for tablet 773cf53745a64f62817113b33e3182e7 with delete_type TABLET_DATA_DELETED (replaced by tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:35.877312 1166 tablet_bootstrap.cc:654] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.880318 1165 ts_tablet_manager.cc:1929] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:35.880559 1165 log.cc:1199] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/8139acb0ae324d0f973ce877edc1cd04
I20251211 20:52:35.880941 1165 ts_tablet_manager.cc:1950] T 8139acb0ae324d0f973ce877edc1cd04 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:35.882001 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 8139acb0ae324d0f973ce877edc1cd04 (table default.Table_d282d0b8f49e4f44998af55480ad3cb8 [id=959101c0af144f52a5680b0c9d4cbd20]) successfully deleted
I20251211 20:52:35.882459 1165 tablet_replica.cc:333] T 05385f20e88c46108e09a78814811209 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:35.882725 1165 raft_consensus.cc:2243] T 05385f20e88c46108e09a78814811209 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:35.882956 1165 raft_consensus.cc:2272] T 05385f20e88c46108e09a78814811209 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:35.883495 1165 ts_tablet_manager.cc:1916] T 05385f20e88c46108e09a78814811209 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:35.883744 1163 tablet_replica.cc:333] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:35.883831 1164 ts_tablet_manager.cc:1916] T d30376baf75b4761971f91da3493a142 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:35.885372 885 tablet_service.cc:1505] Processing CreateTablet for tablet 4810f8d8b53a45eb939bf3c8bf1b4e77 (DEFAULT_TABLE table=default.Table_198aadd702614de39863c677cd5d885b [id=1ee8884f18ec4030b047013b0b7b6d29]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.885831 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4810f8d8b53a45eb939bf3c8bf1b4e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.893083 1163 raft_consensus.cc:2243] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:35.893380 1163 raft_consensus.cc:2272] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:35.894073 1163 ts_tablet_manager.cc:1916] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:35.900950 2024 tablet.cc:1722] T 01ada50100bc48e39d1bc67082da750e P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 6
I20251211 20:52:35.883538 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 55c40b2464f8444ca6c9fa0400f7f7f2 with delete_type TABLET_DATA_DELETED (replaced by tablet fb030a62796248869d46b682656eaeb0) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:35.889269 1168 tablet_bootstrap.cc:492] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:35.904217 1168 ts_tablet_manager.cc:1403] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.112s user 0.004s sys 0.007s
I20251211 20:52:35.905336 1168 raft_consensus.cc:359] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:35.905637 1168 raft_consensus.cc:385] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.905938 1168 raft_consensus.cc:740] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.906538 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet de508d79f30f489cb745401f53115165
I20251211 20:52:35.906560 1168 consensus_queue.cc:260] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:35.906670 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet d53b074865f4490693152a247b0fffdc
I20251211 20:52:35.907714 756 tablet_service.cc:1552] Processing DeleteTablet for tablet d53b074865f4490693152a247b0fffdc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:35.908241 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet d53b074865f4490693152a247b0fffdc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:35.908514 1028 tablet_service.cc:1552] Processing DeleteTablet for tablet de508d79f30f489cb745401f53115165 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:35.908759 751 tablet_service.cc:1552] Processing DeleteTablet for tablet de508d79f30f489cb745401f53115165 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:35.908779 888 tablet_service.cc:1552] Processing DeleteTablet for tablet d53b074865f4490693152a247b0fffdc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:35.909385 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_17b787b4b49444a1ad091fc751bcbd35" } new_table_name: "default.Table_647731023b044ebb8954f7fdcc3faf97" modify_external_catalogs: true
I20251211 20:52:35.910967 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_4e2d2e9566764ee9bc5edf65d0e693a2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:35.911538 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4e2d2e9566764ee9bc5edf65d0e693a2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:35.911675 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_6063375588754775b43c219ddee022de"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:35.912166 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6063375588754775b43c219ddee022de in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:35.912763 2012 tablet.cc:1722] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:35.915475 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_205371a488cd4233a77a9057784ee2dc" } new_table_name: "default.Table_9739da338096425a8cd7aa3038740fa6" modify_external_catalogs: true
W20251211 20:52:35.915793 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 64530.4us
I20251211 20:52:35.916728 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_fdfecb7c7e1e4448a51918d3ad950323"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:35.917178 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fdfecb7c7e1e4448a51918d3ad950323 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:35.918259 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_88717d63417a4513b501264b4cf6f5be" } new_table_name: "default.Table_7809ac85fdfe4c1d9c58afb54edecb27" modify_external_catalogs: true
I20251211 20:52:35.919783 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_cb402f83e7b24e308e46a798daa7db02"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:35.920234 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cb402f83e7b24e308e46a798daa7db02 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:35.922602 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_7c081945e59345a7830dedbfd3f8f3ee" } new_table_name: "default.Table_d1fc77dff3464153a4fff421df27eb33" modify_external_catalogs: true
I20251211 20:52:35.922670 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8aa9c0811b4f4ac987225c8038d27553" } new_table_name: "default.Table_760e3e4714ea4128bd1de3da5a7ccccc" modify_external_catalogs: true
I20251211 20:52:35.924907 1168 ts_tablet_manager.cc:1434] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.020s user 0.001s sys 0.001s
I20251211 20:52:35.925246 1168 tablet_bootstrap.cc:492] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:35.927114 1168 tablet_bootstrap.cc:654] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.927328 885 tablet_service.cc:1552] Processing DeleteTablet for tablet de508d79f30f489cb745401f53115165 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:35.930701 1164 ts_tablet_manager.cc:1929] T d30376baf75b4761971f91da3493a142 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:35.930874 1164 log.cc:1199] T d30376baf75b4761971f91da3493a142 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/d30376baf75b4761971f91da3493a142
I20251211 20:52:35.931209 1164 ts_tablet_manager.cc:1950] T d30376baf75b4761971f91da3493a142 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:35.931897 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet d30376baf75b4761971f91da3493a142 (table default.Table_cec7f350a9d04bd590db8a29fb8ceb6b [id=62d2078174bb4b5f952896c6ada6a414]) successfully deleted
W20251211 20:52:35.932585 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet d53b074865f4490693152a247b0fffdc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d53b074865f4490693152a247b0fffdc already in progress: creating tablet
W20251211 20:52:35.933193 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet de508d79f30f489cb745401f53115165 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet de508d79f30f489cb745401f53115165 already in progress: creating tablet
I20251211 20:52:35.933648 751 tablet_service.cc:1505] Processing CreateTablet for tablet 4810f8d8b53a45eb939bf3c8bf1b4e77 (DEFAULT_TABLE table=default.Table_198aadd702614de39863c677cd5d885b [id=1ee8884f18ec4030b047013b0b7b6d29]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.934033 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4810f8d8b53a45eb939bf3c8bf1b4e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.933979 885 tablet_service.cc:1505] Processing CreateTablet for tablet 1f4eb1a174c648c897ae2b7612e44585 (DEFAULT_TABLE table=default.Table_2282dc6429c345098eeaf08b63262e8b [id=87adcc552a704d56bd2fe812fb3b791f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.934303 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f4eb1a174c648c897ae2b7612e44585. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.939980 2010 tablet.cc:1722] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:35.940629 1696 raft_consensus.cc:493] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.942893 1696 raft_consensus.cc:515] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.945155 1696 leader_election.cc:290] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:35.947304 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28ef13be3b84410d9c786c43502a3478" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:35.948076 591 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 8229b704e7464b30b7d2902a938ce33b (table default.Table_b008891c4e894b02aa0db32d1740371a [id=8fe47ac33eb2461784b859e64fbd3e24]). Expected version 5 got 0
W20251211 20:52:35.946110 710 leader_election.cc:343] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 28ef13be3b84410d9c786c43502a3478
I20251211 20:52:35.947754 824 heartbeater.cc:344] Connected to a master server at 127.0.104.61:40749
I20251211 20:52:35.948339 591 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet c7a2610f20df4c1ca81eae2fc7dba6bb (table default.Table_59072235d93b41c0b5cd5e3debcb8ddf [id=9a15a8a075af4969ae2c8b3be1bfed78]). Expected version 1 got 0
I20251211 20:52:35.948675 591 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3eac21a07ba34e1fa275e25ee9dcabb1 (table default.Table_d571120dedcb48cd9d4b75314be3a654 [id=9700f9fe6e3044748e34954bef8a0038]). Expected version 2 got 0
I20251211 20:52:35.948868 591 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 1a77e3e744ff42ccb68897704b4c2f1d (table default.Table_d70f80161dcd4063815a0e079c5821de [id=85ab3a18cda143e5a7f72e49e6671318]). Expected version 1 got 0
W20251211 20:52:35.951319 711 leader_election.cc:343] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.951814 711 leader_election.cc:304] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:35.952217 1696 raft_consensus.cc:2749] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:35.960489 1166 tablet_bootstrap.cc:492] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:35.960649 1166 ts_tablet_manager.cc:1403] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.089s user 0.008s sys 0.009s
I20251211 20:52:35.961252 1028 tablet_service.cc:1505] Processing CreateTablet for tablet 1f4eb1a174c648c897ae2b7612e44585 (DEFAULT_TABLE table=default.Table_2282dc6429c345098eeaf08b63262e8b [id=87adcc552a704d56bd2fe812fb3b791f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.961639 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f4eb1a174c648c897ae2b7612e44585. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.962340 1013 tablet_service.cc:1505] Processing CreateTablet for tablet c9956749c83a4a468571fdabcaf0e4aa (DEFAULT_TABLE table=default.Table_b011142d3fc34ef2a829989a9bad1114 [id=9e2abaab9bf940a8af1809342d51cd9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.962342 879 tablet_service.cc:1505] Processing CreateTablet for tablet 5773c13034714a008c8d7dbe777b4828 (DEFAULT_TABLE table=default.Table_063f94689e6547e2b871d8465153a098 [id=94ff09564cd649c7a2d0e94b0f3c7b3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.962679 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5773c13034714a008c8d7dbe777b4828. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.962690 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9956749c83a4a468571fdabcaf0e4aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.968138 1166 raft_consensus.cc:359] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.968322 1166 raft_consensus.cc:385] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.968374 1166 raft_consensus.cc:740] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.968595 1166 consensus_queue.cc:260] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.969066 1166 ts_tablet_manager.cc:1434] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.008s user 0.000s sys 0.001s
I20251211 20:52:35.969305 1166 tablet_bootstrap.cc:492] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:35.969635 2012 tablet.cc:1717] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7: Skipping requested alter to schema version 2, tablet already version 2
I20251211 20:52:35.970996 1166 tablet_bootstrap.cc:654] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:35.970505 751 tablet_service.cc:1505] Processing CreateTablet for tablet 1f4eb1a174c648c897ae2b7612e44585 (DEFAULT_TABLE table=default.Table_2282dc6429c345098eeaf08b63262e8b [id=87adcc552a704d56bd2fe812fb3b791f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.972543 1586 raft_consensus.cc:493] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.972664 1586 raft_consensus.cc:515] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.973111 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 0f046ea2637e4a5583b7969e19a605d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:35.973673 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25454e67437541d9ac2bfab29bad55f9" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:35.973786 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f4eb1a174c648c897ae2b7612e44585. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:35.974159 983 leader_election.cc:343] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.974792 1586 leader_election.cc:290] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:35.975201 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 4810f8d8b53a45eb939bf3c8bf1b4e77 (DEFAULT_TABLE table=default.Table_198aadd702614de39863c677cd5d885b [id=1ee8884f18ec4030b047013b0b7b6d29]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.976485 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4810f8d8b53a45eb939bf3c8bf1b4e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.981298 2012 tablet.cc:1722] T 01ada50100bc48e39d1bc67082da750e P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 6
I20251211 20:52:35.988504 1163 ts_tablet_manager.cc:1929] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:35.988715 1163 log.cc:1199] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/773cf53745a64f62817113b33e3182e7
I20251211 20:52:35.989061 1163 ts_tablet_manager.cc:1950] T 773cf53745a64f62817113b33e3182e7 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:35.990260 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 773cf53745a64f62817113b33e3182e7 (table default.Table_021d4fb1667a475d87424f23aa0d5e28 [id=6bbfcc2561f349519a69ae1f56016cc7]) successfully deleted
I20251211 20:52:35.990864 891 tablet_service.cc:1505] Processing CreateTablet for tablet c9956749c83a4a468571fdabcaf0e4aa (DEFAULT_TABLE table=default.Table_b011142d3fc34ef2a829989a9bad1114 [id=9e2abaab9bf940a8af1809342d51cd9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.993979 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 5773c13034714a008c8d7dbe777b4828 (DEFAULT_TABLE table=default.Table_063f94689e6547e2b871d8465153a098 [id=94ff09564cd649c7a2d0e94b0f3c7b3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.994417 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5773c13034714a008c8d7dbe777b4828. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.996417 2023 raft_consensus.cc:493] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.996444 879 tablet_service.cc:1505] Processing CreateTablet for tablet 1944267def4e46949fd680ab0a369be9 (DEFAULT_TABLE table=default.Table_514c179c4a144ce398e7c9b8fbba2bce [id=42d5ddf23fd549d0982312ace7df498e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.996889 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1944267def4e46949fd680ab0a369be9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.997229 2025 raft_consensus.cc:493] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:35.997884 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9956749c83a4a468571fdabcaf0e4aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.998301 1166 tablet_bootstrap.cc:492] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:35.998445 1166 ts_tablet_manager.cc:1403] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20251211 20:52:35.998494 754 tablet_service.cc:1505] Processing CreateTablet for tablet 5773c13034714a008c8d7dbe777b4828 (DEFAULT_TABLE table=default.Table_063f94689e6547e2b871d8465153a098 [id=94ff09564cd649c7a2d0e94b0f3c7b3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:35.998831 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5773c13034714a008c8d7dbe777b4828. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:35.999253 1166 raft_consensus.cc:359] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:35.999420 1166 raft_consensus.cc:385] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:35.999467 1166 raft_consensus.cc:740] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:35.999682 1166 consensus_queue.cc:260] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.000093 1165 ts_tablet_manager.cc:1929] T 05385f20e88c46108e09a78814811209 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:36.000317 1165 log.cc:1199] T 05385f20e88c46108e09a78814811209 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/05385f20e88c46108e09a78814811209
I20251211 20:52:36.001240 1096 heartbeater.cc:344] Connected to a master server at 127.0.104.61:40749
W20251211 20:52:36.002918 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7ffa06c4a7ee48b0b337a27aff090592 consensus state has no leader
I20251211 20:52:36.004855 2010 tablet.cc:1717] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41: Skipping requested alter to schema version 2, tablet already version 2
W20251211 20:52:36.005511 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 75dd170c5828409fa69bd68fdf28a72b consensus state has no leader
I20251211 20:52:36.008496 1696 consensus_queue.cc:1048] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20251211 20:52:36.012861 1166 ts_tablet_manager.cc:1434] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
I20251211 20:52:36.013172 1166 tablet_bootstrap.cc:492] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:36.014117 1696 raft_consensus.cc:493] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.014248 1696 raft_consensus.cc:515] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.014721 1166 tablet_bootstrap.cc:654] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.014882 1696 leader_election.cc:290] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:36.014896 756 tablet_service.cc:1505] Processing CreateTablet for tablet c9956749c83a4a468571fdabcaf0e4aa (DEFAULT_TABLE table=default.Table_b011142d3fc34ef2a829989a9bad1114 [id=9e2abaab9bf940a8af1809342d51cd9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.015511 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9956749c83a4a468571fdabcaf0e4aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.015798 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8feee01fc0aa4b32be926ff2b617b894" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:36.017889 742 tablet_service.cc:1505] Processing CreateTablet for tablet 1944267def4e46949fd680ab0a369be9 (DEFAULT_TABLE table=default.Table_514c179c4a144ce398e7c9b8fbba2bce [id=42d5ddf23fd549d0982312ace7df498e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.018230 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1944267def4e46949fd680ab0a369be9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.018778 592 master_service.cc:954] ReplaceTablet: received request to replace tablet 10746162da4640e6a2a891467690b770 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:36.014878 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25454e67437541d9ac2bfab29bad55f9" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:36.020058 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fb29f983c4524a49bdc5c0c1144ad708 has no consensus state
I20251211 20:52:36.020475 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_1f11bf68fb18474c9a3cce63119e7e5f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:36.021051 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1f11bf68fb18474c9a3cce63119e7e5f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:36.022434 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b929f7410fd44316b3a3571d02066b45"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:36.022949 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b929f7410fd44316b3a3571d02066b45 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:36.024070 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 51d12723f5194afb97d2c05fcbaf3776 consensus state has no leader
I20251211 20:52:36.023828 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8feee01fc0aa4b32be926ff2b617b894" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:36.024403 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_91c76a3c00864c3fa3d653b3f6e0f64f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:36.024856 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91c76a3c00864c3fa3d653b3f6e0f64f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:36.024705 1046 raft_consensus.cc:2468] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
W20251211 20:52:36.025875 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bb9fccb6981e446a939a28d7fdd38e25 consensus state has no leader
I20251211 20:52:36.026264 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_35ba9d3f180646d8b597a9c14b01cf37" } modify_external_catalogs: true
I20251211 20:52:36.026415 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_35ba9d3f180646d8b597a9c14b01cf37" } modify_external_catalogs: true
W20251211 20:52:36.027283 982 leader_election.cc:343] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.027392 982 leader_election.cc:304] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.027787 2000 raft_consensus.cc:2749] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:36.028234 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_2eeb4b33f8424175a9e15a54e38099a2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:36.028713 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2eeb4b33f8424175a9e15a54e38099a2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:35.997350 2025 raft_consensus.cc:515] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.029445 2025 leader_election.cc:290] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:36.029915 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_df5fb8b06973455d8c2b6a335e1f1d95"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:36.030354 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_df5fb8b06973455d8c2b6a335e1f1d95 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:36.031023 711 leader_election.cc:343] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:35.996639 2023 raft_consensus.cc:515] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.039248 596 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 1843dc4548134f29969d46fbc760f5d7 (table default.Table_99917cb4c70840e19004ad381746d77e [id=6abb187bfd214e91b2e11267065a2841]). Expected version 1 got 0
I20251211 20:52:36.039563 2008 raft_consensus.cc:493] T 1843dc4548134f29969d46fbc760f5d7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.039736 2008 raft_consensus.cc:515] T 1843dc4548134f29969d46fbc760f5d7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.040335 2008 leader_election.cc:290] T 1843dc4548134f29969d46fbc760f5d7 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:36.040001 1166 tablet_bootstrap.cc:492] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:36.041231 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1843dc4548134f29969d46fbc760f5d7" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:36.041321 1166 ts_tablet_manager.cc:1403] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.028s user 0.009s sys 0.000s
I20251211 20:52:36.041603 596 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 01ada50100bc48e39d1bc67082da750e (table default.Table_690a1a1fd5d848818451819f38fc1470 [id=a24d8c58b9d84cdea2b278ae9c6e0ba5]). Expected version 6 got 5
I20251211 20:52:36.042069 1166 raft_consensus.cc:359] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.042232 1166 raft_consensus.cc:385] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.042277 1166 raft_consensus.cc:740] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.042452 1166 consensus_queue.cc:260] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.042196 1165 ts_tablet_manager.cc:1950] T 05385f20e88c46108e09a78814811209 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:36.042912 1166 ts_tablet_manager.cc:1434] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:36.043164 1166 tablet_bootstrap.cc:492] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:36.043016 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1843dc4548134f29969d46fbc760f5d7" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:36.044813 1166 tablet_bootstrap.cc:654] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.045621 589 catalog_manager.cc:5654] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:36.045812 710 leader_election.cc:304] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.046118 589 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 40722aef4e194ff094d271a079bda9d9 (table default.Table_12f41f708f1949598d6176d89777aa69 [id=3309205faabf4a42acadc43766e4f57f]). Expected version 1 got 0
I20251211 20:52:36.046357 589 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 9f8aa28c1c9b4fc58dfc128952c180ba (table default.Table_bf7d402a36be44cfb6b79041906d26a1 [id=0327e686e7ed4475a3e8cda8721a3a3a]). Expected version 1 got 0
I20251211 20:52:36.046473 2019 raft_consensus.cc:2804] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:36.046554 2019 raft_consensus.cc:493] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:36.046610 2019 raft_consensus.cc:3060] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:36.047178 589 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 1843dc4548134f29969d46fbc760f5d7 (table default.Table_99917cb4c70840e19004ad381746d77e [id=6abb187bfd214e91b2e11267065a2841]). Expected version 1 got 0
I20251211 20:52:36.047412 847 leader_election.cc:304] T 1843dc4548134f29969d46fbc760f5d7 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:36.047482 589 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 8feee01fc0aa4b32be926ff2b617b894 (table default.Table_265379fa17b74d0f954a2a2c4de113c8 [id=18f8d722faaa4293a112525ff0c96216]). Expected version 2 got 0
I20251211 20:52:36.047881 2008 raft_consensus.cc:2749] T 1843dc4548134f29969d46fbc760f5d7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:36.049104 2019 raft_consensus.cc:515] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.049486 2023 leader_election.cc:290] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:36.049742 2019 leader_election.cc:290] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:36.050433 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8feee01fc0aa4b32be926ff2b617b894" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:36.050624 1048 raft_consensus.cc:3060] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:36.050752 710 leader_election.cc:343] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: bc32a778e5424850b23ccaf74bb9df07
I20251211 20:52:36.052588 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ef4fada332cf4b5c9da26ba5f908b4c6" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:36.053283 1048 raft_consensus.cc:2468] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:36.053966 710 leader_election.cc:304] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:36.054234 2019 raft_consensus.cc:2804] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:36.054358 2019 raft_consensus.cc:697] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:36.054610 2019 consensus_queue.cc:237] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.055961 1046 raft_consensus.cc:2468] T ef4fada332cf4b5c9da26ba5f908b4c6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:36.057638 1168 tablet_bootstrap.cc:492] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:36.057746 1166 tablet_bootstrap.cc:492] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:36.057777 1168 ts_tablet_manager.cc:1403] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.133s user 0.003s sys 0.000s
I20251211 20:52:36.057971 1166 ts_tablet_manager.cc:1403] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20251211 20:52:36.057901 710 leader_election.cc:304] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:36.058233 2023 raft_consensus.cc:2804] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:36.058352 2023 raft_consensus.cc:493] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:36.058467 2023 raft_consensus.cc:3060] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:36.058576 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ef4fada332cf4b5c9da26ba5f908b4c6" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:36.058765 912 raft_consensus.cc:2468] T ef4fada332cf4b5c9da26ba5f908b4c6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:36.058780 1166 raft_consensus.cc:359] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.058933 1166 raft_consensus.cc:385] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.059501 1168 raft_consensus.cc:359] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.059830 1168 raft_consensus.cc:385] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.059852 1166 raft_consensus.cc:740] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.059965 1168 raft_consensus.cc:740] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.060101 1166 consensus_queue.cc:260] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.060194 1168 consensus_queue.cc:260] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.060642 1166 ts_tablet_manager.cc:1434] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251211 20:52:36.060873 1168 ts_tablet_manager.cc:1434] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:36.060945 1166 tablet_bootstrap.cc:492] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:36.061200 1168 tablet_bootstrap.cc:492] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:36.061311 2023 raft_consensus.cc:515] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.062237 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 05385f20e88c46108e09a78814811209 (table default.Table_2a5ff353268344dba84653a0fd41e4a0 [id=c06795252c09401b99e58c702574ac05]) successfully deleted
I20251211 20:52:36.062590 1168 tablet_bootstrap.cc:654] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.062675 1166 tablet_bootstrap.cc:654] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.063911 1165 tablet_replica.cc:333] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:36.065266 1696 raft_consensus.cc:493] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.065394 1696 raft_consensus.cc:515] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.065943 1696 leader_election.cc:290] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:36.067821 710 leader_election.cc:343] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 340730a17a834a77bbc20b95aba98e2c
I20251211 20:52:36.068867 1165 raft_consensus.cc:2243] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.071502 1168 tablet_bootstrap.cc:492] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:36.071280 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ef4fada332cf4b5c9da26ba5f908b4c6" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:36.071676 1168 ts_tablet_manager.cc:1403] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20251211 20:52:36.070420 1165 raft_consensus.cc:2272] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.073175 1166 tablet_bootstrap.cc:492] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:36.073362 1166 ts_tablet_manager.cc:1403] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20251211 20:52:36.074105 1166 raft_consensus.cc:359] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.074290 1166 raft_consensus.cc:385] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.074352 1166 raft_consensus.cc:740] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.074635 1166 consensus_queue.cc:260] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.075119 1166 ts_tablet_manager.cc:1434] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:36.075733 1166 tablet_bootstrap.cc:492] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:36.071964 1046 raft_consensus.cc:3060] T ef4fada332cf4b5c9da26ba5f908b4c6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:36.072228 2023 leader_election.cc:290] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:36.080595 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8feee01fc0aa4b32be926ff2b617b894" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:36.082260 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ef4fada332cf4b5c9da26ba5f908b4c6" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:36.082425 914 raft_consensus.cc:3060] T ef4fada332cf4b5c9da26ba5f908b4c6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:36.078708 1166 tablet_bootstrap.cc:654] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.085059 914 raft_consensus.cc:2468] T ef4fada332cf4b5c9da26ba5f908b4c6 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:36.085917 2000 raft_consensus.cc:493] T c7a2610f20df4c1ca81eae2fc7dba6bb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.086081 2000 raft_consensus.cc:515] T c7a2610f20df4c1ca81eae2fc7dba6bb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.085713 1165 ts_tablet_manager.cc:1916] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:36.086701 2000 leader_election.cc:290] T c7a2610f20df4c1ca81eae2fc7dba6bb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:36.087419 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7a2610f20df4c1ca81eae2fc7dba6bb" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:36.081007 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "340730a17a834a77bbc20b95aba98e2c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:36.087930 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bc32a778e5424850b23ccaf74bb9df07" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:36.088054 982 leader_election.cc:343] T c7a2610f20df4c1ca81eae2fc7dba6bb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.092204 912 raft_consensus.cc:3060] T 1843dc4548134f29969d46fbc760f5d7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:36.088712 711 leader_election.cc:343] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.072355 1168 raft_consensus.cc:359] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.095724 1168 raft_consensus.cc:385] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.095785 1168 raft_consensus.cc:740] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.095995 1168 consensus_queue.cc:260] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.095458 711 leader_election.cc:304] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters:
I20251211 20:52:36.096509 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7a2610f20df4c1ca81eae2fc7dba6bb" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:36.096673 711 leader_election.cc:343] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.096884 2018 raft_consensus.cc:2804] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:36.096999 2018 raft_consensus.cc:697] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:36.097012 1046 raft_consensus.cc:2468] T ef4fada332cf4b5c9da26ba5f908b4c6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:36.097213 2018 consensus_queue.cc:237] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.097612 2013 tablet.cc:1717] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919: Skipping requested alter to schema version 2, tablet already version 2
W20251211 20:52:36.097771 983 leader_election.cc:343] T c7a2610f20df4c1ca81eae2fc7dba6bb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.097911 983 leader_election.cc:304] T c7a2610f20df4c1ca81eae2fc7dba6bb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.096779 711 leader_election.cc:304] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.098222 1696 raft_consensus.cc:2749] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:36.098282 2000 raft_consensus.cc:2749] T c7a2610f20df4c1ca81eae2fc7dba6bb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:36.098354 711 leader_election.cc:343] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.098443 711 leader_election.cc:304] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.098642 1696 raft_consensus.cc:2749] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:36.098912 1696 consensus_queue.cc:1048] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:36.102382 1168 ts_tablet_manager.cc:1434] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.031s user 0.002s sys 0.000s
I20251211 20:52:36.102705 1168 tablet_bootstrap.cc:492] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:36.105711 1168 tablet_bootstrap.cc:654] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.109014 1166 tablet_bootstrap.cc:492] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:36.109189 1166 ts_tablet_manager.cc:1403] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.004s
I20251211 20:52:36.109970 1166 raft_consensus.cc:359] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.110167 1166 raft_consensus.cc:385] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.110248 1166 raft_consensus.cc:740] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.111335 591 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_d465fa4868dd4c4db1036653f8b6f463 [id=cb97ccf262fd4c41a180267fd0678cec] alter complete (version 3)
I20251211 20:52:36.113667 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f541a176465249d8aaba954bc23bcf01" } modify_external_catalogs: true
I20251211 20:52:36.113879 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f541a176465249d8aaba954bc23bcf01" } modify_external_catalogs: true
I20251211 20:52:36.114605 1166 consensus_queue.cc:260] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.115222 1166 ts_tablet_manager.cc:1434] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20251211 20:52:36.116235 1168 tablet_bootstrap.cc:492] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:36.116364 1168 ts_tablet_manager.cc:1403] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.001s
I20251211 20:52:36.116533 2000 raft_consensus.cc:493] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.116691 2000 raft_consensus.cc:515] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.117156 1168 raft_consensus.cc:359] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.117326 1168 raft_consensus.cc:385] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.117369 1168 raft_consensus.cc:740] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.117359 2000 leader_election.cc:290] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:36.118144 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb112b6431af42a6a7b328dfdefbc2a2" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:36.119153 982 leader_election.cc:343] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.120426 1028 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a526a6d1bc84cefad2c5d897223292e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:36.120782 1028 tablet_service.cc:1552] Processing DeleteTablet for tablet 4201da4241aa48dfb670a713e4453ca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:36.120980 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 6c64fd8dd49b4e5d91c1ffb0fc2559e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:36.121198 1028 tablet_service.cc:1552] Processing DeleteTablet for tablet 87863daf77304a399318cdf56da9f9bb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:36.121369 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 92a71dff4ea8445d8484fc2747471148 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:36.121618 1011 tablet_service.cc:1552] Processing DeleteTablet for tablet 25454e67437541d9ac2bfab29bad55f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:36.121551 1168 consensus_queue.cc:260] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.122141 1168 ts_tablet_manager.cc:1434] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20251211 20:52:36.122258 2000 raft_consensus.cc:493] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.122403 1168 tablet_bootstrap.cc:492] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:36.122399 2000 raft_consensus.cc:515] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.123013 2000 leader_election.cc:290] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:36.115571 1166 tablet_bootstrap.cc:492] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:36.123993 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2dde7f28e61d41f48015ba6e4f3757f2" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:36.124693 982 leader_election.cc:343] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.125111 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 10746162da4640e6a2a891467690b770
I20251211 20:52:36.125654 592 catalog_manager.cc:6724] ReplaceTablet: tablet 10746162da4640e6a2a891467690b770 deleted and replaced by tablet 8dc4e730951144d78b77280f6cce6d0e
I20251211 20:52:36.125810 1166 tablet_bootstrap.cc:654] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.125982 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_25aba30fb89c47c5b77403382971cf65" } modify_external_catalogs: true
I20251211 20:52:36.126164 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_25aba30fb89c47c5b77403382971cf65" } modify_external_catalogs: true
I20251211 20:52:36.128302 588 master_service.cc:954] ReplaceTablet: received request to replace tablet 7474a08c51cf4346ae9a518963d56b7a from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:36.129561 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a95581dfcc7a448ea004e1b657388be9" } modify_external_catalogs: true
I20251211 20:52:36.129755 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a95581dfcc7a448ea004e1b657388be9" } modify_external_catalogs: true
I20251211 20:52:36.130208 1168 tablet_bootstrap.cc:654] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.131790 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 2f7b1bc3dd1f4496b3ac6e71652cf012
I20251211 20:52:36.132510 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4e49f78c60234aea972f8f5068e89b9f" } modify_external_catalogs: true
I20251211 20:52:36.132670 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4e49f78c60234aea972f8f5068e89b9f" } modify_external_catalogs: true
I20251211 20:52:36.134662 1165 ts_tablet_manager.cc:1929] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:36.135406 1165 log.cc:1199] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/55c40b2464f8444ca6c9fa0400f7f7f2
I20251211 20:52:36.135538 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 10746162da4640e6a2a891467690b770 with delete_type TABLET_DATA_DELETED (replaced by tablet 8dc4e730951144d78b77280f6cce6d0e) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.135731 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f7b1bc3dd1f4496b3ac6e71652cf012 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.136328 1165 ts_tablet_manager.cc:1950] T 55c40b2464f8444ca6c9fa0400f7f7f2 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:36.136446 1011 tablet_service.cc:1552] Processing DeleteTablet for tablet 10746162da4640e6a2a891467690b770 with delete_type TABLET_DATA_DELETED (replaced by tablet 8dc4e730951144d78b77280f6cce6d0e) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:36.136705 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f7b1bc3dd1f4496b3ac6e71652cf012 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:36.137753 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 55c40b2464f8444ca6c9fa0400f7f7f2 (table default.Table_069bb219e15e43be995ff011e2289a84 [id=bc0b945e86cf43fdb99d56658ca1576c]) successfully deleted
W20251211 20:52:36.139206 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet d53b074865f4490693152a247b0fffdc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d53b074865f4490693152a247b0fffdc already in progress: creating tablet
W20251211 20:52:36.140322 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet de508d79f30f489cb745401f53115165 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet de508d79f30f489cb745401f53115165 already in progress: creating tablet
W20251211 20:52:36.141896 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 10746162da4640e6a2a891467690b770 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 10746162da4640e6a2a891467690b770 already in progress: creating tablet
I20251211 20:52:36.142230 1166 tablet_bootstrap.cc:492] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
W20251211 20:52:36.143849 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 2f7b1bc3dd1f4496b3ac6e71652cf012 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f7b1bc3dd1f4496b3ac6e71652cf012 already in progress: creating tablet
I20251211 20:52:36.144284 1166 ts_tablet_manager.cc:1403] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.029s user 0.005s sys 0.000s
I20251211 20:52:36.145263 1166 raft_consensus.cc:359] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.146623 1166 raft_consensus.cc:385] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.148900 1166 raft_consensus.cc:740] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.149312 1166 consensus_queue.cc:260] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.149979 1166 ts_tablet_manager.cc:1434] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251211 20:52:36.150905 1168 tablet_bootstrap.cc:492] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:36.151186 1168 ts_tablet_manager.cc:1403] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.002s
I20251211 20:52:36.152401 1166 tablet_bootstrap.cc:492] T 72e23f3033e14c9db6de8539830bac3b P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:36.156306 1166 tablet_bootstrap.cc:654] T 72e23f3033e14c9db6de8539830bac3b P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.156482 756 tablet_service.cc:1552] Processing DeleteTablet for tablet bc32a778e5424850b23ccaf74bb9df07 with delete_type TABLET_DATA_DELETED (replaced by tablet 6a206abd6ea2420089cb07714e0d19c0) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.157325 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 340730a17a834a77bbc20b95aba98e2c with delete_type TABLET_DATA_DELETED (replaced by tablet 3b2c6064c5eb4c0f8979c485f5d85a75) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.158529 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d with delete_type TABLET_DATA_DELETED (replaced by tablet e4647105ab57454d883ebfcde7bde891) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.159684 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_983c3bbea3c54b88bebbe065e4660218"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:36.160326 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_983c3bbea3c54b88bebbe065e4660218 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:36.161465 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 3c578d81e8dd41aaa0f707c4e3fed225 with delete_type TABLET_DATA_DELETED (replaced by tablet b115d737d7fc483d9f5a1f040b511bae) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.161763 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 822abda3c85848728205b71847ef7029 with delete_type TABLET_DATA_DELETED (replaced by tablet f9a3fdf17f274c15ac0bdfa9c174f649) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.162079 1165 tablet_replica.cc:333] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:36.162245 1165 raft_consensus.cc:2243] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.162398 1165 raft_consensus.cc:2272] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.163059 1168 raft_consensus.cc:359] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.163236 1168 raft_consensus.cc:385] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.163281 1168 raft_consensus.cc:740] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.163920 1165 ts_tablet_manager.cc:1916] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:36.164072 2008 raft_consensus.cc:493] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.166034 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 10746162da4640e6a2a891467690b770 with delete_type TABLET_DATA_DELETED (replaced by tablet 8dc4e730951144d78b77280f6cce6d0e) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:36.166237 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 804a11c8f7414eb1a6bcff2952be9ae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:36.166257 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 605172f066d242f2abea133c08a145ca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:36.166507 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f7b1bc3dd1f4496b3ac6e71652cf012 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:36.166601 1164 tablet_replica.cc:333] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:36.166709 889 tablet_service.cc:1552] Processing DeleteTablet for tablet c85624ad640041c29bb87416e1992a9e with delete_type TABLET_DATA_DELETED (replaced by tablet fa79b67b9eee45e2b41c4a90967606cc) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:36.166782 1164 raft_consensus.cc:2243] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.166916 1164 raft_consensus.cc:2272] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.166939 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a2442a4ebc940a281f9ea1a394f6621 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:36.138114 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb112b6431af42a6a7b328dfdefbc2a2" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:36.167443 1164 ts_tablet_manager.cc:1916] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:36.167546 983 leader_election.cc:343] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.167740 983 leader_election.cc:304] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.167914 1168 consensus_queue.cc:260] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.168357 2000 raft_consensus.cc:2749] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:36.168432 1168 ts_tablet_manager.cc:1434] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.017s user 0.001s sys 0.001s
I20251211 20:52:36.168558 2008 raft_consensus.cc:515] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.168744 1168 tablet_bootstrap.cc:492] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:36.167656 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2dde7f28e61d41f48015ba6e4f3757f2" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:36.169719 983 leader_election.cc:343] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.169872 983 leader_election.cc:304] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.170315 2000 raft_consensus.cc:2749] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:36.167292 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 8139acb0ae324d0f973ce877edc1cd04 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:36.171352 889 tablet_service.cc:1552] Processing DeleteTablet for tablet d5f0ba3ab16240f9822be69f941f5da2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:36.172662 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 10746162da4640e6a2a891467690b770 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 10746162da4640e6a2a891467690b770 already in progress: creating tablet
I20251211 20:52:36.173709 1168 tablet_bootstrap.cc:654] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.176129 1166 tablet_bootstrap.cc:492] T 72e23f3033e14c9db6de8539830bac3b P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:36.176296 1166 ts_tablet_manager.cc:1403] T 72e23f3033e14c9db6de8539830bac3b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.000s
I20251211 20:52:36.177263 1166 raft_consensus.cc:359] T 72e23f3033e14c9db6de8539830bac3b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.177477 1166 raft_consensus.cc:385] T 72e23f3033e14c9db6de8539830bac3b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.180320 1166 raft_consensus.cc:740] T 72e23f3033e14c9db6de8539830bac3b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.180647 2000 raft_consensus.cc:493] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.180595 1166 consensus_queue.cc:260] T 72e23f3033e14c9db6de8539830bac3b P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.180814 2000 raft_consensus.cc:515] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.181157 1166 ts_tablet_manager.cc:1434] T 72e23f3033e14c9db6de8539830bac3b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251211 20:52:36.181396 1165 ts_tablet_manager.cc:1929] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:36.181468 1166 tablet_bootstrap.cc:492] T 3b2c6064c5eb4c0f8979c485f5d85a75 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:36.181505 1165 log.cc:1199] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/bc32a778e5424850b23ccaf74bb9df07
I20251211 20:52:36.181787 1165 ts_tablet_manager.cc:1950] T bc32a778e5424850b23ccaf74bb9df07 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:36.182418 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet bc32a778e5424850b23ccaf74bb9df07 (table default.Table_e6252813479645a98ce2709dc7aba798 [id=db38368266194271a02ad265fd991439]) successfully deleted
I20251211 20:52:36.182540 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb030a62796248869d46b682656eaeb0" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:36.182835 1165 tablet_replica.cc:333] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:36.183018 1165 raft_consensus.cc:2243] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.183179 1166 tablet_bootstrap.cc:654] T 3b2c6064c5eb4c0f8979c485f5d85a75 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.183171 1165 raft_consensus.cc:2272] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:36.183269 982 leader_election.cc:343] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.181473 2000 leader_election.cc:290] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:36.183740 1165 ts_tablet_manager.cc:1916] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:36.183892 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb030a62796248869d46b682656eaeb0" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:36.184437 2008 leader_election.cc:290] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:36.184438 983 leader_election.cc:343] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.184577 983 leader_election.cc:304] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.185091 2000 raft_consensus.cc:2749] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:36.184595 1168 tablet_bootstrap.cc:492] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:36.185607 1168 ts_tablet_manager.cc:1403] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.002s
I20251211 20:52:36.186703 1168 raft_consensus.cc:359] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.186965 1168 raft_consensus.cc:385] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.187013 1168 raft_consensus.cc:740] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.187189 1168 consensus_queue.cc:260] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.187678 1168 ts_tablet_manager.cc:1434] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:36.187944 1168 tablet_bootstrap.cc:492] T 97ada0641165444a998db26796793507 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:36.189716 1168 tablet_bootstrap.cc:654] T 97ada0641165444a998db26796793507 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.191694 1164 ts_tablet_manager.cc:1929] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:36.191812 1164 log.cc:1199] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/804a11c8f7414eb1a6bcff2952be9ae2
I20251211 20:52:36.192063 1164 ts_tablet_manager.cc:1950] T 804a11c8f7414eb1a6bcff2952be9ae2 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:36.192658 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 804a11c8f7414eb1a6bcff2952be9ae2 (table default.Table_415c09e71243460aaa9acfd57742bcd7 [id=7f9092a9ef3945a7851d0d9f762e161a]) successfully deleted
I20251211 20:52:36.193596 1164 tablet_replica.cc:333] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:36.193780 1164 raft_consensus.cc:2243] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.193928 1164 raft_consensus.cc:2272] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.194411 1164 ts_tablet_manager.cc:1916] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:36.196478 2018 raft_consensus.cc:493] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.196633 2018 raft_consensus.cc:515] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.197211 2018 leader_election.cc:290] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:36.197782 742 tablet_service.cc:1505] Processing CreateTablet for tablet 7ffa06c4a7ee48b0b337a27aff090592 (DEFAULT_TABLE table=default.Table_647731023b044ebb8954f7fdcc3faf97 [id=4b91abd6e9e94a8a941b33e62e23c083]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.198163 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ffa06c4a7ee48b0b337a27aff090592. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.198541 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 1944267def4e46949fd680ab0a369be9 (DEFAULT_TABLE table=default.Table_514c179c4a144ce398e7c9b8fbba2bce [id=42d5ddf23fd549d0982312ace7df498e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.198938 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1944267def4e46949fd680ab0a369be9. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:36.204895 710 leader_election.cc:343] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 60d966c0397c4a4ea3718445e87e9703
I20251211 20:52:36.205690 2000 raft_consensus.cc:493] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.205900 2000 raft_consensus.cc:515] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.206583 2000 leader_election.cc:290] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
W20251211 20:52:36.209575 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet d53b074865f4490693152a247b0fffdc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d53b074865f4490693152a247b0fffdc already in progress: creating tablet
I20251211 20:52:36.211050 886 tablet_service.cc:1505] Processing CreateTablet for tablet cb3937bc6d6a49928af633d10bdd7074 (DEFAULT_TABLE table=default.Table_e2163380d2474e9a93f10e69de094195 [id=041f6b72446c42f68aa2fcaa8438ad0c]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:36.211145 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet de508d79f30f489cb745401f53115165 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet de508d79f30f489cb745401f53115165 already in progress: creating tablet
I20251211 20:52:36.211474 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb3937bc6d6a49928af633d10bdd7074. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.211488 1165 ts_tablet_manager.cc:1929] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:36.211635 1165 log.cc:1199] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/340730a17a834a77bbc20b95aba98e2c
I20251211 20:52:36.211918 1165 ts_tablet_manager.cc:1950] T 340730a17a834a77bbc20b95aba98e2c P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:36.212088 1163 tablet_replica.cc:333] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:36.212510 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 340730a17a834a77bbc20b95aba98e2c (table default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f]) successfully deleted
I20251211 20:52:36.213435 1011 tablet_service.cc:1505] Processing CreateTablet for tablet 7ffa06c4a7ee48b0b337a27aff090592 (DEFAULT_TABLE table=default.Table_647731023b044ebb8954f7fdcc3faf97 [id=4b91abd6e9e94a8a941b33e62e23c083]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.213864 1168 tablet_bootstrap.cc:492] T 97ada0641165444a998db26796793507 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:36.214000 1168 ts_tablet_manager.cc:1403] T 97ada0641165444a998db26796793507 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.000s
I20251211 20:52:36.214783 1011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ffa06c4a7ee48b0b337a27aff090592. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.214784 1168 raft_consensus.cc:359] T 97ada0641165444a998db26796793507 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.214952 1168 raft_consensus.cc:385] T 97ada0641165444a998db26796793507 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.214995 1168 raft_consensus.cc:740] T 97ada0641165444a998db26796793507 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.215181 1168 consensus_queue.cc:260] T 97ada0641165444a998db26796793507 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.215903 742 tablet_service.cc:1505] Processing CreateTablet for tablet cb3937bc6d6a49928af633d10bdd7074 (DEFAULT_TABLE table=default.Table_e2163380d2474e9a93f10e69de094195 [id=041f6b72446c42f68aa2fcaa8438ad0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.216279 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb3937bc6d6a49928af633d10bdd7074. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.217844 1168 ts_tablet_manager.cc:1434] T 97ada0641165444a998db26796793507 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:36.218139 1168 tablet_bootstrap.cc:492] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:36.219848 1168 tablet_bootstrap.cc:654] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.220580 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a2058b67ddf1413db338bfa2f64e7a77" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:36.223270 754 tablet_service.cc:1505] Processing CreateTablet for tablet 80c566822fa94ada983aa7266df4bf9d (DEFAULT_TABLE table=default.Table_6b058d9ed449475e986f56f95463634a [id=330579216bb4452093f90d19c05281f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.223645 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80c566822fa94ada983aa7266df4bf9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.230480 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "60d966c0397c4a4ea3718445e87e9703" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:36.230803 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a2058b67ddf1413db338bfa2f64e7a77" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:36.231366 711 leader_election.cc:343] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.231487 711 leader_election.cc:304] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.232017 1696 raft_consensus.cc:2749] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:36.196712 885 tablet_service.cc:1505] Processing CreateTablet for tablet 7ffa06c4a7ee48b0b337a27aff090592 (DEFAULT_TABLE table=default.Table_647731023b044ebb8954f7fdcc3faf97 [id=4b91abd6e9e94a8a941b33e62e23c083]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.233661 889 tablet_service.cc:1505] Processing CreateTablet for tablet 80c566822fa94ada983aa7266df4bf9d (DEFAULT_TABLE table=default.Table_6b058d9ed449475e986f56f95463634a [id=330579216bb4452093f90d19c05281f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.232627 1165 tablet_replica.cc:333] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:36.234516 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ffa06c4a7ee48b0b337a27aff090592. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.234381 1165 raft_consensus.cc:2243] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.234875 1165 raft_consensus.cc:2272] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:36.235038 847 leader_election.cc:343] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 605172f066d242f2abea133c08a145ca
W20251211 20:52:36.235224 847 leader_election.cc:343] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 605172f066d242f2abea133c08a145ca
I20251211 20:52:36.235318 847 leader_election.cc:304] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
W20251211 20:52:36.235463 847 raft_consensus.cc:2686] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251211 20:52:36.235443 1165 ts_tablet_manager.cc:1916] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:36.236533 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80c566822fa94ada983aa7266df4bf9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.240938 1168 tablet_bootstrap.cc:492] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:36.241091 1168 ts_tablet_manager.cc:1403] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.000s
I20251211 20:52:36.242017 1168 raft_consensus.cc:359] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.242199 1168 raft_consensus.cc:385] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.242249 1168 raft_consensus.cc:740] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.242444 1168 consensus_queue.cc:260] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.243163 1168 ts_tablet_manager.cc:1434] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:36.243467 1168 tablet_bootstrap.cc:492] T 648f43e95e034f2188a20e1b837f2818 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:36.245390 888 tablet_service.cc:1505] Processing CreateTablet for tablet a98a2b091d9b44a7b954ddb45c819da9 (DEFAULT_TABLE table=default.Table_2fddb279b6164c2ba9e81fc55dfeb59a [id=f8c4f76e6a2d48fb93603b9d308a71ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.247825 1168 tablet_bootstrap.cc:654] T 648f43e95e034f2188a20e1b837f2818 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.250061 742 tablet_service.cc:1505] Processing CreateTablet for tablet a98a2b091d9b44a7b954ddb45c819da9 (DEFAULT_TABLE table=default.Table_2fddb279b6164c2ba9e81fc55dfeb59a [id=f8c4f76e6a2d48fb93603b9d308a71ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.250592 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a98a2b091d9b44a7b954ddb45c819da9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.258766 1165 ts_tablet_manager.cc:1929] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:36.261652 883 tablet_service.cc:1505] Processing CreateTablet for tablet 0350405f1d5844d2a987697fd0d8850c (DEFAULT_TABLE table=default.Table_c1cf2780cc3941688af6d55d57d9e6a8 [id=9aa1c4d2d8a34aa5a6170b5c8e502581]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.266637 754 tablet_service.cc:1505] Processing CreateTablet for tablet 0350405f1d5844d2a987697fd0d8850c (DEFAULT_TABLE table=default.Table_c1cf2780cc3941688af6d55d57d9e6a8 [id=9aa1c4d2d8a34aa5a6170b5c8e502581]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.267836 1165 log.cc:1199] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/3c578d81e8dd41aaa0f707c4e3fed225
I20251211 20:52:36.267973 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0350405f1d5844d2a987697fd0d8850c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.268270 1165 ts_tablet_manager.cc:1950] T 3c578d81e8dd41aaa0f707c4e3fed225 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:36.268972 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 3c578d81e8dd41aaa0f707c4e3fed225 (table default.Table_6d94053661f142b79292e19752df02df [id=c5be59a2c3934b3789641ceeb2d4d6e2]) successfully deleted
I20251211 20:52:36.269402 1165 tablet_replica.cc:333] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:36.269827 1165 raft_consensus.cc:2243] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.274989 1165 raft_consensus.cc:2272] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.276297 1165 ts_tablet_manager.cc:1916] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:36.276994 1168 tablet_bootstrap.cc:492] T 648f43e95e034f2188a20e1b837f2818 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:36.277220 1168 ts_tablet_manager.cc:1403] T 648f43e95e034f2188a20e1b837f2818 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.034s user 0.005s sys 0.000s
I20251211 20:52:36.278025 1168 raft_consensus.cc:359] T 648f43e95e034f2188a20e1b837f2818 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.278273 1168 raft_consensus.cc:385] T 648f43e95e034f2188a20e1b837f2818 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.278371 1168 raft_consensus.cc:740] T 648f43e95e034f2188a20e1b837f2818 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.278607 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a98a2b091d9b44a7b954ddb45c819da9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.278627 1168 consensus_queue.cc:260] T 648f43e95e034f2188a20e1b837f2818 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.278834 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0350405f1d5844d2a987697fd0d8850c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.279126 1168 ts_tablet_manager.cc:1434] T 648f43e95e034f2188a20e1b837f2818 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:36.283871 1168 tablet_bootstrap.cc:492] T 33addf3d158941098f240cfcdec45d28 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:36.283739 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_47ff5fcc1088481592447a200ff8432f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:36.284399 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_47ff5fcc1088481592447a200ff8432f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:36.284379 889 tablet_service.cc:1505] Processing CreateTablet for tablet d1ad0dc52d4548e0a9a7b92eae0e2c0e (DEFAULT_TABLE table=default.Table_76916c86042b49f487d21bf7aabfed56 [id=365cb9cb979d45dbaaff790819c90412]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.285712 1168 tablet_bootstrap.cc:654] T 33addf3d158941098f240cfcdec45d28 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.285948 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_01821876da7e47a1a5f2cce0f011e20c" } new_table_name: "default.Table_cb6a4bac7b694f33a37de2ee78c91ca0" modify_external_catalogs: true
I20251211 20:52:36.287343 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_e9b10d4bc5db49289ea753470f1585a6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:36.287552 742 tablet_service.cc:1505] Processing CreateTablet for tablet d1ad0dc52d4548e0a9a7b92eae0e2c0e (DEFAULT_TABLE table=default.Table_76916c86042b49f487d21bf7aabfed56 [id=365cb9cb979d45dbaaff790819c90412]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:36.288007 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e9b10d4bc5db49289ea753470f1585a6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:36.288949 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1ad0dc52d4548e0a9a7b92eae0e2c0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.294401 2018 raft_consensus.cc:493] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.294605 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1ad0dc52d4548e0a9a7b92eae0e2c0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.294550 2018 raft_consensus.cc:515] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.295125 2018 leader_election.cc:290] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:36.296013 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "822abda3c85848728205b71847ef7029" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:36.296567 711 leader_election.cc:343] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.298241 2018 raft_consensus.cc:493] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.298362 2018 raft_consensus.cc:515] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.298493 881 tablet_service.cc:1505] Processing CreateTablet for tablet 2ef524747034491c99f4f9767acd3c9d (DEFAULT_TABLE table=default.Table_3f5d0d2b042c42ed88431232445bef02 [id=19e89fd1209c4cad8ef39476f607f3ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.298729 1165 ts_tablet_manager.cc:1929] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:36.298840 1165 log.cc:1199] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/6dfa7480e2ab49b9b8a938da0dc6cf0d
I20251211 20:52:36.298918 2018 leader_election.cc:290] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:36.299108 1165 ts_tablet_manager.cc:1950] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:36.299716 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5bd801124df84a6c96f4912993f1bdfd" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:36.300258 711 leader_election.cc:343] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.300817 1168 tablet_bootstrap.cc:492] T 33addf3d158941098f240cfcdec45d28 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:36.301175 1168 ts_tablet_manager.cc:1403] T 33addf3d158941098f240cfcdec45d28 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.003s
I20251211 20:52:36.301466 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d (table default.Table_c817dec9a7674f699b3366fc302ead57 [id=d555d1fc6aef4a21b4d3f2804264d030]) successfully deleted
I20251211 20:52:36.301973 1168 raft_consensus.cc:359] T 33addf3d158941098f240cfcdec45d28 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.302130 1168 raft_consensus.cc:385] T 33addf3d158941098f240cfcdec45d28 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.302181 1168 raft_consensus.cc:740] T 33addf3d158941098f240cfcdec45d28 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.302412 1165 tablet_replica.cc:333] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:36.302393 1168 consensus_queue.cc:260] T 33addf3d158941098f240cfcdec45d28 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.302706 1165 raft_consensus.cc:2243] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.306401 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ef524747034491c99f4f9767acd3c9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.306584 1165 raft_consensus.cc:2272] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.307148 1165 ts_tablet_manager.cc:1916] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:36.302939 1168 ts_tablet_manager.cc:1434] T 33addf3d158941098f240cfcdec45d28 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:36.308539 1168 tablet_bootstrap.cc:492] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:36.310225 1168 tablet_bootstrap.cc:654] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.306322 754 tablet_service.cc:1505] Processing CreateTablet for tablet 2ef524747034491c99f4f9767acd3c9d (DEFAULT_TABLE table=default.Table_3f5d0d2b042c42ed88431232445bef02 [id=19e89fd1209c4cad8ef39476f607f3ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.311502 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ef524747034491c99f4f9767acd3c9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.320448 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 0b267c7feaac43f6aacb7229759c7ec3
I20251211 20:52:36.321686 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 0b267c7feaac43f6aacb7229759c7ec3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.322623 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 0b267c7feaac43f6aacb7229759c7ec3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:36.322762 590 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 1843dc4548134f29969d46fbc760f5d7 (table default.Table_99917cb4c70840e19004ad381746d77e [id=6abb187bfd214e91b2e11267065a2841]). Expected version 1 got 0
I20251211 20:52:36.324667 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet e8db5a11a69e4554bc689c38ab57b28a
I20251211 20:52:36.325393 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_265379fa17b74d0f954a2a2c4de113c8" } new_table_name: "default.Table_39446be1bf754b5da6d932892c7280cf" modify_external_catalogs: true
I20251211 20:52:36.325518 757 tablet_service.cc:1552] Processing DeleteTablet for tablet e8db5a11a69e4554bc689c38ab57b28a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.325927 885 tablet_service.cc:1552] Processing DeleteTablet for tablet e8db5a11a69e4554bc689c38ab57b28a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:36.326881 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 76cfb26762d54593a150ec98125c6592
I20251211 20:52:36.328125 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 76cfb26762d54593a150ec98125c6592 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.328094 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_d571120dedcb48cd9d4b75314be3a654" } new_table_name: "default.Table_c6c5666f7b3044589afd38743a1a200c" modify_external_catalogs: true
I20251211 20:52:36.328364 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 76cfb26762d54593a150ec98125c6592 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:36.330204 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet c4e048ca28824329b924845ccf5067cf
I20251211 20:52:36.331696 885 tablet_service.cc:1552] Processing DeleteTablet for tablet c4e048ca28824329b924845ccf5067cf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:36.331987 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_5d7c1dcaff1a420c95fb2907b7cce4b8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:36.332374 1168 tablet_bootstrap.cc:492] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:36.332505 1168 ts_tablet_manager.cc:1403] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.001s
W20251211 20:52:36.332552 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5d7c1dcaff1a420c95fb2907b7cce4b8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:36.333117 885 tablet_service.cc:1505] Processing CreateTablet for tablet 652828db3e31467fad8391e6cfe70846 (DEFAULT_TABLE table=default.Table_20456517a58b40fabd534461d2ab6cc0 [id=f93e5d8158d74f7cb3b77ddefe065c8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.333272 1168 raft_consensus.cc:359] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.333428 1168 raft_consensus.cc:385] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.333484 1168 raft_consensus.cc:740] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.333662 1168 consensus_queue.cc:260] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.333792 1165 ts_tablet_manager.cc:1929] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:36.333904 1165 log.cc:1199] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/822abda3c85848728205b71847ef7029
I20251211 20:52:36.334031 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 652828db3e31467fad8391e6cfe70846. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.334127 1168 ts_tablet_manager.cc:1434] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:36.334453 1168 tablet_bootstrap.cc:492] T 68be21a25c6743f5b86d518d98fbc35e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:36.336098 1168 tablet_bootstrap.cc:654] T 68be21a25c6743f5b86d518d98fbc35e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.339267 757 tablet_service.cc:1505] Processing CreateTablet for tablet 652828db3e31467fad8391e6cfe70846 (DEFAULT_TABLE table=default.Table_20456517a58b40fabd534461d2ab6cc0 [id=f93e5d8158d74f7cb3b77ddefe065c8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.339715 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 652828db3e31467fad8391e6cfe70846. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.334167 1165 ts_tablet_manager.cc:1950] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:36.343456 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 7474a08c51cf4346ae9a518963d56b7a
I20251211 20:52:36.343978 588 catalog_manager.cc:6724] ReplaceTablet: tablet 7474a08c51cf4346ae9a518963d56b7a deleted and replaced by tablet e95c63b4d99a4dd0b2c0fa384ca9d552
I20251211 20:52:36.344349 588 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e4656c3f3f974d68af3f207e635d316d" } new_table_name: "default.Table_279bb02bdbd743159babc9a4e6498437" modify_external_catalogs: true
W20251211 20:52:36.346087 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a7afcce522494c9ca1d80e6eea913e6e consensus state has no leader
I20251211 20:52:36.346421 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_d231bb862aa24617b1f92793007bf755" } new_table_name: "default.Table_e3a99b043e4f420a93ebfee6aa87f2ec" modify_external_catalogs: true
I20251211 20:52:36.347426 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 822abda3c85848728205b71847ef7029 (table default.Table_4aad4cf01eff415f86b8446c29977007 [id=01282adb7b834261a7750c350ad0b250]) successfully deleted
W20251211 20:52:36.347667 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 0b267c7feaac43f6aacb7229759c7ec3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0b267c7feaac43f6aacb7229759c7ec3 already in progress: creating tablet
W20251211 20:52:36.347882 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet e8db5a11a69e4554bc689c38ab57b28a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e8db5a11a69e4554bc689c38ab57b28a already in progress: creating tablet
W20251211 20:52:36.348086 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 76cfb26762d54593a150ec98125c6592 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 76cfb26762d54593a150ec98125c6592 already in progress: creating tablet
I20251211 20:52:36.348977 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 7474a08c51cf4346ae9a518963d56b7a with delete_type TABLET_DATA_DELETED (replaced by tablet e95c63b4d99a4dd0b2c0fa384ca9d552) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.351681 1168 tablet_bootstrap.cc:492] T 68be21a25c6743f5b86d518d98fbc35e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:36.351691 1165 tablet_replica.cc:333] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:36.351843 1168 ts_tablet_manager.cc:1403] T 68be21a25c6743f5b86d518d98fbc35e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.000s
I20251211 20:52:36.351899 1165 raft_consensus.cc:2243] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:36.352263 1165 raft_consensus.cc:2272] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.352514 1168 raft_consensus.cc:359] T 68be21a25c6743f5b86d518d98fbc35e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.352674 1168 raft_consensus.cc:385] T 68be21a25c6743f5b86d518d98fbc35e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.352546 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_9fa0ba597863437e94957c10e3a68b4b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:36.352725 1168 raft_consensus.cc:740] T 68be21a25c6743f5b86d518d98fbc35e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.352912 1168 consensus_queue.cc:260] T 68be21a25c6743f5b86d518d98fbc35e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.352995 1165 ts_tablet_manager.cc:1916] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:36.353125 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9fa0ba597863437e94957c10e3a68b4b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:36.353358 1168 ts_tablet_manager.cc:1434] T 68be21a25c6743f5b86d518d98fbc35e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251211 20:52:36.353606 1168 tablet_bootstrap.cc:492] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:36.354908 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_f762d4c1a3c44c63abb1c9cd599ac9ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:36.355091 1168 tablet_bootstrap.cc:654] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:36.355477 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f762d4c1a3c44c63abb1c9cd599ac9ed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:36.357797 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_25841fca409b44329301efe7768c649b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:36.358335 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_25841fca409b44329301efe7768c649b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:36.359330 1160 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.543s user 0.049s sys 0.086s
W20251211 20:52:36.359442 1160 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.543s user 0.049s sys 0.086s
I20251211 20:52:36.363144 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_fdfecb7c7e1e4448a51918d3ad950323" } modify_external_catalogs: true
I20251211 20:52:36.363355 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_fdfecb7c7e1e4448a51918d3ad950323" } modify_external_catalogs: true
I20251211 20:52:36.361160 1165 ts_tablet_manager.cc:1929] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:36.363186 756 tablet_service.cc:1552] Processing DeleteTablet for tablet d762b4afb5ff447dac3468393826205a with delete_type TABLET_DATA_DELETED (replaced by tablet 2197822e0f11484f9679c4858626ac2c) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.363224 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 28ef13be3b84410d9c786c43502a3478 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.364035 757 tablet_service.cc:1505] Processing CreateTablet for tablet 9b99860ee6be45a0b358e686332d3137 (DEFAULT_TABLE table=default.Table_a5b944a25f4c45cf8c70765ea43d1ac2 [id=0ee0170e570a4d9aa25ea2858e9d9e2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.364193 2018 raft_consensus.cc:493] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.365372 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 7474a08c51cf4346ae9a518963d56b7a with delete_type TABLET_DATA_DELETED (replaced by tablet e95c63b4d99a4dd0b2c0fa384ca9d552) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:36.365499 1160 tablet_bootstrap.cc:492] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:36.365489 2018 raft_consensus.cc:515] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.365618 1160 ts_tablet_manager.cc:1403] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.552s user 0.051s sys 0.086s
I20251211 20:52:36.365679 877 tablet_service.cc:1505] Processing CreateTablet for tablet 9b99860ee6be45a0b358e686332d3137 (DEFAULT_TABLE table=default.Table_a5b944a25f4c45cf8c70765ea43d1ac2 [id=0ee0170e570a4d9aa25ea2858e9d9e2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.366483 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b99860ee6be45a0b358e686332d3137. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.367945 2018 leader_election.cc:290] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:36.367997 1160 raft_consensus.cc:359] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.368188 1160 raft_consensus.cc:385] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.368273 1160 raft_consensus.cc:740] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.369627 1165 log.cc:1199] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/7474a08c51cf4346ae9a518963d56b7a
I20251211 20:52:36.371963 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "773cf53745a64f62817113b33e3182e7" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:36.372515 711 leader_election.cc:343] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.370085 1165 ts_tablet_manager.cc:1950] T 7474a08c51cf4346ae9a518963d56b7a P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:36.380084 883 tablet_service.cc:1505] Processing CreateTablet for tablet 0bbb0b010aae44a3a43abdf910e08b77 (DEFAULT_TABLE table=default.Table_ef6d1803b4bf4d62860e0bbdf745446a [id=b11896be27d848958e5b3fab8f5504a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.380637 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 7474a08c51cf4346ae9a518963d56b7a (table default.Table_d65f65d608a346968cc2f35b9f55912e [id=88e8b1b4524846e18f5a638dee69a688]) successfully deleted
I20251211 20:52:36.381085 1165 tablet_replica.cc:333] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:36.380755 1160 consensus_queue.cc:260] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.381356 1165 raft_consensus.cc:2243] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.381547 1165 raft_consensus.cc:2272] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.381996 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b99860ee6be45a0b358e686332d3137. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.382138 1165 ts_tablet_manager.cc:1916] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:36.390021 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bbb0b010aae44a3a43abdf910e08b77. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.391301 1160 ts_tablet_manager.cc:1434] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.026s user 0.002s sys 0.000s
I20251211 20:52:36.393469 1165 ts_tablet_manager.cc:1929] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20251211 20:52:36.396943 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8feee01fc0aa4b32be926ff2b617b894 consensus state has no leader
W20251211 20:52:36.401683 581 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 7ffa06c4a7ee48b0b337a27aff090592 (table default.Table_647731023b044ebb8954f7fdcc3faf97 [id=4b91abd6e9e94a8a941b33e62e23c083], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 7ffa06c4a7ee48b0b337a27aff090592 consensus state has no leader
I20251211 20:52:36.394488 1168 tablet_bootstrap.cc:492] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:36.402040 1165 log.cc:1199] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/d762b4afb5ff447dac3468393826205a
I20251211 20:52:36.402050 1168 ts_tablet_manager.cc:1403] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.049s user 0.004s sys 0.006s
I20251211 20:52:36.403029 1168 raft_consensus.cc:359] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.403192 1168 raft_consensus.cc:385] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.403245 1168 raft_consensus.cc:740] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.403380 1165 ts_tablet_manager.cc:1950] T d762b4afb5ff447dac3468393826205a P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:36.403313 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_f89cac674abe45038710fc4b675afbc8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:36.403469 1168 consensus_queue.cc:260] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.403980 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet d762b4afb5ff447dac3468393826205a (table default.Table_f6f977edac0f45bea7e94de5bfa51ee4 [id=4a74b9a60cbb4198b55cee630b2f4bef]) successfully deleted
I20251211 20:52:36.404028 1168 ts_tablet_manager.cc:1434] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20251211 20:52:36.404166 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f89cac674abe45038710fc4b675afbc8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:36.404277 1168 tablet_bootstrap.cc:492] T a9b2aa21ab124b28aebc33adba0583d2 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:36.405835 1168 tablet_bootstrap.cc:654] T a9b2aa21ab124b28aebc33adba0583d2 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.406389 1165 tablet_replica.cc:333] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:36.406476 754 tablet_service.cc:1552] Processing DeleteTablet for tablet c4e048ca28824329b924845ccf5067cf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.406618 1165 raft_consensus.cc:2243] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.409474 1165 raft_consensus.cc:2272] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.410091 1165 ts_tablet_manager.cc:1916] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:36.419139 1164 ts_tablet_manager.cc:1929] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:36.426694 1165 ts_tablet_manager.cc:1929] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:36.426867 1165 log.cc:1199] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/28ef13be3b84410d9c786c43502a3478
I20251211 20:52:36.428413 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_bc230026e7c34f16ad80ed668889d340"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:36.428966 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc230026e7c34f16ad80ed668889d340 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:36.430313 590 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_690a1a1fd5d848818451819f38fc1470 [id=a24d8c58b9d84cdea2b278ae9c6e0ba5] alter complete (version 6)
W20251211 20:52:36.431670 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1c4b2c22bd4c456eb77af45576de466f consensus state has no leader
I20251211 20:52:36.433454 1165 ts_tablet_manager.cc:1950] T 28ef13be3b84410d9c786c43502a3478 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:36.434346 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_bcb5c15cfcc2448a9f814e5ff5665f93"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:36.434911 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bcb5c15cfcc2448a9f814e5ff5665f93 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:36.439425 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 28ef13be3b84410d9c786c43502a3478 (table default.Table_88dcd36b4447481487bc918a05511862 [id=d9453cea23a8430c98a3317856b3cec3]) successfully deleted
I20251211 20:52:36.440018 1165 tablet_replica.cc:333] T c4e048ca28824329b924845ccf5067cf P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:36.440286 1165 raft_consensus.cc:2243] T c4e048ca28824329b924845ccf5067cf P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.440555 1165 raft_consensus.cc:2272] T c4e048ca28824329b924845ccf5067cf P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.443501 1168 tablet_bootstrap.cc:492] T a9b2aa21ab124b28aebc33adba0583d2 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:36.443683 1168 ts_tablet_manager.cc:1403] T a9b2aa21ab124b28aebc33adba0583d2 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.039s user 0.003s sys 0.000s
I20251211 20:52:36.444454 1168 raft_consensus.cc:359] T a9b2aa21ab124b28aebc33adba0583d2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.444633 1168 raft_consensus.cc:385] T a9b2aa21ab124b28aebc33adba0583d2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.444638 589 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 22a4205be75e4a6aae2feac3ea1667a7
I20251211 20:52:36.444685 1168 raft_consensus.cc:740] T a9b2aa21ab124b28aebc33adba0583d2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.444909 1168 consensus_queue.cc:260] T a9b2aa21ab124b28aebc33adba0583d2 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.445134 2023 raft_consensus.cc:493] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.445232 2023 raft_consensus.cc:515] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.445806 2023 leader_election.cc:290] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:36.446475 1168 ts_tablet_manager.cc:1434] T a9b2aa21ab124b28aebc33adba0583d2 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251211 20:52:36.446798 1168 tablet_bootstrap.cc:492] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:36.447921 1165 ts_tablet_manager.cc:1916] T c4e048ca28824329b924845ccf5067cf P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:36.448503 1168 tablet_bootstrap.cc:654] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.454679 1165 ts_tablet_manager.cc:1929] T c4e048ca28824329b924845ccf5067cf P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251211 20:52:36.454931 1165 log.cc:1199] T c4e048ca28824329b924845ccf5067cf P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/c4e048ca28824329b924845ccf5067cf
I20251211 20:52:36.455605 1165 ts_tablet_manager.cc:1950] T c4e048ca28824329b924845ccf5067cf P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:36.456393 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet c4e048ca28824329b924845ccf5067cf (table default.Table_f541a176465249d8aaba954bc23bcf01 [id=6b55ad645bec4a7daefc996978420295]) successfully deleted
I20251211 20:52:36.468217 2018 raft_consensus.cc:493] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.468417 1168 tablet_bootstrap.cc:492] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:36.468355 2018 raft_consensus.cc:515] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.468520 1168 ts_tablet_manager.cc:1403] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.000s
I20251211 20:52:36.467501 592 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 5bd801124df84a6c96f4912993f1bdfd (table default.Table_9aa05c766d874348a9e0eb2c016058d3 [id=44f2f2bf53af4b038ced294f20db9e48]). Expected version 1 got 0
I20251211 20:52:36.468959 2018 leader_election.cc:290] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:36.469012 592 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 3eac21a07ba34e1fa275e25ee9dcabb1 (table default.Table_c6c5666f7b3044589afd38743a1a200c [id=9700f9fe6e3044748e34954bef8a0038]). Expected version 3 got 2
I20251211 20:52:36.469182 592 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b072b759d1254859b0b8d401e3078792 (table default.Table_b56de1e85a1b42ea81308593884dadb8 [id=a137f0ddf52145a99c62444602ab3fdb]). Expected version 4 got 0
I20251211 20:52:36.469264 1168 raft_consensus.cc:359] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.469417 1168 raft_consensus.cc:385] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.469475 1168 raft_consensus.cc:740] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.469681 1168 consensus_queue.cc:260] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.469938 592 catalog_manager.cc:5654] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:36.470218 1168 ts_tablet_manager.cc:1434] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:36.470476 592 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 8feee01fc0aa4b32be926ff2b617b894 (table default.Table_39446be1bf754b5da6d932892c7280cf [id=18f8d722faaa4293a112525ff0c96216]). Expected version 3 got 0
I20251211 20:52:36.470847 592 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 1843dc4548134f29969d46fbc760f5d7 (table default.Table_99917cb4c70840e19004ad381746d77e [id=6abb187bfd214e91b2e11267065a2841]). Expected version 1 got 0
I20251211 20:52:36.471276 592 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet c8e17645a26947ab8c01f9b093fa422c (table default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5]). Expected version 2 got 0
I20251211 20:52:36.471966 1168 tablet_bootstrap.cc:492] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:36.471922 592 catalog_manager.cc:5654] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:36.474530 1168 tablet_bootstrap.cc:654] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.483355 754 tablet_service.cc:1552] Processing DeleteTablet for tablet fb2d30147cd74d1893783be1e1694d1f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.483639 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 97ada0641165444a998db26796793507 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.483803 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 33addf3d158941098f240cfcdec45d28 with delete_type TABLET_DATA_DELETED (replaced by tablet 7be44df39a8f4d6da793479d425fa684) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.484861 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 773cf53745a64f62817113b33e3182e7 with delete_type TABLET_DATA_DELETED (replaced by tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.484925 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 648f43e95e034f2188a20e1b837f2818 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.485093 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 68be21a25c6743f5b86d518d98fbc35e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.485111 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 60d966c0397c4a4ea3718445e87e9703 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.485219 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 3cd7a465249246c28f07b8fb2e42675f with delete_type TABLET_DATA_DELETED (replaced by tablet 0763a475527546bb83cee8fb823893ac) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.485379 1165 tablet_replica.cc:333] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:36.485553 1165 raft_consensus.cc:2243] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.485726 1165 raft_consensus.cc:2272] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.486748 1165 ts_tablet_manager.cc:1916] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:36.490691 1168 tablet_bootstrap.cc:492] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:36.490909 1168 ts_tablet_manager.cc:1403] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.003s
I20251211 20:52:36.491858 754 tablet_service.cc:1552] Processing DeleteTablet for tablet a9b2aa21ab124b28aebc33adba0583d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.491786 1168 raft_consensus.cc:359] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.492004 1168 raft_consensus.cc:385] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.492121 1168 raft_consensus.cc:740] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.492367 1168 consensus_queue.cc:260] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.492890 1168 ts_tablet_manager.cc:1434] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:36.493216 1168 tablet_bootstrap.cc:492] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:36.493537 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 0f046ea2637e4a5583b7969e19a605d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.494753 1168 tablet_bootstrap.cc:654] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.498081 589 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 1843dc4548134f29969d46fbc760f5d7 (table default.Table_99917cb4c70840e19004ad381746d77e [id=6abb187bfd214e91b2e11267065a2841]). Expected version 1 got 0
I20251211 20:52:36.500362 1165 ts_tablet_manager.cc:1929] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:36.500485 1165 log.cc:1199] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/fb2d30147cd74d1893783be1e1694d1f
I20251211 20:52:36.500797 1165 ts_tablet_manager.cc:1950] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:36.500877 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 171d5b17dfee49159925b476b08787a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.504964 592 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 1a77e3e744ff42ccb68897704b4c2f1d (table default.Table_d70f80161dcd4063815a0e079c5821de [id=85ab3a18cda143e5a7f72e49e6671318]). Expected version 1 got 0
I20251211 20:52:36.512151 1165 tablet_replica.cc:333] T 97ada0641165444a998db26796793507 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:36.512369 1165 raft_consensus.cc:2243] T 97ada0641165444a998db26796793507 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.512334 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet fb2d30147cd74d1893783be1e1694d1f (table default.Table_59fd4cd22d6b4065bc70dc11849fa628 [id=c5b5110e2f1f4eb29d233b6a9eef4737]) successfully deleted
I20251211 20:52:36.512518 1165 raft_consensus.cc:2272] T 97ada0641165444a998db26796793507 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.514264 1165 ts_tablet_manager.cc:1916] T 97ada0641165444a998db26796793507 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:36.517279 2000 raft_consensus.cc:493] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.517439 2000 raft_consensus.cc:515] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.518076 1012 tablet_service.cc:1505] Processing CreateTablet for tablet d1ad0dc52d4548e0a9a7b92eae0e2c0e (DEFAULT_TABLE table=default.Table_76916c86042b49f487d21bf7aabfed56 [id=365cb9cb979d45dbaaff790819c90412]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.518571 1586 raft_consensus.cc:493] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.518693 1586 raft_consensus.cc:515] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:36.520738 982 leader_election.cc:343] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.521901 1013 tablet_service.cc:1505] Processing CreateTablet for tablet a98a2b091d9b44a7b954ddb45c819da9 (DEFAULT_TABLE table=default.Table_2fddb279b6164c2ba9e81fc55dfeb59a [id=f8c4f76e6a2d48fb93603b9d308a71ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.524411 1163 raft_consensus.cc:2243] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.525236 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_3542335cc1af47b3817527896d07f03a" } modify_external_catalogs: true
I20251211 20:52:36.525671 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_3542335cc1af47b3817527896d07f03a" } modify_external_catalogs: true
I20251211 20:52:36.526684 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_274e759f68cb487da862431c34237c5c" } modify_external_catalogs: true
I20251211 20:52:36.526961 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_274e759f68cb487da862431c34237c5c" } modify_external_catalogs: true
I20251211 20:52:36.425606 1160 tablet_bootstrap.cc:492] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:36.528687 1164 log.cc:1199] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/605172f066d242f2abea133c08a145ca
I20251211 20:52:36.529031 1164 ts_tablet_manager.cc:1950] T 605172f066d242f2abea133c08a145ca P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:36.530301 1160 tablet_bootstrap.cc:654] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.531446 1165 ts_tablet_manager.cc:1929] T 97ada0641165444a998db26796793507 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:36.487974 1813 raft_consensus.cc:493] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.494935 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f8aa28c1c9b4fc58dfc128952c180ba" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:36.499852 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb2d30147cd74d1893783be1e1694d1f" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:36.505963 888 tablet_service.cc:1552] Processing DeleteTablet for tablet f7f52d2547534ef99255c5d3f59410c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:36.536427 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 605172f066d242f2abea133c08a145ca (table default.Table_40b5190e58a440e9a13d6c9221572914 [id=e2b03132f53845138e4fd9731c2d79bf]) successfully deleted
I20251211 20:52:36.537113 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_faa83de4f1eb4c8fb577bcaaa63687d7" } modify_external_catalogs: true
I20251211 20:52:36.537317 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_faa83de4f1eb4c8fb577bcaaa63687d7" } modify_external_catalogs: true
I20251211 20:52:36.538081 1164 tablet_replica.cc:333] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:36.538259 1164 raft_consensus.cc:2243] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.538405 1164 raft_consensus.cc:2272] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.538740 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_7f5df390c11943918af61066a7c67c66" } modify_external_catalogs: true
I20251211 20:52:36.538936 1164 ts_tablet_manager.cc:1916] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:36.538918 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_7f5df390c11943918af61066a7c67c66" } modify_external_catalogs: true
W20251211 20:52:36.535358 711 consensus_peers.cc:597] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:36.539994 711 leader_election.cc:343] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:36.540184 711 leader_election.cc:343] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.531567 1165 log.cc:1199] T 97ada0641165444a998db26796793507 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/97ada0641165444a998db26796793507
I20251211 20:52:36.535478 1813 raft_consensus.cc:515] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.541039 1813 leader_election.cc:290] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:36.542268 847 leader_election.cc:343] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 8139acb0ae324d0f973ce877edc1cd04
I20251211 20:52:36.542338 2018 raft_consensus.cc:493] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.542477 2018 raft_consensus.cc:515] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.543370 2018 leader_election.cc:290] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:36.543597 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8e17645a26947ab8c01f9b093fa422c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:36.545338 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 2f7b1bc3dd1f4496b3ac6e71652cf012 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f7b1bc3dd1f4496b3ac6e71652cf012 already in progress: creating tablet
W20251211 20:52:36.545377 711 leader_election.cc:343] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.547672 2018 raft_consensus.cc:493] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.547890 1165 ts_tablet_manager.cc:1950] T 97ada0641165444a998db26796793507 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:36.547814 2018 raft_consensus.cc:515] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.548396 2018 leader_election.cc:290] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:36.548749 1165 tablet_replica.cc:333] T 33addf3d158941098f240cfcdec45d28 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:36.548959 1165 raft_consensus.cc:2243] T 33addf3d158941098f240cfcdec45d28 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.549038 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 97ada0641165444a998db26796793507 (table default.Table_a66a87006b684bbd87431e2aad969dcf [id=021234dec4df4bcb9ec0fee4cb1f887e]) successfully deleted
I20251211 20:52:36.549114 1165 raft_consensus.cc:2272] T 33addf3d158941098f240cfcdec45d28 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.549361 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3cd7a465249246c28f07b8fb2e42675f" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:36.550096 711 leader_election.cc:343] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.550679 1165 ts_tablet_manager.cc:1916] T 33addf3d158941098f240cfcdec45d28 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:36.551033 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_02295a0d7c9c43bab5f8b6db77d27dd1" } new_table_name: "default.Table_97ad73b83395434cbb3c2ca0b25d5c1d" modify_external_catalogs: true
W20251211 20:52:36.551025 711 consensus_peers.cc:597] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:36.553251 1168 tablet_bootstrap.cc:492] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:36.553416 1168 ts_tablet_manager.cc:1403] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.060s user 0.005s sys 0.008s
I20251211 20:52:36.554845 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 3562b5effa124378beac7ccf0d27ee7e
I20251211 20:52:36.555447 1168 raft_consensus.cc:359] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.555693 1168 raft_consensus.cc:385] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.555785 1168 raft_consensus.cc:740] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.555994 1168 consensus_queue.cc:260] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.556589 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet bbe3529c301a4a65b58b634a443e1fb6
I20251211 20:52:36.557049 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 3562b5effa124378beac7ccf0d27ee7e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.557687 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 07e4336171714da8adc21977ce33ce50
I20251211 20:52:36.558657 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 07e4336171714da8adc21977ce33ce50 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.561180 754 tablet_service.cc:1552] Processing DeleteTablet for tablet bbe3529c301a4a65b58b634a443e1fb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.562546 1168 ts_tablet_manager.cc:1434] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20251211 20:52:36.562894 1168 tablet_bootstrap.cc:492] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:36.564127 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 87863daf77304a399318cdf56da9f9bb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:36.565673 888 tablet_service.cc:1552] Processing DeleteTablet for tablet bbe3529c301a4a65b58b634a443e1fb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:36.569164 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_6b8536dae8ce48be97d778489d1da99c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:36.569788 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6b8536dae8ce48be97d778489d1da99c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:36.570642 1168 tablet_bootstrap.cc:654] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.571828 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b36aed5743884aa7bca0cb01b2af1bac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:36.572180 1165 ts_tablet_manager.cc:1929] T 33addf3d158941098f240cfcdec45d28 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20251211 20:52:36.572443 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b36aed5743884aa7bca0cb01b2af1bac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:36.573961 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_3f8e5876c38a4c5aac82efa6e26de84d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:36.574616 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3f8e5876c38a4c5aac82efa6e26de84d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:36.576046 1165 log.cc:1199] T 33addf3d158941098f240cfcdec45d28 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/33addf3d158941098f240cfcdec45d28
I20251211 20:52:36.576511 1165 ts_tablet_manager.cc:1950] T 33addf3d158941098f240cfcdec45d28 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:36.579001 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 33addf3d158941098f240cfcdec45d28 (table default.Table_f6a10556ff104db5bd5ecc9fc380e228 [id=f75813add54e46189ff1bbed82d83ba7]) successfully deleted
I20251211 20:52:36.579463 1165 tablet_replica.cc:333] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:36.581044 1165 raft_consensus.cc:2243] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.581362 1165 raft_consensus.cc:2272] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.582434 1165 ts_tablet_manager.cc:1916] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:36.583089 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f26ba8f4bd3a4a1f8fa2ba6e80d543f8 consensus state has no leader
I20251211 20:52:36.587532 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 07e4336171714da8adc21977ce33ce50 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:36.598637 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_97ad73b83395434cbb3c2ca0b25d5c1d" } new_table_name: "default.Table_71e9c158dc22435a8fbb02aecca8919d" modify_external_catalogs: true
I20251211 20:52:36.608793 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 3562b5effa124378beac7ccf0d27ee7e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:36.609275 1164 ts_tablet_manager.cc:1929] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:36.610342 1164 log.cc:1199] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/c85624ad640041c29bb87416e1992a9e
I20251211 20:52:36.615288 1165 ts_tablet_manager.cc:1929] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:36.616321 1165 log.cc:1199] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/773cf53745a64f62817113b33e3182e7
I20251211 20:52:36.616783 1165 ts_tablet_manager.cc:1950] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:36.617647 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 773cf53745a64f62817113b33e3182e7 (table default.Table_021d4fb1667a475d87424f23aa0d5e28 [id=6bbfcc2561f349519a69ae1f56016cc7]) successfully deleted
I20251211 20:52:36.616190 1168 tablet_bootstrap.cc:492] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:36.618135 1168 ts_tablet_manager.cc:1403] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.055s user 0.001s sys 0.009s
I20251211 20:52:36.619030 1168 raft_consensus.cc:359] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.619459 1165 tablet_replica.cc:333] T 648f43e95e034f2188a20e1b837f2818 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:36.619850 1165 raft_consensus.cc:2243] T 648f43e95e034f2188a20e1b837f2818 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.620062 1165 raft_consensus.cc:2272] T 648f43e95e034f2188a20e1b837f2818 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.620936 1165 ts_tablet_manager.cc:1916] T 648f43e95e034f2188a20e1b837f2818 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:36.621315 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f26ba8f4bd3a4a1f8fa2ba6e80d543f8 consensus state has no leader
I20251211 20:52:36.621057 1168 raft_consensus.cc:385] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.621528 1168 raft_consensus.cc:740] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.621879 1168 consensus_queue.cc:260] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.622452 1168 ts_tablet_manager.cc:1434] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20251211 20:52:36.622584 1164 ts_tablet_manager.cc:1950] T c85624ad640041c29bb87416e1992a9e P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:36.624811 1168 tablet_bootstrap.cc:492] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:36.625182 1028 tablet_service.cc:1505] Processing CreateTablet for tablet 80c566822fa94ada983aa7266df4bf9d (DEFAULT_TABLE table=default.Table_6b058d9ed449475e986f56f95463634a [id=330579216bb4452093f90d19c05281f2]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:36.625897 982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.1:42177, user_credentials={real_user=slave}} blocked reactor thread for 299246us
I20251211 20:52:36.626648 1024 tablet_service.cc:1505] Processing CreateTablet for tablet 0350405f1d5844d2a987697fd0d8850c (DEFAULT_TABLE table=default.Table_c1cf2780cc3941688af6d55d57d9e6a8 [id=9aa1c4d2d8a34aa5a6170b5c8e502581]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.627135 1026 tablet_service.cc:1505] Processing CreateTablet for tablet cb3937bc6d6a49928af633d10bdd7074 (DEFAULT_TABLE table=default.Table_e2163380d2474e9a93f10e69de094195 [id=041f6b72446c42f68aa2fcaa8438ad0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.627357 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet e8db5a11a69e4554bc689c38ab57b28a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:36.627499 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 0b267c7feaac43f6aacb7229759c7ec3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:36.628880 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c74427cb9d6f4a679e43eae796ff8c3f" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:36.628347 1168 tablet_bootstrap.cc:654] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:36.629536 983 leader_election.cc:343] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.629590 1016 tablet_service.cc:1552] Processing DeleteTablet for tablet 7474a08c51cf4346ae9a518963d56b7a with delete_type TABLET_DATA_DELETED (replaced by tablet e95c63b4d99a4dd0b2c0fa384ca9d552) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:36.629740 1014 tablet_service.cc:1552] Processing DeleteTablet for tablet c4e048ca28824329b924845ccf5067cf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:36.630050 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 2ef524747034491c99f4f9767acd3c9d (DEFAULT_TABLE table=default.Table_3f5d0d2b042c42ed88431232445bef02 [id=19e89fd1209c4cad8ef39476f607f3ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.630267 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 76cfb26762d54593a150ec98125c6592 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:36.630426 1164 tablet_replica.cc:333] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:36.632529 1012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1ad0dc52d4548e0a9a7b92eae0e2c0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.632346 1164 raft_consensus.cc:2243] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.634205 1164 raft_consensus.cc:2272] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.636319 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 9b99860ee6be45a0b358e686332d3137 (DEFAULT_TABLE table=default.Table_a5b944a25f4c45cf8c70765ea43d1ac2 [id=0ee0170e570a4d9aa25ea2858e9d9e2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.637322 1165 ts_tablet_manager.cc:1929] T 648f43e95e034f2188a20e1b837f2818 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:36.637470 1165 log.cc:1199] T 648f43e95e034f2188a20e1b837f2818 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/648f43e95e034f2188a20e1b837f2818
I20251211 20:52:36.637604 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80c566822fa94ada983aa7266df4bf9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.637784 1165 ts_tablet_manager.cc:1950] T 648f43e95e034f2188a20e1b837f2818 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:36.638774 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 648f43e95e034f2188a20e1b837f2818 (table default.Table_fc80da3fac43460dabeea7e43bf098df [id=75a5a167f3f84710afbf197ea1eff3cc]) successfully deleted
I20251211 20:52:36.640003 1164 ts_tablet_manager.cc:1916] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:36.640733 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb3937bc6d6a49928af633d10bdd7074. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.643836 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 652828db3e31467fad8391e6cfe70846 (DEFAULT_TABLE table=default.Table_20456517a58b40fabd534461d2ab6cc0 [id=f93e5d8158d74f7cb3b77ddefe065c8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.649173 983 leader_election.cc:304] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.651234 1168 tablet_bootstrap.cc:492] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:36.651409 1168 ts_tablet_manager.cc:1403] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.004s
I20251211 20:52:36.652076 2000 leader_election.cc:290] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:36.652261 1586 leader_election.cc:290] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:36.652266 1168 raft_consensus.cc:359] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.652436 1168 raft_consensus.cc:385] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.652526 1168 raft_consensus.cc:740] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.652730 1168 consensus_queue.cc:260] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.639266 2062 raft_consensus.cc:493] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.653115 2018 raft_consensus.cc:493] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.653256 2018 raft_consensus.cc:515] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.653499 1168 ts_tablet_manager.cc:1434] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251211 20:52:36.653697 1165 tablet_replica.cc:333] T 68be21a25c6743f5b86d518d98fbc35e P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:36.653857 1165 raft_consensus.cc:2243] T 68be21a25c6743f5b86d518d98fbc35e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.653903 2018 leader_election.cc:290] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:36.654022 1165 raft_consensus.cc:2272] T 68be21a25c6743f5b86d518d98fbc35e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.654089 1586 raft_consensus.cc:2749] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:36.654403 1168 tablet_bootstrap.cc:492] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:36.654562 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "87863daf77304a399318cdf56da9f9bb" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:36.654771 1165 ts_tablet_manager.cc:1916] T 68be21a25c6743f5b86d518d98fbc35e P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:36.655164 711 leader_election.cc:343] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.656342 1168 tablet_bootstrap.cc:654] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.660352 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet c85624ad640041c29bb87416e1992a9e (table default.Table_8f7e347b9ac64793b1073dfa955ebe21 [id=a6ab64c3e0d94462b716f2d153bc3314]) successfully deleted
I20251211 20:52:36.660580 1165 ts_tablet_manager.cc:1929] T 68be21a25c6743f5b86d518d98fbc35e P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20251211 20:52:36.669549 983 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 137906us
I20251211 20:52:36.669999 1165 log.cc:1199] T 68be21a25c6743f5b86d518d98fbc35e P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/68be21a25c6743f5b86d518d98fbc35e
I20251211 20:52:36.670279 2000 raft_consensus.cc:493] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.670356 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c74427cb9d6f4a679e43eae796ff8c3f" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:36.670397 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b267c7feaac43f6aacb7229759c7ec3" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:36.671206 1165 ts_tablet_manager.cc:1950] T 68be21a25c6743f5b86d518d98fbc35e P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:36.672909 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 68be21a25c6743f5b86d518d98fbc35e (table default.Table_afd3c9ee35ad41caa1717a299443409e [id=b0ebaab8cd5a4deb9a5ea1e7bd3ccec3]) successfully deleted
I20251211 20:52:36.674075 1165 tablet_replica.cc:333] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:36.674561 1165 raft_consensus.cc:2243] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.675065 1165 raft_consensus.cc:2272] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.672854 2062 raft_consensus.cc:515] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.677268 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b99860ee6be45a0b358e686332d3137. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.678061 1165 ts_tablet_manager.cc:1916] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:36.678210 1014 tablet_service.cc:1552] Processing DeleteTablet for tablet bbe3529c301a4a65b58b634a443e1fb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:36.679225 1016 tablet_service.cc:1552] Processing DeleteTablet for tablet 07e4336171714da8adc21977ce33ce50 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:36.680402 710 leader_election.cc:343] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 822abda3c85848728205b71847ef7029
W20251211 20:52:36.680516 847 leader_election.cc:343] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 8139acb0ae324d0f973ce877edc1cd04
I20251211 20:52:36.680578 710 leader_election.cc:304] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.680639 847 leader_election.cc:304] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
W20251211 20:52:36.680755 710 raft_consensus.cc:2686] T 822abda3c85848728205b71847ef7029 P 6f8d784fd1cd492bbe3809403ec34919: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251211 20:52:36.681282 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b267c7feaac43f6aacb7229759c7ec3" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:36.681420 710 leader_election.cc:343] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 773cf53745a64f62817113b33e3182e7
I20251211 20:52:36.681542 710 leader_election.cc:304] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:36.681711 710 raft_consensus.cc:2686] T 773cf53745a64f62817113b33e3182e7 P 6f8d784fd1cd492bbe3809403ec34919: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251211 20:52:36.681850 1586 raft_consensus.cc:493] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.682101 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0350405f1d5844d2a987697fd0d8850c. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:36.682236 710 leader_election.cc:343] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: fb2d30147cd74d1893783be1e1694d1f
I20251211 20:52:36.682350 710 leader_election.cc:304] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:36.682490 710 raft_consensus.cc:2686] T fb2d30147cd74d1893783be1e1694d1f P 6f8d784fd1cd492bbe3809403ec34919: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251211 20:52:36.683213 1813 raft_consensus.cc:2749] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:36.692260 2019 raft_consensus.cc:493] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.692550 2019 raft_consensus.cc:515] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.692620 1586 raft_consensus.cc:515] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:36.692385 982 leader_election.cc:343] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.693732 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0f046ea2637e4a5583b7969e19a605d5" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:36.693919 2019 leader_election.cc:290] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:36.694316 711 leader_election.cc:343] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.670719 1168 tablet_bootstrap.cc:492] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:36.696121 2000 raft_consensus.cc:515] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.698944 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5bd801124df84a6c96f4912993f1bdfd" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:36.699148 1047 raft_consensus.cc:2468] T 5bd801124df84a6c96f4912993f1bdfd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:36.698091 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 3562b5effa124378beac7ccf0d27ee7e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:36.701192 2000 leader_election.cc:290] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:36.701094 1165 ts_tablet_manager.cc:1929] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:36.702057 1165 log.cc:1199] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/60d966c0397c4a4ea3718445e87e9703
I20251211 20:52:36.698366 1168 ts_tablet_manager.cc:1403] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.044s user 0.003s sys 0.001s
I20251211 20:52:36.704186 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f8aa28c1c9b4fc58dfc128952c180ba" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:36.704746 1164 ts_tablet_manager.cc:1929] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:36.704823 1165 ts_tablet_manager.cc:1950] T 60d966c0397c4a4ea3718445e87e9703 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:36.705538 1046 raft_consensus.cc:2468] T 9f8aa28c1c9b4fc58dfc128952c180ba P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:36.705605 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8e17645a26947ab8c01f9b093fa422c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:36.705925 1164 log.cc:1199] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/6a2442a4ebc940a281f9ea1a394f6621
W20251211 20:52:36.705483 1044 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.696s user 0.055s sys 0.129s
I20251211 20:52:36.709117 1045 raft_consensus.cc:1275] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Refusing update from remote peer 6f8d784fd1cd492bbe3809403ec34919: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251211 20:52:36.709074 1164 ts_tablet_manager.cc:1950] T 6a2442a4ebc940a281f9ea1a394f6621 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:36.709425 1049 raft_consensus.cc:2468] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:36.704150 1041 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "87863daf77304a399318cdf56da9f9bb" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:36.709710 1041 raft_consensus.cc:2468] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:36.710227 1168 raft_consensus.cc:359] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.711469 1168 raft_consensus.cc:385] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.711993 1168 raft_consensus.cc:740] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.713840 1164 tablet_replica.cc:333] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:36.714002 1168 consensus_queue.cc:260] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.715021 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ef524747034491c99f4f9767acd3c9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.716039 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 6a2442a4ebc940a281f9ea1a394f6621 (table default.Table_cbe68d84efef4745822f6aef5a723f11 [id=f9c1230a7d71435cac705d4675a9908d]) successfully deleted
I20251211 20:52:36.716311 1168 ts_tablet_manager.cc:1434] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20251211 20:52:36.716598 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 652828db3e31467fad8391e6cfe70846. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.716686 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 60d966c0397c4a4ea3718445e87e9703 (table default.Table_f51ac924827c4dbea700109489db0963 [id=cae3f6884694487e82fc81328e9527d1]) successfully deleted
I20251211 20:52:36.715950 1164 raft_consensus.cc:2243] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
W20251211 20:52:36.717908 982 leader_election.cc:343] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.718060 1586 leader_election.cc:290] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:36.718870 2062 leader_election.cc:290] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:36.717950 1042 raft_consensus.cc:1275] T 40722aef4e194ff094d271a079bda9d9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Refusing update from remote peer 6f8d784fd1cd492bbe3809403ec34919: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251211 20:52:36.723699 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4201da4241aa48dfb670a713e4453ca5" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:36.723801 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ccc3ddd4b7f74f11a3835997cb7ae8d0" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:36.723673 710 leader_election.cc:343] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 3cd7a465249246c28f07b8fb2e42675f
I20251211 20:52:36.723711 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2d849c62f7434099a16f62c2a2fc5e85" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:36.724218 710 leader_election.cc:304] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.724651 1168 tablet_bootstrap.cc:492] T e28d66ce4711430cb09a9650e99fbe9e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:36.724709 710 leader_election.cc:304] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.725095 710 leader_election.cc:304] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.725446 2018 raft_consensus.cc:2749] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:36.725648 2018 raft_consensus.cc:2804] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:36.725724 2018 raft_consensus.cc:493] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:36.725829 2018 raft_consensus.cc:3060] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:36.726286 1168 tablet_bootstrap.cc:654] T e28d66ce4711430cb09a9650e99fbe9e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.728013 1696 raft_consensus.cc:2804] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:36.728103 1696 raft_consensus.cc:493] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:36.728195 1696 raft_consensus.cc:3060] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:36.732378 1164 raft_consensus.cc:2272] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.734824 1696 raft_consensus.cc:515] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.735488 1696 leader_election.cc:290] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:36.735901 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f8aa28c1c9b4fc58dfc128952c180ba" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:36.736335 1165 tablet_replica.cc:333] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
W20251211 20:52:36.736454 711 leader_election.cc:343] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.737174 1165 raft_consensus.cc:2243] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.738569 1164 ts_tablet_manager.cc:1916] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:36.723274 2000 raft_consensus.cc:493] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251211 20:52:36.723280 1044 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.697s user 0.055s sys 0.129s
I20251211 20:52:36.740962 2000 raft_consensus.cc:515] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.741565 2000 leader_election.cc:290] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:36.743041 1165 raft_consensus.cc:2272] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.743250 710 leader_election.cc:304] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.744019 710 leader_election.cc:304] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.744402 1165 ts_tablet_manager.cc:1916] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:36.744441 2023 raft_consensus.cc:2804] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:36.746140 2023 raft_consensus.cc:493] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:36.746205 2023 raft_consensus.cc:3060] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:36.748749 2023 raft_consensus.cc:515] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.749938 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8e17645a26947ab8c01f9b093fa422c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:36.750520 711 leader_election.cc:343] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.750847 2023 leader_election.cc:290] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:36.744846 2019 raft_consensus.cc:2804] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:36.751173 2019 raft_consensus.cc:493] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:36.751262 2019 raft_consensus.cc:3060] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:36.754081 2019 raft_consensus.cc:515] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.755136 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "87863daf77304a399318cdf56da9f9bb" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:36.755681 711 leader_election.cc:343] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.756090 2019 leader_election.cc:290] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:36.743453 1696 consensus_queue.cc:1048] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:36.756142 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f8aa28c1c9b4fc58dfc128952c180ba" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:36.756304 1049 raft_consensus.cc:3060] T 9f8aa28c1c9b4fc58dfc128952c180ba P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:36.745868 1168 tablet_bootstrap.cc:492] T e28d66ce4711430cb09a9650e99fbe9e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:36.758648 1168 ts_tablet_manager.cc:1403] T e28d66ce4711430cb09a9650e99fbe9e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.001s
I20251211 20:52:36.758857 1049 raft_consensus.cc:2468] T 9f8aa28c1c9b4fc58dfc128952c180ba P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:36.759287 1041 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0f046ea2637e4a5583b7969e19a605d5" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:36.761834 1168 raft_consensus.cc:359] T e28d66ce4711430cb09a9650e99fbe9e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.763885 1168 raft_consensus.cc:385] T e28d66ce4711430cb09a9650e99fbe9e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.764158 1168 raft_consensus.cc:740] T e28d66ce4711430cb09a9650e99fbe9e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.764668 1168 consensus_queue.cc:260] T e28d66ce4711430cb09a9650e99fbe9e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.766112 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a98a2b091d9b44a7b954ddb45c819da9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.765021 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "92a71dff4ea8445d8484fc2747471148" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:36.765164 1696 consensus_queue.cc:1048] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:36.765529 710 leader_election.cc:304] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.766939 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8e17645a26947ab8c01f9b093fa422c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:36.767093 1049 raft_consensus.cc:3060] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:36.767397 1696 raft_consensus.cc:2804] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:36.767560 1696 raft_consensus.cc:697] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:36.767850 1696 consensus_queue.cc:237] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.770026 1163 raft_consensus.cc:2272] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:36.771891 982 leader_election.cc:343] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:36.772275 982 leader_election.cc:343] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.772302 1168 ts_tablet_manager.cc:1434] T e28d66ce4711430cb09a9650e99fbe9e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
W20251211 20:52:36.772500 982 leader_election.cc:343] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:36.774443 982 leader_election.cc:343] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.774891 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "87863daf77304a399318cdf56da9f9bb" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:36.775070 1044 raft_consensus.cc:3060] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:36.777087 1168 tablet_bootstrap.cc:492] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:36.778522 1168 tablet_bootstrap.cc:654] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.786433 1049 raft_consensus.cc:2468] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:36.788996 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ccc3ddd4b7f74f11a3835997cb7ae8d0" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:36.789829 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2d849c62f7434099a16f62c2a2fc5e85" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:36.790606 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4201da4241aa48dfb670a713e4453ca5" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:36.791992 1164 ts_tablet_manager.cc:1929] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:36.792572 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "92a71dff4ea8445d8484fc2747471148" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:36.792826 983 leader_election.cc:343] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.792935 983 leader_election.cc:304] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.798599 754 tablet_service.cc:1505] Processing CreateTablet for tablet 0bbb0b010aae44a3a43abdf910e08b77 (DEFAULT_TABLE table=default.Table_ef6d1803b4bf4d62860e0bbdf745446a [id=b11896be27d848958e5b3fab8f5504a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.799079 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bbb0b010aae44a3a43abdf910e08b77. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.800531 747 tablet_service.cc:1505] Processing CreateTablet for tablet 586f9d1d24ad40cea44a1704f85b2296 (DEFAULT_TABLE table=default.Table_11d5dfcc3a7d4db59119bf9c9666c50a [id=011d18ce4d764af8b3e399443dc217e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.800882 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 586f9d1d24ad40cea44a1704f85b2296. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.800889 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_d4afd6ed615a41ac8f6714c8aac27857" } new_table_name: "default.Table_0cc01b2a751d4591b9b227f46093e552" modify_external_catalogs: true
I20251211 20:52:36.802258 595 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_6412ddf2aa9540e29cf2e6aa2ca8ae47" } new_table_name: "default.Table_5db1ac98e36d45b1b01889ee275ad83d" modify_external_catalogs: true
I20251211 20:52:36.803433 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_89da60cc38f54ac48f12f3b5988d48ad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:36.803552 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_7e3cf73eaea24e34b5dd291f8a738947"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:36.803978 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_89da60cc38f54ac48f12f3b5988d48ad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:36.804051 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e3cf73eaea24e34b5dd291f8a738947 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:36.804050 1164 log.cc:1199] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/d5f0ba3ab16240f9822be69f941f5da2
I20251211 20:52:36.805220 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_1c43c2cd6c064780936260e292759cf1" } new_table_name: "default.Table_dfb3a2820f32459e98c153c4566d3812" modify_external_catalogs: true
I20251211 20:52:36.805406 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_86e88e19fbd645d3bd5a919328a0969c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:36.805840 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_86e88e19fbd645d3bd5a919328a0969c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:36.806427 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b8ccf690baa544a8a6f41b0c6eea6b38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:36.807008 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b8ccf690baa544a8a6f41b0c6eea6b38 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:36.806938 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_f5a4d53f860a4ab48ac66679cd67c7eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:36.806733 1164 ts_tablet_manager.cc:1950] T d5f0ba3ab16240f9822be69f941f5da2 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
W20251211 20:52:36.807359 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f5a4d53f860a4ab48ac66679cd67c7eb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:36.810870 589 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 1843dc4548134f29969d46fbc760f5d7 (table default.Table_99917cb4c70840e19004ad381746d77e [id=6abb187bfd214e91b2e11267065a2841]). Expected version 1 got 0
I20251211 20:52:36.811916 589 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3eac21a07ba34e1fa275e25ee9dcabb1 (table default.Table_c6c5666f7b3044589afd38743a1a200c [id=9700f9fe6e3044748e34954bef8a0038]). Expected version 3 got 2
I20251211 20:52:36.812146 589 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 2d849c62f7434099a16f62c2a2fc5e85 (table default.Table_c504f8f3796f43b697488bd604268543 [id=2af76b9d32094a4ca656060a4478e08a]). Expected version 1 got 0
I20251211 20:52:36.812090 883 tablet_service.cc:1505] Processing CreateTablet for tablet 586f9d1d24ad40cea44a1704f85b2296 (DEFAULT_TABLE table=default.Table_11d5dfcc3a7d4db59119bf9c9666c50a [id=011d18ce4d764af8b3e399443dc217e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.812512 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 586f9d1d24ad40cea44a1704f85b2296. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.814924 1164 tablet_replica.cc:333] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:36.816332 1164 raft_consensus.cc:2243] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.817354 1164 raft_consensus.cc:2272] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.819288 2019 raft_consensus.cc:493] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.818918 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet d5f0ba3ab16240f9822be69f941f5da2 (table default.Table_d73d156809724a399ad27846bfcc460b [id=baf6ca043ec048b894cf9716fad105b2]) successfully deleted
I20251211 20:52:36.819511 2019 raft_consensus.cc:515] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.820689 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b072b759d1254859b0b8d401e3078792" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:36.821295 711 leader_election.cc:343] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.821806 2019 leader_election.cc:290] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:36.822731 1164 ts_tablet_manager.cc:1916] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:36.835781 1168 tablet_bootstrap.cc:492] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:36.836020 1168 ts_tablet_manager.cc:1403] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.059s user 0.003s sys 0.000s
I20251211 20:52:36.836861 1168 raft_consensus.cc:359] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.837090 1168 raft_consensus.cc:385] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.837185 1168 raft_consensus.cc:740] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.837478 1165 ts_tablet_manager.cc:1929] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:36.837426 1168 consensus_queue.cc:260] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.841046 1165 log.cc:1199] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/3cd7a465249246c28f07b8fb2e42675f
I20251211 20:52:36.845556 1163 ts_tablet_manager.cc:1916] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:36.846060 1165 ts_tablet_manager.cc:1950] T 3cd7a465249246c28f07b8fb2e42675f P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:36.841427 1168 ts_tablet_manager.cc:1434] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20251211 20:52:36.847481 1168 tablet_bootstrap.cc:492] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:36.847520 710 leader_election.cc:343] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.847664 710 leader_election.cc:304] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.848265 710 leader_election.cc:304] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.848979 1168 tablet_bootstrap.cc:654] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.849442 2019 raft_consensus.cc:2749] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:36.849814 2019 raft_consensus.cc:2804] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
W20251211 20:52:36.850440 983 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 57620.9us
W20251211 20:52:36.850718 983 leader_election.cc:343] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.850838 983 leader_election.cc:304] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.850533 2019 raft_consensus.cc:697] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:36.851449 2019 consensus_queue.cc:237] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.860669 1019 tablet_service.cc:1552] Processing DeleteTablet for tablet c74427cb9d6f4a679e43eae796ff8c3f with delete_type TABLET_DATA_DELETED (replaced by tablet 4e123b96b7534f07ae48a808f47e21a1) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:36.861068 1012 tablet_service.cc:1552] Processing DeleteTablet for tablet bb112b6431af42a6a7b328dfdefbc2a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:36.866852 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet fb030a62796248869d46b682656eaeb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:36.870272 1044 raft_consensus.cc:2468] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:36.872488 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b072b759d1254859b0b8d401e3078792" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:36.872709 1044 raft_consensus.cc:2468] T b072b759d1254859b0b8d401e3078792 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:36.873853 2062 raft_consensus.cc:2749] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:36.874174 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 0bbb0b010aae44a3a43abdf910e08b77 (DEFAULT_TABLE table=default.Table_ef6d1803b4bf4d62860e0bbdf745446a [id=b11896be27d848958e5b3fab8f5504a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.874598 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bbb0b010aae44a3a43abdf910e08b77. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.876957 1165 tablet_replica.cc:333] T a9b2aa21ab124b28aebc33adba0583d2 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:36.877354 1165 raft_consensus.cc:2243] T a9b2aa21ab124b28aebc33adba0583d2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.884187 1165 raft_consensus.cc:2272] T a9b2aa21ab124b28aebc33adba0583d2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.884855 1164 ts_tablet_manager.cc:1929] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:36.885607 1164 log.cc:1199] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/8139acb0ae324d0f973ce877edc1cd04
I20251211 20:52:36.887156 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 3cd7a465249246c28f07b8fb2e42675f (table default.Table_227fbf051d5643528315ca00194583bc [id=1515561ab78d46f6b5aa37a20d90d3f5]) successfully deleted
I20251211 20:52:36.887403 1164 ts_tablet_manager.cc:1950] T 8139acb0ae324d0f973ce877edc1cd04 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:36.888761 591 catalog_manager.cc:5654] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:36.890400 1165 ts_tablet_manager.cc:1916] T a9b2aa21ab124b28aebc33adba0583d2 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:36.891728 591 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 9f8aa28c1c9b4fc58dfc128952c180ba (table default.Table_bf7d402a36be44cfb6b79041906d26a1 [id=0327e686e7ed4475a3e8cda8721a3a3a]). Expected version 1 got 0
I20251211 20:52:36.891865 2071 tablet.cc:1722] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:36.901183 592 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 7452dd736b0440fcb4626858e794676f
I20251211 20:52:36.901723 1163 ts_tablet_manager.cc:1929] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:36.901842 1163 log.cc:1199] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/0f046ea2637e4a5583b7969e19a605d5
I20251211 20:52:36.902100 1163 ts_tablet_manager.cc:1950] T 0f046ea2637e4a5583b7969e19a605d5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:36.902391 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 716) took 1042 ms. Trace:
I20251211 20:52:36.902470 1163 rpcz_store.cc:276] 1211 20:52:35.859908 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:35.973057 (+113149us) service_pool.cc:225] Handling call
1211 20:52:36.211480 (+238423us) ts_tablet_manager.cc:1151] Deleting tablet 0f046ea2637e4a5583b7969e19a605d5
1211 20:52:36.211710 (+ 230us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:36.888556 (+676846us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:36.901616 (+ 13060us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:36.902376 (+ 760us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":238249}
I20251211 20:52:36.902667 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet 7452dd736b0440fcb4626858e794676f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:36.902832 1163 tablet_replica.cc:333] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:36.902992 1163 raft_consensus.cc:2243] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.903143 1163 raft_consensus.cc:2272] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.903206 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 0f046ea2637e4a5583b7969e19a605d5 (table default.Table_dffb2ba445c0475e96e343895b21cb51 [id=e96fd7db496847a5b1e0b79546a336d0]) successfully deleted
I20251211 20:52:36.905776 1163 ts_tablet_manager.cc:1916] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:36.906903 889 tablet_service.cc:1505] Processing CreateTablet for tablet 3003ef71a3554c71a3b24cdfb9c293bc (DEFAULT_TABLE table=default.Table_c676a5b83a30404b9e0e9b189701845a [id=6e7fb3271e0c48c0bb9d56fe4fccee3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.907325 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3003ef71a3554c71a3b24cdfb9c293bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.914642 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 8139acb0ae324d0f973ce877edc1cd04 (table default.Table_d282d0b8f49e4f44998af55480ad3cb8 [id=959101c0af144f52a5680b0c9d4cbd20]) successfully deleted
W20251211 20:52:36.915844 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 0b267c7feaac43f6aacb7229759c7ec3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0b267c7feaac43f6aacb7229759c7ec3 already in progress: creating tablet
W20251211 20:52:36.922662 595 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ce1799e6604f4ed7875ddedecab6f327 consensus state has no leader
I20251211 20:52:36.922454 1164 tablet_replica.cc:333] T c4e048ca28824329b924845ccf5067cf P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:36.923699 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a9542664049c4b818c73d1958c2f2030" } modify_external_catalogs: true
I20251211 20:52:36.923867 1164 raft_consensus.cc:2243] T c4e048ca28824329b924845ccf5067cf P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.924067 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a9542664049c4b818c73d1958c2f2030" } modify_external_catalogs: true
I20251211 20:52:36.924876 1164 raft_consensus.cc:2272] T c4e048ca28824329b924845ccf5067cf P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:36.925729 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet e8db5a11a69e4554bc689c38ab57b28a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e8db5a11a69e4554bc689c38ab57b28a already in progress: creating tablet
W20251211 20:52:36.926083 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 76cfb26762d54593a150ec98125c6592 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 76cfb26762d54593a150ec98125c6592 already in progress: creating tablet
W20251211 20:52:36.926667 1166 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.726s user 0.064s sys 0.172s
W20251211 20:52:36.926796 1166 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.735s user 0.064s sys 0.172s
I20251211 20:52:36.926837 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 586f9d1d24ad40cea44a1704f85b2296 (DEFAULT_TABLE table=default.Table_11d5dfcc3a7d4db59119bf9c9666c50a [id=011d18ce4d764af8b3e399443dc217e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.927271 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 586f9d1d24ad40cea44a1704f85b2296. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.929095 1166 tablet_bootstrap.cc:492] T 3b2c6064c5eb4c0f8979c485f5d85a75 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:36.929239 1166 ts_tablet_manager.cc:1403] T 3b2c6064c5eb4c0f8979c485f5d85a75 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.748s user 0.066s sys 0.172s
I20251211 20:52:36.930022 1166 raft_consensus.cc:359] T 3b2c6064c5eb4c0f8979c485f5d85a75 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.930183 1166 raft_consensus.cc:385] T 3b2c6064c5eb4c0f8979c485f5d85a75 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.930229 1166 raft_consensus.cc:740] T 3b2c6064c5eb4c0f8979c485f5d85a75 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.930423 1166 consensus_queue.cc:260] T 3b2c6064c5eb4c0f8979c485f5d85a75 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.930879 1166 ts_tablet_manager.cc:1434] T 3b2c6064c5eb4c0f8979c485f5d85a75 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:36.931149 1166 tablet_bootstrap.cc:492] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:36.934401 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 3003ef71a3554c71a3b24cdfb9c293bc (DEFAULT_TABLE table=default.Table_c676a5b83a30404b9e0e9b189701845a [id=6e7fb3271e0c48c0bb9d56fe4fccee3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.934808 1166 tablet_bootstrap.cc:654] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:36.935115 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3003ef71a3554c71a3b24cdfb9c293bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.935488 2000 raft_consensus.cc:493] T 72e23f3033e14c9db6de8539830bac3b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:36.935631 2000 raft_consensus.cc:515] T 72e23f3033e14c9db6de8539830bac3b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.936347 1163 ts_tablet_manager.cc:1929] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:36.936563 1163 log.cc:1199] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/6a526a6d1bc84cefad2c5d897223292e
I20251211 20:52:36.938652 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_37849576518b4320a3bac34baa096242"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:36.938773 2000 leader_election.cc:290] T 72e23f3033e14c9db6de8539830bac3b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
W20251211 20:52:36.939314 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_37849576518b4320a3bac34baa096242 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:36.939131 1163 ts_tablet_manager.cc:1950] T 6a526a6d1bc84cefad2c5d897223292e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:36.941026 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_4d6139c7976d414fbdcbdb96787d186b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:36.941310 1164 ts_tablet_manager.cc:1916] T c4e048ca28824329b924845ccf5067cf P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:36.941768 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4d6139c7976d414fbdcbdb96787d186b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:36.942834 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 6a526a6d1bc84cefad2c5d897223292e (table default.Table_254d66657cb14d0f8bbf713f794806f6 [id=f96a54c77528480ebf36ecc5741f7e17]) successfully deleted
I20251211 20:52:36.943830 1163 tablet_replica.cc:333] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:36.945300 1163 raft_consensus.cc:2243] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:36.946483 1163 raft_consensus.cc:2272] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:36.953888 1163 ts_tablet_manager.cc:1916] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:36.961328 421 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 122455us
I20251211 20:52:36.962227 1166 tablet_bootstrap.cc:492] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:36.962414 1166 ts_tablet_manager.cc:1403] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.031s user 0.011s sys 0.003s
I20251211 20:52:36.964841 1166 raft_consensus.cc:359] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.968709 889 tablet_service.cc:1505] Processing CreateTablet for tablet bb9fccb6981e446a939a28d7fdd38e25 (DEFAULT_TABLE table=default.Table_760e3e4714ea4128bd1de3da5a7ccccc [id=fe9b692163f943559ba64021faa648f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.969277 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb9fccb6981e446a939a28d7fdd38e25. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.966578 2000 raft_consensus.cc:2749] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:36.971575 1166 raft_consensus.cc:385] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.971724 1166 raft_consensus.cc:740] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.971976 1166 consensus_queue.cc:260] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:36.966890 983 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 116173us
I20251211 20:52:36.969370 2071 tablet.cc:1722] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20251211 20:52:36.972676 1166 ts_tablet_manager.cc:1434] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.010s user 0.001s sys 0.000s
I20251211 20:52:36.973003 1166 tablet_bootstrap.cc:492] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:36.974360 1166 tablet_bootstrap.cc:654] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:36.976683 983 leader_election.cc:343] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.976897 983 leader_election.cc:304] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.977188 2000 raft_consensus.cc:2749] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:36.977649 983 leader_election.cc:343] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.977825 983 leader_election.cc:304] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.978098 2000 raft_consensus.cc:2749] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:36.978498 983 leader_election.cc:343] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.978650 983 leader_election.cc:304] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:36.978849 983 raft_consensus.cc:2686] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7: unable to run election callback: Service unavailable: Thread pool token was shut down
W20251211 20:52:36.979822 983 leader_election.cc:343] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:36.980002 983 leader_election.cc:304] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:36.980216 2000 raft_consensus.cc:2749] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:36.981966 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "72e23f3033e14c9db6de8539830bac3b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:36.982344 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8b66a5217c494c8ebfa10a2b986e5e78" } modify_external_catalogs: true
I20251211 20:52:36.982545 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_8b66a5217c494c8ebfa10a2b986e5e78" } modify_external_catalogs: true
I20251211 20:52:36.983636 1166 tablet_bootstrap.cc:492] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:36.983824 1166 ts_tablet_manager.cc:1403] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.002s
I20251211 20:52:36.987654 1166 raft_consensus.cc:359] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.988191 1166 raft_consensus.cc:385] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:36.988438 1166 raft_consensus.cc:740] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:36.988859 1166 consensus_queue.cc:260] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:36.989815 1166 ts_tablet_manager.cc:1434] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:36.990692 1166 tablet_bootstrap.cc:492] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:36.995040 1027 tablet_service.cc:1505] Processing CreateTablet for tablet bb9fccb6981e446a939a28d7fdd38e25 (DEFAULT_TABLE table=default.Table_760e3e4714ea4128bd1de3da5a7ccccc [id=fe9b692163f943559ba64021faa648f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:36.995474 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb9fccb6981e446a939a28d7fdd38e25. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:36.996981 1166 tablet_bootstrap.cc:654] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.000710 1166 tablet_bootstrap.cc:492] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:37.001331 1166 ts_tablet_manager.cc:1403] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20251211 20:52:37.002281 1166 raft_consensus.cc:359] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.006799 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 7452dd736b0440fcb4626858e794676f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:37.008241 1166 raft_consensus.cc:385] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.008219 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_aa83784f17a84101a53b3e111f31d93e" } modify_external_catalogs: true
I20251211 20:52:37.008455 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_aa83784f17a84101a53b3e111f31d93e" } modify_external_catalogs: true
I20251211 20:52:37.008558 1166 raft_consensus.cc:740] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.008992 1166 consensus_queue.cc:260] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.010016 1166 ts_tablet_manager.cc:1434] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20251211 20:52:37.010752 1166 tablet_bootstrap.cc:492] T f737a7bfb0f44196abdc831bb22d5285 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:37.012707 1166 tablet_bootstrap.cc:654] T f737a7bfb0f44196abdc831bb22d5285 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:37.015530 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8675e2a18f0f4a0e95a87005cf82d8b4 has no consensus state
I20251211 20:52:37.020833 883 tablet_service.cc:1505] Processing CreateTablet for tablet 4b55b82d5fc94aabb50d46a8d3a66351 (DEFAULT_TABLE table=default.Table_434ed373bf494a40b55b1f275d648d28 [id=1224a8665eaf45c1a4ab549024bb4200]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.021497 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b55b82d5fc94aabb50d46a8d3a66351. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.024348 1160 tablet_bootstrap.cc:492] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:37.024730 1160 ts_tablet_manager.cc:1403] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.599s user 0.003s sys 0.010s
I20251211 20:52:37.036187 2081 tablet.cc:1722] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20251211 20:52:37.036994 2083 tablet.cc:1722] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:37.033799 1160 raft_consensus.cc:359] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.039968 1160 raft_consensus.cc:385] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.040174 1160 raft_consensus.cc:740] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.042766 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 4b55b82d5fc94aabb50d46a8d3a66351 (DEFAULT_TABLE table=default.Table_434ed373bf494a40b55b1f275d648d28 [id=1224a8665eaf45c1a4ab549024bb4200]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.045037 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b55b82d5fc94aabb50d46a8d3a66351. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.049998 1160 consensus_queue.cc:260] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.055445 596 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 40722aef4e194ff094d271a079bda9d9 (table default.Table_12f41f708f1949598d6176d89777aa69 [id=3309205faabf4a42acadc43766e4f57f]). Expected version 1 got 0
I20251211 20:52:37.055936 2018 raft_consensus.cc:515] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:37.056788 983 consensus_peers.cc:597] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:37.057107 983 leader_election.cc:343] T 72e23f3033e14c9db6de8539830bac3b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.057135 596 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 8feee01fc0aa4b32be926ff2b617b894 (table default.Table_39446be1bf754b5da6d932892c7280cf [id=18f8d722faaa4293a112525ff0c96216]). Expected version 3 got 0
I20251211 20:52:37.057730 2062 raft_consensus.cc:493] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:37.057853 2062 raft_consensus.cc:515] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.058393 2062 leader_election.cc:290] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:37.061811 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b529b38d754f46ba8cb60b19e9705256" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:37.067656 1160 ts_tablet_manager.cc:1434] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.043s user 0.002s sys 0.001s
I20251211 20:52:37.075345 1163 ts_tablet_manager.cc:1929] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:37.075697 1163 log.cc:1199] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/4201da4241aa48dfb670a713e4453ca5
W20251211 20:52:37.076282 983 leader_election.cc:343] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.077939 1164 ts_tablet_manager.cc:1929] T c4e048ca28824329b924845ccf5067cf P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251211 20:52:37.078114 1164 log.cc:1199] T c4e048ca28824329b924845ccf5067cf P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/c4e048ca28824329b924845ccf5067cf
I20251211 20:52:37.078513 1164 ts_tablet_manager.cc:1950] T c4e048ca28824329b924845ccf5067cf P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:37.080487 1160 tablet_bootstrap.cc:492] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:37.081138 961 heartbeater.cc:344] Connected to a master server at 127.0.104.61:40749
I20251211 20:52:37.082046 1160 tablet_bootstrap.cc:654] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.086408 1166 tablet_bootstrap.cc:492] T f737a7bfb0f44196abdc831bb22d5285 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:37.087411 1166 ts_tablet_manager.cc:1403] T f737a7bfb0f44196abdc831bb22d5285 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.077s user 0.012s sys 0.012s
I20251211 20:52:37.089641 1166 raft_consensus.cc:359] T f737a7bfb0f44196abdc831bb22d5285 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.090288 1166 raft_consensus.cc:385] T f737a7bfb0f44196abdc831bb22d5285 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.090528 1166 raft_consensus.cc:740] T f737a7bfb0f44196abdc831bb22d5285 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.091099 1166 consensus_queue.cc:260] T f737a7bfb0f44196abdc831bb22d5285 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.092197 1166 ts_tablet_manager.cc:1434] T f737a7bfb0f44196abdc831bb22d5285 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:37.092975 1166 tablet_bootstrap.cc:492] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:37.095074 1166 tablet_bootstrap.cc:654] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.106001 1166 tablet_bootstrap.cc:492] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:37.107365 1166 ts_tablet_manager.cc:1403] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.002s
I20251211 20:52:37.108224 1164 tablet_replica.cc:333] T 7474a08c51cf4346ae9a518963d56b7a P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:37.108481 1164 raft_consensus.cc:2243] T 7474a08c51cf4346ae9a518963d56b7a P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:37.108469 1166 raft_consensus.cc:359] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.108621 888 tablet_service.cc:1505] Processing CreateTablet for tablet 4e123b96b7534f07ae48a808f47e21a1 (DEFAULT_TABLE table=default.Table_31d573e7af144e9997dfe480f8466902 [id=f689abc175c94366bfda8217545156aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.108721 1164 raft_consensus.cc:2272] T 7474a08c51cf4346ae9a518963d56b7a P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:37.108996 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e123b96b7534f07ae48a808f47e21a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.109254 1164 ts_tablet_manager.cc:1916] T 7474a08c51cf4346ae9a518963d56b7a P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:37.109735 1166 raft_consensus.cc:385] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.109799 1166 raft_consensus.cc:740] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.110242 1166 consensus_queue.cc:260] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.112012 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet c4e048ca28824329b924845ccf5067cf (table default.Table_f541a176465249d8aaba954bc23bcf01 [id=6b55ad645bec4a7daefc996978420295]) successfully deleted
I20251211 20:52:37.113014 1166 ts_tablet_manager.cc:1434] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251211 20:52:37.113407 1166 tablet_bootstrap.cc:492] T 8b793241ee3b42b1a38e9f2f05e1173b P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:37.115917 1163 ts_tablet_manager.cc:1950] T 4201da4241aa48dfb670a713e4453ca5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:37.116783 1166 tablet_bootstrap.cc:654] T 8b793241ee3b42b1a38e9f2f05e1173b P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.122249 1160 tablet_bootstrap.cc:492] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:37.123034 1160 ts_tablet_manager.cc:1403] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.043s user 0.003s sys 0.000s
I20251211 20:52:37.124965 2008 raft_consensus.cc:493] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:37.125583 2008 raft_consensus.cc:515] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.126431 2008 leader_election.cc:290] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:37.128405 2018 leader_election.cc:290] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:37.129855 847 leader_election.cc:343] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 340730a17a834a77bbc20b95aba98e2c
I20251211 20:52:37.130800 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 729) took 1010 ms. Trace:
I20251211 20:52:37.132413 1160 raft_consensus.cc:359] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.132934 1160 raft_consensus.cc:385] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.133083 1160 raft_consensus.cc:740] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.133545 1160 consensus_queue.cc:260] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.130993 1163 rpcz_store.cc:276] 1211 20:52:36.120666 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:36.120713 (+ 47us) service_pool.cc:225] Handling call
1211 20:52:36.943370 (+822657us) ts_tablet_manager.cc:1151] Deleting tablet 4201da4241aa48dfb670a713e4453ca5
1211 20:52:36.943383 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:37.030610 (+ 87227us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:37.075165 (+ 44555us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:37.130773 (+ 55608us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":822215}
I20251211 20:52:37.147315 1163 tablet_replica.cc:333] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:37.147269 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 4201da4241aa48dfb670a713e4453ca5 (table default.Table_4000fbd0bb614d0c902f539be2923421 [id=a8f3e7213c3040a3bb34e8827a1029bf]) successfully deleted
I20251211 20:52:37.147835 1163 raft_consensus.cc:2243] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:37.148213 1163 raft_consensus.cc:2272] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:37.149294 1163 ts_tablet_manager.cc:1916] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:37.146710 1166 tablet_bootstrap.cc:492] T 8b793241ee3b42b1a38e9f2f05e1173b P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:37.157498 1168 tablet_bootstrap.cc:492] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:37.157653 1168 ts_tablet_manager.cc:1403] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.310s user 0.039s sys 0.074s
I20251211 20:52:37.157944 1165 ts_tablet_manager.cc:1929] T a9b2aa21ab124b28aebc33adba0583d2 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:37.158042 1165 log.cc:1199] T a9b2aa21ab124b28aebc33adba0583d2 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/a9b2aa21ab124b28aebc33adba0583d2
I20251211 20:52:37.158299 1165 ts_tablet_manager.cc:1950] T a9b2aa21ab124b28aebc33adba0583d2 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:37.159312 2073 tablet.cc:1722] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:37.161314 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 4e123b96b7534f07ae48a808f47e21a1 (DEFAULT_TABLE table=default.Table_31d573e7af144e9997dfe480f8466902 [id=f689abc175c94366bfda8217545156aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.161643 1165 tablet_replica.cc:333] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:37.161743 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e123b96b7534f07ae48a808f47e21a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.157500 1166 ts_tablet_manager.cc:1403] T 8b793241ee3b42b1a38e9f2f05e1173b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.044s user 0.006s sys 0.010s
I20251211 20:52:37.163489 1166 raft_consensus.cc:359] T 8b793241ee3b42b1a38e9f2f05e1173b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.163820 1166 raft_consensus.cc:385] T 8b793241ee3b42b1a38e9f2f05e1173b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.163925 1166 raft_consensus.cc:740] T 8b793241ee3b42b1a38e9f2f05e1173b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.164212 1166 consensus_queue.cc:260] T 8b793241ee3b42b1a38e9f2f05e1173b P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.164741 1166 ts_tablet_manager.cc:1434] T 8b793241ee3b42b1a38e9f2f05e1173b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251211 20:52:37.165037 1166 tablet_bootstrap.cc:492] T 4f807596fa6a4e66b89566b5d5dd1080 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:37.166836 1166 tablet_bootstrap.cc:654] T 4f807596fa6a4e66b89566b5d5dd1080 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.170004 888 tablet_service.cc:1505] Processing CreateTablet for tablet da81bf8854a24954ba4548185812295d (DEFAULT_TABLE table=default.Table_18262e22c26d4d66ba4c6134fbde0368 [id=da8cf2b99b42409091484a9513623d6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.170492 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da81bf8854a24954ba4548185812295d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.168262 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "72e23f3033e14c9db6de8539830bac3b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:37.179200 982 consensus_peers.cc:597] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 -> Peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Couldn't send request to peer 6f8d784fd1cd492bbe3809403ec34919. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4e048ca28824329b924845ccf5067cf. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:37.179754 982 leader_election.cc:343] T 72e23f3033e14c9db6de8539830bac3b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.179890 982 leader_election.cc:304] T 72e23f3033e14c9db6de8539830bac3b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.180308 2000 raft_consensus.cc:2749] T 72e23f3033e14c9db6de8539830bac3b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:37.178218 1160 ts_tablet_manager.cc:1434] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.050s user 0.002s sys 0.000s
I20251211 20:52:37.181439 1165 raft_consensus.cc:2243] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:37.181645 1165 raft_consensus.cc:2272] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:37.183720 1165 ts_tablet_manager.cc:1916] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:37.184887 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b529b38d754f46ba8cb60b19e9705256" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:37.185401 982 leader_election.cc:343] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.185527 982 leader_election.cc:304] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.185901 2000 raft_consensus.cc:2749] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:37.187192 1168 raft_consensus.cc:359] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.187378 1168 raft_consensus.cc:385] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.187424 1168 raft_consensus.cc:740] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.191011 1160 tablet_bootstrap.cc:492] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:37.191675 710 leader_election.cc:304] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.192052 2092 raft_consensus.cc:2804] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:37.192173 2092 raft_consensus.cc:697] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:37.192422 2092 consensus_queue.cc:237] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.193964 1168 consensus_queue.cc:260] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.195185 2086 raft_consensus.cc:493] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:37.195302 2086 raft_consensus.cc:515] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.195844 2086 leader_election.cc:290] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:37.197206 1160 tablet_bootstrap.cc:654] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.199046 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5bd801124df84a6c96f4912993f1bdfd" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:37.199862 1168 ts_tablet_manager.cc:1434] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.042s user 0.000s sys 0.002s
I20251211 20:52:37.200176 1168 tablet_bootstrap.cc:492] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:37.200205 1164 ts_tablet_manager.cc:1929] T 7474a08c51cf4346ae9a518963d56b7a P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:37.201400 1164 log.cc:1199] T 7474a08c51cf4346ae9a518963d56b7a P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/7474a08c51cf4346ae9a518963d56b7a
I20251211 20:52:37.204864 2086 raft_consensus.cc:493] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:37.205015 2086 raft_consensus.cc:515] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:37.206182 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_32ebd46bd3cc460986098c79c9b67ca7" } modify_external_catalogs: true
I20251211 20:52:37.206403 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_32ebd46bd3cc460986098c79c9b67ca7" } modify_external_catalogs: true
I20251211 20:52:37.223124 1168 tablet_bootstrap.cc:654] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.224078 2086 leader_election.cc:290] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:37.225778 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "171d5b17dfee49159925b476b08787a4" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:37.225926 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 7452dd736b0440fcb4626858e794676f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:37.226315 742 tablet_service.cc:1505] Processing CreateTablet for tablet 4b55b82d5fc94aabb50d46a8d3a66351 (DEFAULT_TABLE table=default.Table_434ed373bf494a40b55b1f275d648d28 [id=1224a8665eaf45c1a4ab549024bb4200]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.226477 754 tablet_service.cc:1505] Processing CreateTablet for tablet bb9fccb6981e446a939a28d7fdd38e25 (DEFAULT_TABLE table=default.Table_760e3e4714ea4128bd1de3da5a7ccccc [id=fe9b692163f943559ba64021faa648f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.226493 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e4c970184c9c4300b80e462a20527759" } new_table_name: "default.Table_083c478bedec45a5bf5203e4ed333860" modify_external_catalogs: true
I20251211 20:52:37.226696 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b55b82d5fc94aabb50d46a8d3a66351. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.226315 747 tablet_service.cc:1505] Processing CreateTablet for tablet 3003ef71a3554c71a3b24cdfb9c293bc (DEFAULT_TABLE table=default.Table_c676a5b83a30404b9e0e9b189701845a [id=6e7fb3271e0c48c0bb9d56fe4fccee3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.227084 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3003ef71a3554c71a3b24cdfb9c293bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.227703 757 tablet_service.cc:1505] Processing CreateTablet for tablet 4e123b96b7534f07ae48a808f47e21a1 (DEFAULT_TABLE table=default.Table_31d573e7af144e9997dfe480f8466902 [id=f689abc175c94366bfda8217545156aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.227875 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb9fccb6981e446a939a28d7fdd38e25. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.228057 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e123b96b7534f07ae48a808f47e21a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.228525 2073 tablet.cc:1722] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20251211 20:52:37.224969 710 leader_election.cc:304] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.229979 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet a9b2aa21ab124b28aebc33adba0583d2 (table default.Table_b70392ad5fb6451482100395256494bd [id=eafa1350ab384c8cba766017840d59a6]) successfully deleted
I20251211 20:52:37.230026 751 tablet_service.cc:1505] Processing CreateTablet for tablet da81bf8854a24954ba4548185812295d (DEFAULT_TABLE table=default.Table_18262e22c26d4d66ba4c6134fbde0368 [id=da8cf2b99b42409091484a9513623d6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.230227 2077 raft_consensus.cc:2804] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:37.230296 2077 raft_consensus.cc:493] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:37.230340 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da81bf8854a24954ba4548185812295d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.230353 2077 raft_consensus.cc:3060] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:37.238519 1027 tablet_service.cc:1505] Processing CreateTablet for tablet da81bf8854a24954ba4548185812295d (DEFAULT_TABLE table=default.Table_18262e22c26d4d66ba4c6134fbde0368 [id=da8cf2b99b42409091484a9513623d6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.240970 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b94ece9030c84d37a4ed8d28c2a12a91"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:37.241266 2086 raft_consensus.cc:493] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:37.241405 2086 raft_consensus.cc:515] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:37.241611 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b94ece9030c84d37a4ed8d28c2a12a91 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:37.241986 2086 leader_election.cc:290] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:37.244766 888 tablet_service.cc:1505] Processing CreateTablet for tablet 84bb3583857848459e0e865283de623d (DEFAULT_TABLE table=default.Table_742d698db63f432aa7b88ceccf6dc0fc [id=6e5fe0a78e4b48f98dac028802f1ddf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.245162 740 tablet_service.cc:1552] Processing DeleteTablet for tablet e28d66ce4711430cb09a9650e99fbe9e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:37.245227 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84bb3583857848459e0e865283de623d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.246021 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a526a6d1bc84cefad2c5d897223292e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:37.248152 1813 raft_consensus.cc:493] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:37.248281 1813 raft_consensus.cc:515] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.248693 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6c64fd8dd49b4e5d91c1ffb0fc2559e0" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:37.248849 1813 leader_election.cc:290] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:37.249097 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3fc173c396ff4af5a8589cff18165e85" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:37.249897 757 tablet_service.cc:1505] Processing CreateTablet for tablet 84bb3583857848459e0e865283de623d (DEFAULT_TABLE table=default.Table_742d698db63f432aa7b88ceccf6dc0fc [id=6e5fe0a78e4b48f98dac028802f1ddf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.250279 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84bb3583857848459e0e865283de623d. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:37.251201 847 leader_election.cc:343] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: f7f52d2547534ef99255c5d3f59410c8
I20251211 20:52:37.251688 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da81bf8854a24954ba4548185812295d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.251816 1165 ts_tablet_manager.cc:1929] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:37.251929 1165 log.cc:1199] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/0f046ea2637e4a5583b7969e19a605d5
I20251211 20:52:37.253216 2073 tablet.cc:1722] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:37.253298 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 84bb3583857848459e0e865283de623d (DEFAULT_TABLE table=default.Table_742d698db63f432aa7b88ceccf6dc0fc [id=6e5fe0a78e4b48f98dac028802f1ddf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.253881 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84bb3583857848459e0e865283de623d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.254129 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5bd801124df84a6c96f4912993f1bdfd" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:37.254316 1048 raft_consensus.cc:3060] T 5bd801124df84a6c96f4912993f1bdfd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:37.254856 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6c64fd8dd49b4e5d91c1ffb0fc2559e0" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:37.255249 1043 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3fc173c396ff4af5a8589cff18165e85" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:37.255440 1043 raft_consensus.cc:2468] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:37.255641 1696 raft_consensus.cc:493] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:37.255751 1696 raft_consensus.cc:515] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.256131 2092 raft_consensus.cc:493] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:37.256250 1696 leader_election.cc:290] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:37.256232 2092 raft_consensus.cc:515] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.257020 1048 raft_consensus.cc:2468] T 5bd801124df84a6c96f4912993f1bdfd P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:37.257275 2071 tablet.cc:1722] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:37.262948 2092 leader_election.cc:290] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:37.206223 1164 ts_tablet_manager.cc:1950] T 7474a08c51cf4346ae9a518963d56b7a P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
W20251211 20:52:37.262420 711 leader_election.cc:343] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.268399 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 6c64fd8dd49b4e5d91c1ffb0fc2559e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:37.269351 711 leader_election.cc:343] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.269528 2000 raft_consensus.cc:493] T 3b2c6064c5eb4c0f8979c485f5d85a75 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:37.269657 2000 raft_consensus.cc:515] T 3b2c6064c5eb4c0f8979c485f5d85a75 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.269984 2073 tablet.cc:1722] T 40722aef4e194ff094d271a079bda9d9 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20251211 20:52:37.270035 711 leader_election.cc:343] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:37.270186 711 leader_election.cc:343] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.270414 2000 leader_election.cc:290] T 3b2c6064c5eb4c0f8979c485f5d85a75 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:37.271517 2071 tablet.cc:1722] T 40722aef4e194ff094d271a079bda9d9 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:37.273156 2086 consensus_queue.cc:1048] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251211 20:52:37.270741 847 leader_election.cc:343] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 340730a17a834a77bbc20b95aba98e2c
I20251211 20:52:37.276230 847 leader_election.cc:304] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:37.276639 1813 raft_consensus.cc:2749] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:37.270979 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b2c6064c5eb4c0f8979c485f5d85a75" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:37.271334 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a77e3e744ff42ccb68897704b4c2f1d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:37.272765 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8229b704e7464b30b7d2902a938ce33b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:37.277575 983 leader_election.cc:343] T 3b2c6064c5eb4c0f8979c485f5d85a75 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.273119 2081 tablet.cc:1722] T dd425601d89148eb95525f796a3abf1e P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:37.278728 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 7474a08c51cf4346ae9a518963d56b7a (table default.Table_d65f65d608a346968cc2f35b9f55912e [id=88e8b1b4524846e18f5a638dee69a688]) successfully deleted
I20251211 20:52:37.279906 1164 tablet_replica.cc:333] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:37.284340 1164 raft_consensus.cc:2243] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:37.285198 1164 raft_consensus.cc:2272] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:37.286428 1165 ts_tablet_manager.cc:1950] T 0f046ea2637e4a5583b7969e19a605d5 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:37.286880 1165 tablet_replica.cc:333] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:37.287050 1165 raft_consensus.cc:2243] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
W20251211 20:52:37.287117 711 leader_election.cc:343] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.287129 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 0f046ea2637e4a5583b7969e19a605d5 (table default.Table_dffb2ba445c0475e96e343895b21cb51 [id=e96fd7db496847a5b1e0b79546a336d0]) successfully deleted
I20251211 20:52:37.287305 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b2c6064c5eb4c0f8979c485f5d85a75" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:37.287415 1165 raft_consensus.cc:2272] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:37.287389 847 leader_election.cc:343] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: f7f52d2547534ef99255c5d3f59410c8
I20251211 20:52:37.287503 847 leader_election.cc:304] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
W20251211 20:52:37.287707 847 raft_consensus.cc:2686] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41: unable to run election callback: Service unavailable: Thread pool token was shut down
W20251211 20:52:37.287778 982 leader_election.cc:343] T 3b2c6064c5eb4c0f8979c485f5d85a75 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.287883 982 leader_election.cc:304] T 3b2c6064c5eb4c0f8979c485f5d85a75 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.287969 1165 ts_tablet_manager.cc:1916] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:37.288239 2000 raft_consensus.cc:2749] T 3b2c6064c5eb4c0f8979c485f5d85a75 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:37.288492 711 leader_election.cc:343] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.292647 2073 tablet.cc:1722] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:37.292899 1164 ts_tablet_manager.cc:1916] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:37.295569 2087 raft_consensus.cc:493] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:37.302713 2087 raft_consensus.cc:515] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.303647 1165 ts_tablet_manager.cc:1929] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:37.303768 1165 log.cc:1199] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/171d5b17dfee49159925b476b08787a4
I20251211 20:52:37.306355 2086 raft_consensus.cc:493] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:37.306488 2086 raft_consensus.cc:515] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.308861 1160 tablet_bootstrap.cc:492] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:37.312378 2086 leader_election.cc:290] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:37.312978 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6a526a6d1bc84cefad2c5d897223292e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:37.313540 711 leader_election.cc:343] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.316777 1163 ts_tablet_manager.cc:1929] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:37.317003 1160 ts_tablet_manager.cc:1403] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.126s user 0.004s sys 0.003s
I20251211 20:52:37.322005 1160 raft_consensus.cc:359] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.325074 1165 ts_tablet_manager.cc:1950] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:37.325897 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 171d5b17dfee49159925b476b08787a4 (table default.Table_75c7933bd82449a1a24a7481815ee23b [id=4f5722552e534a00874e4113c0dfbaf3]) successfully deleted
I20251211 20:52:37.325968 1165 tablet_replica.cc:333] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:37.326133 1165 raft_consensus.cc:2243] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
W20251211 20:52:37.326238 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 3562b5effa124378beac7ccf0d27ee7e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3562b5effa124378beac7ccf0d27ee7e already in progress: creating tablet
I20251211 20:52:37.326359 1165 pending_rounds.cc:70] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919: Trying to abort 1 pending ops.
I20251211 20:52:37.326398 1165 pending_rounds.cc:77] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432119064600576 op_type: NO_OP noop_request { }
I20251211 20:52:37.326485 1165 raft_consensus.cc:2889] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:37.326552 1165 raft_consensus.cc:2272] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:37.326817 2077 raft_consensus.cc:515] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.327392 2077 leader_election.cc:290] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:37.328104 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b072b759d1254859b0b8d401e3078792" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:37.328641 711 leader_election.cc:343] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:37.332340 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 07e4336171714da8adc21977ce33ce50 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07e4336171714da8adc21977ce33ce50 already in progress: creating tablet
W20251211 20:52:37.332652 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet bbe3529c301a4a65b58b634a443e1fb6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bbe3529c301a4a65b58b634a443e1fb6 already in progress: creating tablet
I20251211 20:52:37.334973 1163 log.cc:1199] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/6c64fd8dd49b4e5d91c1ffb0fc2559e0
I20251211 20:52:37.341337 2087 leader_election.cc:290] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:37.340915 2096 consensus_queue.cc:1175] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6f8d784fd1cd492bbe3809403ec34919" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20251211 20:52:37.342005 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2d849c62f7434099a16f62c2a2fc5e85" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:37.342528 1165 ts_tablet_manager.cc:1916] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:37.342595 711 leader_election.cc:343] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.323854 1160 raft_consensus.cc:385] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.344367 1160 raft_consensus.cc:740] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.345515 1163 ts_tablet_manager.cc:1950] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:37.347334 1904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Bootstrap replayed 1/1 log segments. Stats: ops{read=439 overwritten=0 applied=439 ignored=0} inserts{seen=409 ignored=0} mutations{seen=513 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251211 20:52:37.346680 1160 consensus_queue.cc:260] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.348222 1041 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a77e3e744ff42ccb68897704b4c2f1d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:37.348429 1041 raft_consensus.cc:2468] T 1a77e3e744ff42ccb68897704b4c2f1d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:37.349404 1041 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8229b704e7464b30b7d2902a938ce33b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:37.349670 1041 raft_consensus.cc:2468] T 8229b704e7464b30b7d2902a938ce33b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:37.351900 1041 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b072b759d1254859b0b8d401e3078792" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:37.353955 1813 raft_consensus.cc:493] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:37.354091 1904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Bootstrap complete.
I20251211 20:52:37.354087 1813 raft_consensus.cc:515] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.354576 1041 raft_consensus.cc:3060] T b072b759d1254859b0b8d401e3078792 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:37.354673 1813 leader_election.cc:290] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:37.355510 847 leader_election.cc:343] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 05385f20e88c46108e09a78814811209
W20251211 20:52:37.355720 847 leader_election.cc:343] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 05385f20e88c46108e09a78814811209
I20251211 20:52:37.355806 847 leader_election.cc:304] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:37.356216 1813 raft_consensus.cc:2749] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:37.356313 1168 tablet_bootstrap.cc:492] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:37.356451 1168 ts_tablet_manager.cc:1403] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.156s user 0.000s sys 0.036s
I20251211 20:52:37.357602 1160 ts_tablet_manager.cc:1434] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.040s user 0.003s sys 0.000s
I20251211 20:52:37.361158 1041 raft_consensus.cc:2468] T b072b759d1254859b0b8d401e3078792 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
W20251211 20:52:37.357851 710 leader_election.cc:343] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 171d5b17dfee49159925b476b08787a4
I20251211 20:52:37.363386 1160 tablet_bootstrap.cc:492] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:37.365549 710 leader_election.cc:304] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:37.365753 710 raft_consensus.cc:2686] T 171d5b17dfee49159925b476b08787a4 P 6f8d784fd1cd492bbe3809403ec34919: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251211 20:52:37.367107 1904 raft_consensus.cc:359] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:37.368299 1904 raft_consensus.cc:740] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f5f94732f049d0be4c97cebfffcb05, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.369275 1904 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 439, Last appended: 1.439, Last appended by leader: 439, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:37.370689 883 tablet_service.cc:1505] Processing CreateTablet for tablet 69bfeae3909b45959d591a8bbcb70ce5 (DEFAULT_TABLE table=default.Table_842a98db481041b98480204da25a6419 [id=4fa6a5b9f21e41f4ae0fb953da11a51b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.371160 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 80df719919a44aba9b4e3c62d6830cf3
I20251211 20:52:37.371309 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69bfeae3909b45959d591a8bbcb70ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.375365 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 80df719919a44aba9b4e3c62d6830cf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:37.376241 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet 80df719919a44aba9b4e3c62d6830cf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:37.376451 2107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } } }
I20251211 20:52:37.378762 2107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [sys.catalog]: This master's current role is: FOLLOWER
I20251211 20:52:37.378615 1904 sys_catalog.cc:565] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [sys.catalog]: configured and running, proceeding with master startup.
I20251211 20:52:37.364459 1168 raft_consensus.cc:359] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.389282 1168 raft_consensus.cc:385] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.389390 1168 raft_consensus.cc:740] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.389616 1168 consensus_queue.cc:260] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.390144 1168 ts_tablet_manager.cc:1434] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.034s user 0.000s sys 0.002s
I20251211 20:52:37.390470 1168 tablet_bootstrap.cc:492] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:37.392127 1168 tablet_bootstrap.cc:654] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.379737 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 80df719919a44aba9b4e3c62d6830cf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:37.395435 1160 tablet_bootstrap.cc:654] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.399855 1168 tablet_bootstrap.cc:492] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:37.400046 1168 ts_tablet_manager.cc:1403] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.003s
I20251211 20:52:37.400841 1168 raft_consensus.cc:359] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.401052 1168 raft_consensus.cc:385] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.401130 1168 raft_consensus.cc:740] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.401340 1168 consensus_queue.cc:260] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.401719 589 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 8229b704e7464b30b7d2902a938ce33b (table default.Table_b008891c4e894b02aa0db32d1740371a [id=8fe47ac33eb2461784b859e64fbd3e24]). Expected version 5 got 0
I20251211 20:52:37.401830 1168 ts_tablet_manager.cc:1434] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251211 20:52:37.402091 1168 tablet_bootstrap.cc:492] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:37.401957 589 catalog_manager.cc:5654] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:37.402330 589 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet c8e17645a26947ab8c01f9b093fa422c (table default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5]). Expected version 2 got 0
I20251211 20:52:37.402560 589 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 1843dc4548134f29969d46fbc760f5d7 (table default.Table_99917cb4c70840e19004ad381746d77e [id=6abb187bfd214e91b2e11267065a2841]). Expected version 1 got 0
I20251211 20:52:37.402720 589 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 2d849c62f7434099a16f62c2a2fc5e85 (table default.Table_c504f8f3796f43b697488bd604268543 [id=2af76b9d32094a4ca656060a4478e08a]). Expected version 1 got 0
I20251211 20:52:37.404023 1168 tablet_bootstrap.cc:654] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:37.407493 581 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 645b08b33676420c96e880de3d460829 (table default.Table_e02933dd91ce4fe7a40d1c3b2f9b2d4b [id=6d42f260246e420d8818b3841926302b], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 645b08b33676420c96e880de3d460829 consensus state has no leader
I20251211 20:52:37.416064 757 tablet_service.cc:1505] Processing CreateTablet for tablet 69bfeae3909b45959d591a8bbcb70ce5 (DEFAULT_TABLE table=default.Table_842a98db481041b98480204da25a6419 [id=4fa6a5b9f21e41f4ae0fb953da11a51b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.416482 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69bfeae3909b45959d591a8bbcb70ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.419672 891 tablet_service.cc:1505] Processing CreateTablet for tablet a7afcce522494c9ca1d80e6eea913e6e (DEFAULT_TABLE table=default.Table_cb6a4bac7b694f33a37de2ee78c91ca0 [id=8940f18527c249a9baefc170ed82f45a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.420084 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7afcce522494c9ca1d80e6eea913e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.420885 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 86c5228210954839ae7bf5bf0b5aa027 (DEFAULT_TABLE table=default.Table_79cef56b856548afb009a80ec288ad1b [id=a4bb425aab7d4a9183630f3db61a6310]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.421224 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86c5228210954839ae7bf5bf0b5aa027. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.425335 747 tablet_service.cc:1505] Processing CreateTablet for tablet 86c5228210954839ae7bf5bf0b5aa027 (DEFAULT_TABLE table=default.Table_79cef56b856548afb009a80ec288ad1b [id=a4bb425aab7d4a9183630f3db61a6310]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.425769 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86c5228210954839ae7bf5bf0b5aa027. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.426426 889 tablet_service.cc:1505] Processing CreateTablet for tablet 86c5228210954839ae7bf5bf0b5aa027 (DEFAULT_TABLE table=default.Table_79cef56b856548afb009a80ec288ad1b [id=a4bb425aab7d4a9183630f3db61a6310]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.426853 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86c5228210954839ae7bf5bf0b5aa027. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.428400 742 tablet_service.cc:1505] Processing CreateTablet for tablet 0c6266d18f674b41bc0e1689b6b8cbf6 (DEFAULT_TABLE table=default.Table_3a9e5e76c5324906b05c61141dfb7034 [id=c356cc5067cd4dbb877c660a70aa71e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.428995 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c6266d18f674b41bc0e1689b6b8cbf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.430563 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 0c6266d18f674b41bc0e1689b6b8cbf6 (DEFAULT_TABLE table=default.Table_3a9e5e76c5324906b05c61141dfb7034 [id=c356cc5067cd4dbb877c660a70aa71e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.431052 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c6266d18f674b41bc0e1689b6b8cbf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.434966 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 726) took 1314 ms. Trace:
I20251211 20:52:37.435066 1163 rpcz_store.cc:276] 1211 20:52:36.120356 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:36.120938 (+ 582us) service_pool.cc:225] Handling call
1211 20:52:37.147258 (+1026320us) ts_tablet_manager.cc:1151] Deleting tablet 6c64fd8dd49b4e5d91c1ffb0fc2559e0
1211 20:52:37.147270 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:37.220292 (+ 73022us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:37.316662 (+ 96370us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:37.434943 (+118281us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":68908,"tablet-delete.queue_time_us":1026148}
I20251211 20:52:37.435559 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 6c64fd8dd49b4e5d91c1ffb0fc2559e0 (table default.Table_cc48404bbbf24a7caae7641a8d42c247 [id=048a032339b3462ba676bcb44f4e2082]) successfully deleted
I20251211 20:52:37.435957 1163 tablet_replica.cc:333] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:37.436143 1163 raft_consensus.cc:2243] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:37.436300 1163 raft_consensus.cc:2272] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:37.436854 1163 ts_tablet_manager.cc:1916] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:37.438246 1165 ts_tablet_manager.cc:1929] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:37.439060 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_c3617a747d9d4e4294df0b575520d39f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:37.439760 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c3617a747d9d4e4294df0b575520d39f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:37.440223 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 69bfeae3909b45959d591a8bbcb70ce5 (DEFAULT_TABLE table=default.Table_842a98db481041b98480204da25a6419 [id=4fa6a5b9f21e41f4ae0fb953da11a51b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.440623 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69bfeae3909b45959d591a8bbcb70ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.443470 888 tablet_service.cc:1505] Processing CreateTablet for tablet 0c6266d18f674b41bc0e1689b6b8cbf6 (DEFAULT_TABLE table=default.Table_3a9e5e76c5324906b05c61141dfb7034 [id=c356cc5067cd4dbb877c660a70aa71e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.443919 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c6266d18f674b41bc0e1689b6b8cbf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.400915 1164 ts_tablet_manager.cc:1929] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:37.445585 1164 log.cc:1199] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/f7f52d2547534ef99255c5d3f59410c8
I20251211 20:52:37.445894 1164 ts_tablet_manager.cc:1950] T f7f52d2547534ef99255c5d3f59410c8 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:37.447009 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet f7f52d2547534ef99255c5d3f59410c8 (table default.Table_b3e8496af4d340218ce858c81796a29c [id=ecb6d90c41224c2d8f771a78922eaf05]) successfully deleted
W20251211 20:52:37.450104 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet bbe3529c301a4a65b58b634a443e1fb6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bbe3529c301a4a65b58b634a443e1fb6 already in progress: creating tablet
W20251211 20:52:37.450364 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 07e4336171714da8adc21977ce33ce50 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07e4336171714da8adc21977ce33ce50 already in progress: creating tablet
W20251211 20:52:37.450529 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 3562b5effa124378beac7ccf0d27ee7e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3562b5effa124378beac7ccf0d27ee7e already in progress: creating tablet
W20251211 20:52:37.450693 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 7452dd736b0440fcb4626858e794676f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7452dd736b0440fcb4626858e794676f already in progress: creating tablet
W20251211 20:52:37.450862 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 80df719919a44aba9b4e3c62d6830cf3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 80df719919a44aba9b4e3c62d6830cf3 already in progress: creating tablet
I20251211 20:52:37.452749 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_bcd4dcb9828242a18cc8ca757e2180ba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:37.453404 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bcd4dcb9828242a18cc8ca757e2180ba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:37.460683 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 2dde7f28e61d41f48015ba6e4f3757f2
I20251211 20:52:37.461596 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 2dde7f28e61d41f48015ba6e4f3757f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:37.464565 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 2dde7f28e61d41f48015ba6e4f3757f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:37.467901 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a891ad881584489e87e69014ed59fed9" } new_table_name: "default.Table_c7ed8c0c309c40a3a371ffa130b47cd2" modify_external_catalogs: true
I20251211 20:52:37.468869 1166 tablet_bootstrap.cc:492] T 4f807596fa6a4e66b89566b5d5dd1080 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:37.469012 1166 ts_tablet_manager.cc:1403] T 4f807596fa6a4e66b89566b5d5dd1080 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.304s user 0.027s sys 0.034s
I20251211 20:52:37.475524 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 32f8d104f6a948d1a97a83769ea82449
I20251211 20:52:37.476224 593 master_service.cc:954] ReplaceTablet: received request to replace tablet 098054406a1d459b824a094f50e71e7e from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:37.477329 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet 32f8d104f6a948d1a97a83769ea82449 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:37.477782 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 32f8d104f6a948d1a97a83769ea82449 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:37.483213 883 tablet_service.cc:1505] Processing CreateTablet for tablet 9c5c937cc7db4ed18fe03f4a8a99a66a (DEFAULT_TABLE table=default.Table_68b9d0d7258a4b5caa1779a4769bdd7f [id=06f467911b6f45849e99f0b6fe615163]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.484858 891 tablet_service.cc:1505] Processing CreateTablet for tablet 7d4e63f54d8443ae8322d4659a165e76 (DEFAULT_TABLE table=default.Table_35338ef5e24d4a1c9972400654a56d83 [id=ad4931c1fa584e83bae93e064eca9621]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.485554 879 tablet_service.cc:1505] Processing CreateTablet for tablet 88e4282d370e4df3a7dcab7819dbac45 (DEFAULT_TABLE table=default.Table_95bf875841aa45c8bf54c51b5839eb51 [id=71660a3234e74d639bca2b039ff420c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.486058 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88e4282d370e4df3a7dcab7819dbac45. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.492188 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d4e63f54d8443ae8322d4659a165e76. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.495005 596 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_c6c5666f7b3044589afd38743a1a200c [id=9700f9fe6e3044748e34954bef8a0038] alter complete (version 3)
I20251211 20:52:37.499682 879 tablet_service.cc:1552] Processing DeleteTablet for tablet 340730a17a834a77bbc20b95aba98e2c with delete_type TABLET_DATA_DELETED (replaced by tablet 3b2c6064c5eb4c0f8979c485f5d85a75) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:37.500195 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c5c937cc7db4ed18fe03f4a8a99a66a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.507869 2000 raft_consensus.cc:493] T f737a7bfb0f44196abdc831bb22d5285 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251211 20:52:37.438372 710 leader_election.cc:343] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Illegal state: Tablet not RUNNING: STOPPED
I20251211 20:52:37.462051 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 2dde7f28e61d41f48015ba6e4f3757f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:37.508843 710 leader_election.cc:304] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.477595 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 32f8d104f6a948d1a97a83769ea82449 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:37.480832 757 tablet_service.cc:1505] Processing CreateTablet for tablet a7afcce522494c9ca1d80e6eea913e6e (DEFAULT_TABLE table=default.Table_cb6a4bac7b694f33a37de2ee78c91ca0 [id=8940f18527c249a9baefc170ed82f45a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.482668 751 tablet_service.cc:1505] Processing CreateTablet for tablet 9c5c937cc7db4ed18fe03f4a8a99a66a (DEFAULT_TABLE table=default.Table_68b9d0d7258a4b5caa1779a4769bdd7f [id=06f467911b6f45849e99f0b6fe615163]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.483942 746 tablet_service.cc:1505] Processing CreateTablet for tablet 7d4e63f54d8443ae8322d4659a165e76 (DEFAULT_TABLE table=default.Table_35338ef5e24d4a1c9972400654a56d83 [id=ad4931c1fa584e83bae93e064eca9621]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:37.509155 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 118860us
I20251211 20:52:37.486042 756 tablet_service.cc:1505] Processing CreateTablet for tablet 88e4282d370e4df3a7dcab7819dbac45 (DEFAULT_TABLE table=default.Table_95bf875841aa45c8bf54c51b5839eb51 [id=71660a3234e74d639bca2b039ff420c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.487042 754 tablet_service.cc:1505] Processing CreateTablet for tablet 745ce63314834c8abdecaa367133a068 (DEFAULT_TABLE table=default.Table_0040a1e388eb4e58a352a1a3884345c4 [id=c2fb490b2d5d44f793304b8939018ca2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.509361 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7afcce522494c9ca1d80e6eea913e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.509510 1696 raft_consensus.cc:2749] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:37.509542 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d4e63f54d8443ae8322d4659a165e76. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.510051 710 leader_election.cc:304] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.510238 1696 raft_consensus.cc:2804] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:37.510288 1696 raft_consensus.cc:493] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:37.510337 1696 raft_consensus.cc:3060] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:37.512694 1696 raft_consensus.cc:515] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:37.512920 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88e4282d370e4df3a7dcab7819dbac45. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.513402 1696 leader_election.cc:290] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:37.514937 710 leader_election.cc:304] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.515182 2092 raft_consensus.cc:2804] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:37.515264 2092 raft_consensus.cc:697] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:37.515461 2092 consensus_queue.cc:237] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.515795 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3fc173c396ff4af5a8589cff18165e85" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:37.516314 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 745ce63314834c8abdecaa367133a068. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:37.516415 711 leader_election.cc:343] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.517025 2062 raft_consensus.cc:493] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:37.519316 710 leader_election.cc:304] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.519690 2093 raft_consensus.cc:2804] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:37.519752 2093 raft_consensus.cc:493] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:37.519801 2093 raft_consensus.cc:3060] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:37.520608 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c5c937cc7db4ed18fe03f4a8a99a66a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.522338 2093 raft_consensus.cc:515] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.523339 2093 leader_election.cc:290] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:37.523836 710 leader_election.cc:304] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.523873 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a77e3e744ff42ccb68897704b4c2f1d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:37.524396 711 leader_election.cc:343] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:37.525157 710 leader_election.cc:343] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 6a526a6d1bc84cefad2c5d897223292e
I20251211 20:52:37.525729 710 leader_election.cc:304] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.525329 2093 raft_consensus.cc:2804] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:37.526088 2093 raft_consensus.cc:493] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:37.526203 2093 raft_consensus.cc:3060] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:37.527745 1696 raft_consensus.cc:2749] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:37.529992 1165 log.cc:1199] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/87863daf77304a399318cdf56da9f9bb
I20251211 20:52:37.530896 1165 ts_tablet_manager.cc:1950] T 87863daf77304a399318cdf56da9f9bb P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:37.532222 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_5b7fdd64f8f245a69b384dd22aedba9e" } new_table_name: "default.Table_defdeab28749411590cf75b5251f604b" modify_external_catalogs: true
I20251211 20:52:37.533864 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9c7198623d9b41158b2ac00adc92f327" } modify_external_catalogs: true
I20251211 20:52:37.534026 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9c7198623d9b41158b2ac00adc92f327" } modify_external_catalogs: true
I20251211 20:52:37.538139 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 87863daf77304a399318cdf56da9f9bb (table default.Table_28b13655ab2b47e68f2bd01d618a2b58 [id=9b92dd859a6641b8ade2b49515d1e810]) successfully deleted
I20251211 20:52:37.531164 710 leader_election.cc:304] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.538642 1696 raft_consensus.cc:2804] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:37.538875 1696 raft_consensus.cc:697] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:37.539146 1696 consensus_queue.cc:237] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.508623 824 heartbeater.cc:344] Connected to a master server at 127.0.104.61:40749
I20251211 20:52:37.548887 1168 tablet_bootstrap.cc:492] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:37.549113 1168 ts_tablet_manager.cc:1403] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.147s user 0.018s sys 0.047s
I20251211 20:52:37.549580 2092 consensus_queue.cc:1048] T ef4fada332cf4b5c9da26ba5f908b4c6 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:37.550099 1168 raft_consensus.cc:359] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.550407 1168 raft_consensus.cc:385] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.550503 1168 raft_consensus.cc:740] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.550796 1168 consensus_queue.cc:260] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.551338 1168 ts_tablet_manager.cc:1434] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:37.551769 1168 tablet_bootstrap.cc:492] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:37.553575 1168 tablet_bootstrap.cc:654] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.554613 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet cce44b80c97948e1af7ac68e77cccfd6
I20251211 20:52:37.558504 751 tablet_service.cc:1552] Processing DeleteTablet for tablet cce44b80c97948e1af7ac68e77cccfd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:37.560587 1168 tablet_bootstrap.cc:492] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:37.561226 1168 ts_tablet_manager.cc:1403] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.001s
I20251211 20:52:37.562256 1168 raft_consensus.cc:359] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.563548 1168 raft_consensus.cc:385] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.563794 1168 raft_consensus.cc:740] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.564148 1168 consensus_queue.cc:260] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:37.564651 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 7452dd736b0440fcb4626858e794676f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7452dd736b0440fcb4626858e794676f already in progress: creating tablet
I20251211 20:52:37.565282 2093 raft_consensus.cc:515] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.565457 1168 ts_tablet_manager.cc:1434] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:37.566154 1168 tablet_bootstrap.cc:492] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:37.568152 883 tablet_service.cc:1552] Processing DeleteTablet for tablet cce44b80c97948e1af7ac68e77cccfd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:37.570688 1168 tablet_bootstrap.cc:654] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.571189 1165 tablet_replica.cc:333] T e28d66ce4711430cb09a9650e99fbe9e P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:37.571424 1165 raft_consensus.cc:2243] T e28d66ce4711430cb09a9650e99fbe9e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:37.571686 1165 raft_consensus.cc:2272] T e28d66ce4711430cb09a9650e99fbe9e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:37.572274 1165 ts_tablet_manager.cc:1916] T e28d66ce4711430cb09a9650e99fbe9e P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:37.574807 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8229b704e7464b30b7d2902a938ce33b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:37.575377 711 leader_election.cc:343] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.575795 2093 leader_election.cc:290] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:37.576581 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2e800d8cc5ea4535a065d7d9243326b4 has no consensus state
I20251211 20:52:37.577085 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_39446be1bf754b5da6d932892c7280cf" } new_table_name: "default.Table_e6f3a819faad48aabfad3a235c3c1b95" modify_external_catalogs: true
I20251211 20:52:37.579945 1813 raft_consensus.cc:493] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:37.580111 1813 raft_consensus.cc:515] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.580715 1813 leader_election.cc:290] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:37.582137 847 leader_election.cc:343] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 55c40b2464f8444ca6c9fa0400f7f7f2
I20251211 20:52:37.588434 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 87863daf77304a399318cdf56da9f9bb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:37.589195 1168 tablet_bootstrap.cc:492] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:37.589325 1168 ts_tablet_manager.cc:1403] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.002s
I20251211 20:52:37.590389 1168 raft_consensus.cc:359] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.590596 1168 raft_consensus.cc:385] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.590654 1168 raft_consensus.cc:740] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.590843 1168 consensus_queue.cc:260] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.591315 1168 ts_tablet_manager.cc:1434] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:37.591625 1168 tablet_bootstrap.cc:492] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:37.593284 1168 tablet_bootstrap.cc:654] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.607612 2093 raft_consensus.cc:493] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:37.607820 2093 raft_consensus.cc:515] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.608527 2093 leader_election.cc:290] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:37.609314 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7bd218b3ff340b2a589c9517b49a649" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:37.609907 1160 tablet_bootstrap.cc:492] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
W20251211 20:52:37.609907 711 leader_election.cc:343] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.610114 1160 ts_tablet_manager.cc:1403] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.247s user 0.005s sys 0.007s
I20251211 20:52:37.610878 1160 raft_consensus.cc:359] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.611039 1160 raft_consensus.cc:385] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.611090 1160 raft_consensus.cc:740] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.611270 1160 consensus_queue.cc:260] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.613216 1160 ts_tablet_manager.cc:1434] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
W20251211 20:52:37.615446 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 2dde7f28e61d41f48015ba6e4f3757f2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2dde7f28e61d41f48015ba6e4f3757f2 already in progress: creating tablet
I20251211 20:52:37.623679 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3fc173c396ff4af5a8589cff18165e85" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:37.623869 1049 raft_consensus.cc:3060] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:37.624722 1160 tablet_bootstrap.cc:492] T 9f8aa28c1c9b4fc58dfc128952c180ba P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:37.626335 1160 tablet_bootstrap.cc:654] T 9f8aa28c1c9b4fc58dfc128952c180ba P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:37.627230 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 59acb8474e114bf7ae7f1f043d611459 consensus state has no leader
I20251211 20:52:37.627421 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_6fbcf571523d4693b48035d60c6322ad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:37.627619 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_264b707b3aba4b1a93ee5b8131af5192"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:37.628049 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6fbcf571523d4693b48035d60c6322ad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:37.628175 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_264b707b3aba4b1a93ee5b8131af5192 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:37.629170 2000 raft_consensus.cc:515] T f737a7bfb0f44196abdc831bb22d5285 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.629518 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_d8d764377e524faf931940bfe23100be"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:37.629843 2000 leader_election.cc:290] T f737a7bfb0f44196abdc831bb22d5285 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
W20251211 20:52:37.630082 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d8d764377e524faf931940bfe23100be in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:37.546129 1041 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2d849c62f7434099a16f62c2a2fc5e85" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:37.630935 1041 raft_consensus.cc:2468] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:37.632185 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a77e3e744ff42ccb68897704b4c2f1d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:37.633332 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_2c4e58d88571405eab18aee814981780" } new_table_name: "default.Table_b6e6aea68c88426fb01353146c06b6ac" modify_external_catalogs: true
I20251211 20:52:37.635740 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f737a7bfb0f44196abdc831bb22d5285" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:37.636098 1164 tablet_replica.cc:333] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:37.636281 1164 raft_consensus.cc:2243] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
W20251211 20:52:37.636322 983 leader_election.cc:343] T f737a7bfb0f44196abdc831bb22d5285 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.636433 1164 raft_consensus.cc:2272] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:37.636966 1164 ts_tablet_manager.cc:1916] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:37.637259 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 32f8d104f6a948d1a97a83769ea82449 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 32f8d104f6a948d1a97a83769ea82449 already in progress: creating tablet
I20251211 20:52:37.638226 2000 raft_consensus.cc:493] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:37.619890 2062 raft_consensus.cc:515] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.640630 2062 leader_election.cc:290] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:37.640743 2062 raft_consensus.cc:493] T 8b793241ee3b42b1a38e9f2f05e1173b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:37.640828 2062 raft_consensus.cc:515] T 8b793241ee3b42b1a38e9f2f05e1173b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.641286 2062 leader_election.cc:290] T 8b793241ee3b42b1a38e9f2f05e1173b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:37.644282 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f737a7bfb0f44196abdc831bb22d5285" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:37.644713 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e6ba12f94f5f4d688f4222a4c2fb5436" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:37.645035 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b793241ee3b42b1a38e9f2f05e1173b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:37.645495 710 leader_election.cc:304] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.645928 2093 raft_consensus.cc:2804] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:37.646001 2093 raft_consensus.cc:493] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:37.646062 2093 raft_consensus.cc:3060] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:37.648669 2093 raft_consensus.cc:515] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.649257 2093 leader_election.cc:290] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:37.649565 1041 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7bd218b3ff340b2a589c9517b49a649" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:37.649765 1041 raft_consensus.cc:2468] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
W20251211 20:52:37.649868 421 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 46720.4us
I20251211 20:52:37.650842 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2d849c62f7434099a16f62c2a2fc5e85" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:37.650902 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e6ba12f94f5f4d688f4222a4c2fb5436" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:37.651233 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b793241ee3b42b1a38e9f2f05e1173b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:37.651394 711 leader_election.cc:343] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.652361 1160 tablet_bootstrap.cc:492] T 9f8aa28c1c9b4fc58dfc128952c180ba P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
W20251211 20:52:37.652438 983 leader_election.cc:343] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.652519 1160 ts_tablet_manager.cc:1403] T 9f8aa28c1c9b4fc58dfc128952c180ba P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.028s user 0.004s sys 0.000s
W20251211 20:52:37.652638 983 leader_election.cc:343] T 8b793241ee3b42b1a38e9f2f05e1173b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.653226 1160 raft_consensus.cc:359] T 9f8aa28c1c9b4fc58dfc128952c180ba P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.653362 1160 raft_consensus.cc:385] T 9f8aa28c1c9b4fc58dfc128952c180ba P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.653400 1160 raft_consensus.cc:740] T 9f8aa28c1c9b4fc58dfc128952c180ba P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.653609 1160 consensus_queue.cc:260] T 9f8aa28c1c9b4fc58dfc128952c180ba P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.653729 1043 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8229b704e7464b30b7d2902a938ce33b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:37.653882 1043 raft_consensus.cc:3060] T 8229b704e7464b30b7d2902a938ce33b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:37.638353 2000 raft_consensus.cc:515] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:37.654789 847 leader_election.cc:343] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 55c40b2464f8444ca6c9fa0400f7f7f2
W20251211 20:52:37.654908 982 leader_election.cc:343] T f737a7bfb0f44196abdc831bb22d5285 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.654896 847 leader_election.cc:304] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:37.654989 982 leader_election.cc:304] T f737a7bfb0f44196abdc831bb22d5285 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.655179 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2d849c62f7434099a16f62c2a2fc5e85" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:37.655267 1813 raft_consensus.cc:2749] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:37.655305 1044 raft_consensus.cc:3060] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:37.655470 961 heartbeater.cc:344] Connected to a master server at 127.0.104.61:40749
I20251211 20:52:37.655783 710 leader_election.cc:304] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.656206 1165 ts_tablet_manager.cc:1929] T e28d66ce4711430cb09a9650e99fbe9e P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:37.656286 883 tablet_service.cc:1505] Processing CreateTablet for tablet 745ce63314834c8abdecaa367133a068 (DEFAULT_TABLE table=default.Table_0040a1e388eb4e58a352a1a3884345c4 [id=c2fb490b2d5d44f793304b8939018ca2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.656674 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 745ce63314834c8abdecaa367133a068. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.659436 1813 raft_consensus.cc:493] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:37.659550 1813 raft_consensus.cc:515] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.660104 1813 leader_election.cc:290] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:37.660404 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b81a4cef27164824b52695039744db15 consensus state has no leader
I20251211 20:52:37.660737 596 master_service.cc:954] ReplaceTablet: received request to replace tablet 042f8bf13a7a4712b5bdc04671cd9938 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:37.661080 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet c91ff29847d54e55b19321652a857e9f
I20251211 20:52:37.661270 1160 ts_tablet_manager.cc:1434] T 9f8aa28c1c9b4fc58dfc128952c180ba P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.009s user 0.001s sys 0.000s
W20251211 20:52:37.661450 847 leader_election.cc:343] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: d762b4afb5ff447dac3468393826205a
I20251211 20:52:37.661567 1160 tablet_bootstrap.cc:492] T 40722aef4e194ff094d271a079bda9d9 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:37.661893 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 098054406a1d459b824a094f50e71e7e
I20251211 20:52:37.662201 1023 tablet_service.cc:1505] Processing CreateTablet for tablet a7afcce522494c9ca1d80e6eea913e6e (DEFAULT_TABLE table=default.Table_cb6a4bac7b694f33a37de2ee78c91ca0 [id=8940f18527c249a9baefc170ed82f45a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.662307 593 catalog_manager.cc:6724] ReplaceTablet: tablet 098054406a1d459b824a094f50e71e7e deleted and replaced by tablet aa50ddb2789b4be3bfa9aedef9287109
I20251211 20:52:37.663214 888 tablet_service.cc:1552] Processing DeleteTablet for tablet c91ff29847d54e55b19321652a857e9f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:37.663331 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 098054406a1d459b824a094f50e71e7e with delete_type TABLET_DATA_DELETED (replaced by tablet aa50ddb2789b4be3bfa9aedef9287109) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:37.665788 595 catalog_manager.cc:5654] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:37.667546 595 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 5bd801124df84a6c96f4912993f1bdfd (table default.Table_9aa05c766d874348a9e0eb2c016058d3 [id=44f2f2bf53af4b038ced294f20db9e48]). Expected version 1 got 0
I20251211 20:52:37.668331 1160 tablet_bootstrap.cc:654] T 40722aef4e194ff094d271a079bda9d9 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.668184 595 catalog_manager.cc:5654] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:37.669124 595 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b072b759d1254859b0b8d401e3078792 (table default.Table_b56de1e85a1b42ea81308593884dadb8 [id=a137f0ddf52145a99c62444602ab3fdb]). Expected version 4 got 0
I20251211 20:52:37.669736 595 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet c7a2610f20df4c1ca81eae2fc7dba6bb (table default.Table_59072235d93b41c0b5cd5e3debcb8ddf [id=9a15a8a075af4969ae2c8b3be1bfed78]). Expected version 1 got 0
I20251211 20:52:37.663321 751 tablet_service.cc:1552] Processing DeleteTablet for tablet c91ff29847d54e55b19321652a857e9f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:37.663444 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 098054406a1d459b824a094f50e71e7e with delete_type TABLET_DATA_DELETED (replaced by tablet aa50ddb2789b4be3bfa9aedef9287109) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:37.676467 2092 raft_consensus.cc:2804] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:37.678856 2092 raft_consensus.cc:493] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:37.678946 2092 raft_consensus.cc:3060] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:37.676957 2101 raft_consensus.cc:493] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:37.679922 2101 raft_consensus.cc:515] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.681317 2101 leader_election.cc:290] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:37.681689 1168 tablet_bootstrap.cc:492] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:37.681813 1168 ts_tablet_manager.cc:1403] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.090s user 0.010s sys 0.014s
I20251211 20:52:37.681995 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ccc3ddd4b7f74f11a3835997cb7ae8d0" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:37.682521 711 leader_election.cc:343] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.682576 1168 raft_consensus.cc:359] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.682730 1168 raft_consensus.cc:385] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.682788 1168 raft_consensus.cc:740] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.682981 1168 consensus_queue.cc:260] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.683454 1168 ts_tablet_manager.cc:1434] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:37.683763 1168 tablet_bootstrap.cc:492] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:37.673405 1164 ts_tablet_manager.cc:1929] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:37.687781 1164 log.cc:1199] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/340730a17a834a77bbc20b95aba98e2c
I20251211 20:52:37.687908 2092 raft_consensus.cc:515] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.688305 1164 ts_tablet_manager.cc:1950] T 340730a17a834a77bbc20b95aba98e2c P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:37.688553 2092 leader_election.cc:290] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:37.689142 1168 tablet_bootstrap.cc:654] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.689261 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7bd218b3ff340b2a589c9517b49a649" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:37.689775 711 leader_election.cc:343] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.690821 1160 tablet_bootstrap.cc:492] T 40722aef4e194ff094d271a079bda9d9 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:37.690999 1160 ts_tablet_manager.cc:1403] T 40722aef4e194ff094d271a079bda9d9 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.002s
I20251211 20:52:37.691622 2092 raft_consensus.cc:493] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:37.691808 1160 raft_consensus.cc:359] T 40722aef4e194ff094d271a079bda9d9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.691788 2092 raft_consensus.cc:515] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.692015 1160 raft_consensus.cc:385] T 40722aef4e194ff094d271a079bda9d9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.692499 2092 leader_election.cc:290] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:37.692529 1160 raft_consensus.cc:740] T 40722aef4e194ff094d271a079bda9d9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.692808 1160 consensus_queue.cc:260] T 40722aef4e194ff094d271a079bda9d9 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.687901 1165 log.cc:1199] T e28d66ce4711430cb09a9650e99fbe9e P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/e28d66ce4711430cb09a9650e99fbe9e
I20251211 20:52:37.693264 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7a2610f20df4c1ca81eae2fc7dba6bb" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:37.695195 711 leader_election.cc:343] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.695484 1165 ts_tablet_manager.cc:1950] T e28d66ce4711430cb09a9650e99fbe9e P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:37.696995 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet e28d66ce4711430cb09a9650e99fbe9e (table default.Table_8b8ed719985f42128e56bc26105e7a9c [id=401822f4521f46eeb815147f643a6fcf]) successfully deleted
I20251211 20:52:37.698875 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_ee696fa128ac464da5a6f99bd3d7ed7a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:37.699529 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ee696fa128ac464da5a6f99bd3d7ed7a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:37.700614 1048 raft_consensus.cc:3060] T 1a77e3e744ff42ccb68897704b4c2f1d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:37.701267 1160 ts_tablet_manager.cc:1434] T 40722aef4e194ff094d271a079bda9d9 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.010s user 0.001s sys 0.000s
I20251211 20:52:37.701177 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_6a85485dbf204937a1c5a8e0e151fc68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:37.701598 982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.1:42177, user_credentials={real_user=slave}} blocked reactor thread for 46692.6us
W20251211 20:52:37.701723 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6a85485dbf204937a1c5a8e0e151fc68 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:37.701753 2062 raft_consensus.cc:2749] T f737a7bfb0f44196abdc831bb22d5285 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:37.701853 982 leader_election.cc:343] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.701960 982 leader_election.cc:304] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.702077 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7afcce522494c9ca1d80e6eea913e6e. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:37.702298 982 leader_election.cc:343] T 8b793241ee3b42b1a38e9f2f05e1173b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.702380 982 leader_election.cc:304] T 8b793241ee3b42b1a38e9f2f05e1173b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.702891 1166 raft_consensus.cc:359] T 4f807596fa6a4e66b89566b5d5dd1080 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.703051 1166 raft_consensus.cc:385] T 4f807596fa6a4e66b89566b5d5dd1080 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.703095 1166 raft_consensus.cc:740] T 4f807596fa6a4e66b89566b5d5dd1080 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.703292 1166 consensus_queue.cc:260] T 4f807596fa6a4e66b89566b5d5dd1080 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.704439 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ccc3ddd4b7f74f11a3835997cb7ae8d0" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:37.704600 1047 raft_consensus.cc:2468] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:37.705281 2062 raft_consensus.cc:2749] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:37.705412 2062 raft_consensus.cc:2749] T 8b793241ee3b42b1a38e9f2f05e1173b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:37.708065 2000 leader_election.cc:290] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:37.708572 588 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_279bb02bdbd743159babc9a4e6498437 [id=f78e1da479af4d23a552af1c4fc55f81] alter complete (version 4)
I20251211 20:52:37.708683 1160 tablet_bootstrap.cc:492] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:37.715961 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 340730a17a834a77bbc20b95aba98e2c (table default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f]) successfully deleted
W20251211 20:52:37.716442 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet cce44b80c97948e1af7ac68e77cccfd6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cce44b80c97948e1af7ac68e77cccfd6 already in progress: creating tablet
W20251211 20:52:37.716809 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet c91ff29847d54e55b19321652a857e9f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c91ff29847d54e55b19321652a857e9f already in progress: creating tablet
W20251211 20:52:37.717146 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 098054406a1d459b824a094f50e71e7e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 098054406a1d459b824a094f50e71e7e already in progress: creating tablet
I20251211 20:52:37.717952 1049 raft_consensus.cc:2468] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:37.721412 597 master_service.cc:954] ReplaceTablet: received request to replace tablet 2d849c62f7434099a16f62c2a2fc5e85 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:37.721983 591 master_service.cc:954] ReplaceTablet: received request to replace tablet 2197822e0f11484f9679c4858626ac2c from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:37.710945 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14e2650f8ec1482ea9511b989fdc963c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:37.710822 1160 tablet_bootstrap.cc:654] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:37.722862 983 leader_election.cc:343] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.725710 1044 raft_consensus.cc:2468] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:37.726085 1165 tablet_replica.cc:333] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:37.727670 1043 raft_consensus.cc:2468] T 8229b704e7464b30b7d2902a938ce33b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:37.729492 1048 raft_consensus.cc:2468] T 1a77e3e744ff42ccb68897704b4c2f1d P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:37.731662 1163 ts_tablet_manager.cc:1929] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:37.731848 1163 log.cc:1199] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/87863daf77304a399318cdf56da9f9bb
I20251211 20:52:37.732195 1163 ts_tablet_manager.cc:1950] T 87863daf77304a399318cdf56da9f9bb P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:37.732623 1160 tablet_bootstrap.cc:492] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:37.732833 1160 ts_tablet_manager.cc:1403] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.001s
I20251211 20:52:37.728615 1168 tablet_bootstrap.cc:492] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:37.734437 1168 ts_tablet_manager.cc:1403] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.051s user 0.002s sys 0.010s
I20251211 20:52:37.734472 1165 raft_consensus.cc:2243] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:37.734721 1165 raft_consensus.cc:2272] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:37.735219 1168 raft_consensus.cc:359] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.735399 1168 raft_consensus.cc:385] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.735450 1168 raft_consensus.cc:740] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.735678 1168 consensus_queue.cc:260] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:37.735904 847 leader_election.cc:343] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: d762b4afb5ff447dac3468393826205a
I20251211 20:52:37.736047 847 leader_election.cc:304] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:37.736106 1168 ts_tablet_manager.cc:1434] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:37.736372 1168 tablet_bootstrap.cc:492] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:37.736470 1813 raft_consensus.cc:2749] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:37.737901 1168 tablet_bootstrap.cc:654] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.737937 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14e2650f8ec1482ea9511b989fdc963c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:37.738690 1160 raft_consensus.cc:359] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:37.738842 982 leader_election.cc:343] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.739005 982 leader_election.cc:304] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.739404 1160 raft_consensus.cc:385] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.739622 1160 raft_consensus.cc:740] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.739643 710 leader_election.cc:304] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.740013 710 leader_election.cc:304] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.739998 1160 consensus_queue.cc:260] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.740360 2077 consensus_queue.cc:1048] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:37.740454 2086 raft_consensus.cc:2804] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:37.740566 2086 raft_consensus.cc:493] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:37.740621 2086 raft_consensus.cc:3060] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:37.740728 2077 raft_consensus.cc:2804] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:37.740846 2077 raft_consensus.cc:697] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:37.741070 2077 consensus_queue.cc:237] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:37.743134 2086 raft_consensus.cc:515] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.744258 710 leader_election.cc:304] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.745208 2101 raft_consensus.cc:2804] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:37.745319 2101 raft_consensus.cc:697] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:37.745592 2101 consensus_queue.cc:237] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.746845 1160 ts_tablet_manager.cc:1434] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.014s user 0.001s sys 0.001s
I20251211 20:52:37.746874 710 leader_election.cc:304] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.747313 2019 raft_consensus.cc:2804] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:37.747435 2019 raft_consensus.cc:697] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:37.747663 2019 consensus_queue.cc:237] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.748965 710 leader_election.cc:304] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.749110 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f2dc337cf8ba44408562bb0710ec4527" } modify_external_catalogs: true
I20251211 20:52:37.749266 2087 raft_consensus.cc:2804] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:37.749364 2087 raft_consensus.cc:697] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:37.749347 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f2dc337cf8ba44408562bb0710ec4527" } modify_external_catalogs: true
I20251211 20:52:37.749599 2087 consensus_queue.cc:237] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.750780 1160 tablet_bootstrap.cc:492] T 3c578d81e8dd41aaa0f707c4e3fed225 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:37.754297 1160 tablet_bootstrap.cc:654] T 3c578d81e8dd41aaa0f707c4e3fed225 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.757332 2086 leader_election.cc:290] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:37.758052 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ccc3ddd4b7f74f11a3835997cb7ae8d0" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:37.758746 711 leader_election.cc:343] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.760236 1168 tablet_bootstrap.cc:492] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:37.760452 1168 ts_tablet_manager.cc:1403] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.002s
I20251211 20:52:37.761209 1168 raft_consensus.cc:359] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.761421 1168 raft_consensus.cc:385] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.761531 1168 raft_consensus.cc:740] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.761795 1168 consensus_queue.cc:260] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.762377 1168 ts_tablet_manager.cc:1434] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:37.762702 1168 tablet_bootstrap.cc:492] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:37.764245 1168 tablet_bootstrap.cc:654] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:37.767493 592 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ee7b93b0b87541d9af6eb9e68b4cea41 consensus state has no leader
I20251211 20:52:37.768240 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_c15e1d9cc43c4d0b951d51344fb6ad09"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:37.768819 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c15e1d9cc43c4d0b951d51344fb6ad09 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:37.774430 1168 tablet_bootstrap.cc:492] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:37.774655 1168 ts_tablet_manager.cc:1403] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.004s
I20251211 20:52:37.775527 1168 raft_consensus.cc:359] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.775803 1168 raft_consensus.cc:385] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.775903 1168 raft_consensus.cc:740] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.776191 1168 consensus_queue.cc:260] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.776379 1160 tablet_bootstrap.cc:492] T 3c578d81e8dd41aaa0f707c4e3fed225 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:37.776580 1160 ts_tablet_manager.cc:1403] T 3c578d81e8dd41aaa0f707c4e3fed225 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.026s user 0.004s sys 0.000s
I20251211 20:52:37.776845 1168 ts_tablet_manager.cc:1434] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:37.777158 1168 tablet_bootstrap.cc:492] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:37.778743 1168 tablet_bootstrap.cc:654] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.778688 1160 raft_consensus.cc:359] T 3c578d81e8dd41aaa0f707c4e3fed225 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.778915 2062 raft_consensus.cc:2749] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:37.778942 1160 raft_consensus.cc:385] T 3c578d81e8dd41aaa0f707c4e3fed225 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.778995 1160 raft_consensus.cc:740] T 3c578d81e8dd41aaa0f707c4e3fed225 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.779170 1160 consensus_queue.cc:260] T 3c578d81e8dd41aaa0f707c4e3fed225 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.754752 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7bd218b3ff340b2a589c9517b49a649" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:37.779685 1048 raft_consensus.cc:3060] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:37.782500 1048 raft_consensus.cc:2468] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:37.783296 1160 ts_tablet_manager.cc:1434] T 3c578d81e8dd41aaa0f707c4e3fed225 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20251211 20:52:37.762555 1166 ts_tablet_manager.cc:1434] T 4f807596fa6a4e66b89566b5d5dd1080 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.293s user 0.003s sys 0.003s
I20251211 20:52:37.784159 1166 tablet_bootstrap.cc:492] T e4647105ab57454d883ebfcde7bde891 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:37.785729 1160 tablet_bootstrap.cc:492] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:37.785775 1166 tablet_bootstrap.cc:654] T e4647105ab57454d883ebfcde7bde891 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.787315 1160 tablet_bootstrap.cc:654] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.769677 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 731) took 1648 ms. Trace:
I20251211 20:52:37.787720 1163 rpcz_store.cc:276] 1211 20:52:36.120882 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:36.120947 (+ 65us) service_pool.cc:225] Handling call
1211 20:52:37.435893 (+1314946us) ts_tablet_manager.cc:1151] Deleting tablet 87863daf77304a399318cdf56da9f9bb
1211 20:52:37.435904 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:37.619384 (+183480us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:37.731520 (+112136us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:37.769659 (+ 38139us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1314596}
I20251211 20:52:37.788188 1163 tablet_replica.cc:333] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:37.788431 1163 raft_consensus.cc:2243] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:37.788676 1163 raft_consensus.cc:2272] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:37.789249 1163 ts_tablet_manager.cc:1916] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:37.760360 1043 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7a2610f20df4c1ca81eae2fc7dba6bb" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:37.783008 2100 tablet.cc:1722] T dd425601d89148eb95525f796a3abf1e P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:37.791723 593 master_service.cc:954] ReplaceTablet: received request to replace tablet 3eac21a07ba34e1fa275e25ee9dcabb1 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:37.794432 1043 raft_consensus.cc:2468] T c7a2610f20df4c1ca81eae2fc7dba6bb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:37.796968 914 raft_consensus.cc:3060] T 9f8aa28c1c9b4fc58dfc128952c180ba P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:37.800487 2092 consensus_queue.cc:1048] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:37.807797 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ccc3ddd4b7f74f11a3835997cb7ae8d0" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:37.808010 1047 raft_consensus.cc:3060] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:37.810648 1047 raft_consensus.cc:2468] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:37.811703 710 leader_election.cc:304] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.812045 2092 raft_consensus.cc:2804] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:37.812162 2092 raft_consensus.cc:697] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:37.812389 2092 consensus_queue.cc:237] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.812351 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_86d6443f6a564414a7f024e4aa12d450"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:37.813058 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_86d6443f6a564414a7f024e4aa12d450 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:37.813967 710 leader_election.cc:304] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.814318 588 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_0cc01b2a751d4591b9b227f46093e552 [id=c842792b01cb4c689fcfd3d796151508] alter complete (version 2)
I20251211 20:52:37.815757 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_3f5d0d2b042c42ed88431232445bef02" } modify_external_catalogs: true
I20251211 20:52:37.815953 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_3f5d0d2b042c42ed88431232445bef02" } modify_external_catalogs: true
I20251211 20:52:37.815977 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 340730a17a834a77bbc20b95aba98e2c with delete_type TABLET_DATA_DELETED (replaced by tablet 3b2c6064c5eb4c0f8979c485f5d85a75) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:37.816196 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 55c40b2464f8444ca6c9fa0400f7f7f2 with delete_type TABLET_DATA_DELETED (replaced by tablet fb030a62796248869d46b682656eaeb0) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:37.817186 1164 tablet_replica.cc:333] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:37.817373 1164 raft_consensus.cc:2243] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:37.817504 1164 raft_consensus.cc:2272] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:37.818041 1164 ts_tablet_manager.cc:1916] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:37.818871 2086 raft_consensus.cc:2804] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:37.818949 2086 raft_consensus.cc:493] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:37.819002 2086 raft_consensus.cc:3060] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:37.821590 2086 raft_consensus.cc:515] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.822296 2086 leader_election.cc:290] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:37.822871 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7a2610f20df4c1ca81eae2fc7dba6bb" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:37.823328 711 leader_election.cc:343] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.823976 710 leader_election.cc:304] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:37.824426 582 catalog_manager.cc:4952] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 340730a17a834a77bbc20b95aba98e2c because the tablet was not found. No further retry: Not found: Tablet not found: 340730a17a834a77bbc20b95aba98e2c
I20251211 20:52:37.824977 1164 ts_tablet_manager.cc:1929] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:37.825165 1164 log.cc:1199] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/55c40b2464f8444ca6c9fa0400f7f7f2
I20251211 20:52:37.825522 1164 ts_tablet_manager.cc:1950] T 55c40b2464f8444ca6c9fa0400f7f7f2 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:37.826581 592 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 9f8aa28c1c9b4fc58dfc128952c180ba (table default.Table_bf7d402a36be44cfb6b79041906d26a1 [id=0327e686e7ed4475a3e8cda8721a3a3a]). Expected version 1 got 0
I20251211 20:52:37.826612 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 55c40b2464f8444ca6c9fa0400f7f7f2 (table default.Table_069bb219e15e43be995ff011e2289a84 [id=bc0b945e86cf43fdb99d56658ca1576c]) successfully deleted
I20251211 20:52:37.826875 592 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 40722aef4e194ff094d271a079bda9d9 (table default.Table_12f41f708f1949598d6176d89777aa69 [id=3309205faabf4a42acadc43766e4f57f]). Expected version 1 got 0
I20251211 20:52:37.835794 1160 tablet_bootstrap.cc:492] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:37.836052 1160 ts_tablet_manager.cc:1403] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.050s user 0.003s sys 0.003s
I20251211 20:52:37.836855 1160 raft_consensus.cc:359] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.837121 1160 raft_consensus.cc:385] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.837226 1160 raft_consensus.cc:740] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.837504 1160 consensus_queue.cc:260] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.838426 1160 ts_tablet_manager.cc:1434] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:37.843047 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 3eac21a07ba34e1fa275e25ee9dcabb1
I20251211 20:52:37.837705 1168 tablet_bootstrap.cc:492] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:37.843518 593 catalog_manager.cc:6724] ReplaceTablet: tablet 3eac21a07ba34e1fa275e25ee9dcabb1 deleted and replaced by tablet 2085f871ea0a4caebbf2b1459b8c1e3b
I20251211 20:52:37.843688 1168 ts_tablet_manager.cc:1403] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.067s user 0.016s sys 0.009s
I20251211 20:52:37.844110 2086 raft_consensus.cc:2804] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:37.844193 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 3eac21a07ba34e1fa275e25ee9dcabb1 with delete_type TABLET_DATA_DELETED (replaced by tablet 2085f871ea0a4caebbf2b1459b8c1e3b) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:37.844251 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 3eac21a07ba34e1fa275e25ee9dcabb1 with delete_type TABLET_DATA_DELETED (replaced by tablet 2085f871ea0a4caebbf2b1459b8c1e3b) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:37.844290 2086 raft_consensus.cc:697] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:37.844561 2086 consensus_queue.cc:237] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.844638 1168 raft_consensus.cc:359] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.844806 1168 raft_consensus.cc:385] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.844895 1168 raft_consensus.cc:740] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.845093 1168 consensus_queue.cc:260] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.845214 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_279bb02bdbd743159babc9a4e6498437" } modify_external_catalogs: true
I20251211 20:52:37.845371 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_279bb02bdbd743159babc9a4e6498437" } modify_external_catalogs: true
I20251211 20:52:37.845603 1168 ts_tablet_manager.cc:1434] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:37.845887 1168 tablet_bootstrap.cc:492] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:37.847690 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7a2610f20df4c1ca81eae2fc7dba6bb" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:37.847565 1168 tablet_bootstrap.cc:654] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.847882 1047 raft_consensus.cc:3060] T c7a2610f20df4c1ca81eae2fc7dba6bb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:37.850859 1047 raft_consensus.cc:2468] T c7a2610f20df4c1ca81eae2fc7dba6bb P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:37.853422 710 leader_election.cc:304] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.853775 2077 raft_consensus.cc:2804] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:37.853940 2077 raft_consensus.cc:697] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:37.854190 2077 consensus_queue.cc:237] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.859175 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_983c3bbea3c54b88bebbe065e4660218" } modify_external_catalogs: true
I20251211 20:52:37.859438 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_983c3bbea3c54b88bebbe065e4660218" } modify_external_catalogs: true
I20251211 20:52:37.860704 2093 raft_consensus.cc:493] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:37.860930 2093 raft_consensus.cc:515] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.862123 2093 leader_election.cc:290] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:37.862368 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 05385f20e88c46108e09a78814811209 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:37.862444 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b631e6b3048e428f901af1b5b0b1ab3c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:37.863013 711 leader_election.cc:343] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.863955 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b631e6b3048e428f901af1b5b0b1ab3c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
W20251211 20:52:37.871929 710 leader_election.cc:343] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.872187 710 leader_election.cc:304] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:37.872526 2077 raft_consensus.cc:2749] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:37.872725 1168 tablet_bootstrap.cc:492] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:37.872898 1168 ts_tablet_manager.cc:1403] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.027s user 0.005s sys 0.004s
I20251211 20:52:37.874377 1168 raft_consensus.cc:359] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.874580 1168 raft_consensus.cc:385] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.874653 1168 raft_consensus.cc:740] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.874867 1168 consensus_queue.cc:260] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.875326 1168 ts_tablet_manager.cc:1434] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:37.875651 1168 tablet_bootstrap.cc:492] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:37.877936 1168 tablet_bootstrap.cc:654] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.880528 1160 tablet_bootstrap.cc:492] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:37.882236 1160 tablet_bootstrap.cc:654] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.886155 1164 tablet_replica.cc:333] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:37.886359 1164 raft_consensus.cc:2243] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:37.886560 1164 raft_consensus.cc:2272] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:37.735219 1165 ts_tablet_manager.cc:1916] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:37.887570 1164 ts_tablet_manager.cc:1916] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:37.896440 1164 ts_tablet_manager.cc:1929] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20251211 20:52:37.898206 1164 log.cc:1199] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/3eac21a07ba34e1fa275e25ee9dcabb1
I20251211 20:52:37.898783 1164 ts_tablet_manager.cc:1950] T 3eac21a07ba34e1fa275e25ee9dcabb1 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:37.899864 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 3eac21a07ba34e1fa275e25ee9dcabb1 (table default.Table_c6c5666f7b3044589afd38743a1a200c [id=9700f9fe6e3044748e34954bef8a0038]) successfully deleted
I20251211 20:52:37.900640 1164 tablet_replica.cc:333] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:37.901069 1164 raft_consensus.cc:2243] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:37.901456 1164 raft_consensus.cc:2272] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:37.902133 1164 ts_tablet_manager.cc:1916] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:37.906265 711 consensus_peers.cc:597] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:37.910220 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 042f8bf13a7a4712b5bdc04671cd9938
I20251211 20:52:37.910748 596 catalog_manager.cc:6724] ReplaceTablet: tablet 042f8bf13a7a4712b5bdc04671cd9938 deleted and replaced by tablet 53b4116db0a44dfbb4c1ec5b2acebc67
I20251211 20:52:37.911104 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_64fa73fbf85e4a40834ce3d6d7d84b11" } new_table_name: "default.Table_ed03ad475bed4804a65abe20efef0625" modify_external_catalogs: true
I20251211 20:52:37.911264 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 042f8bf13a7a4712b5bdc04671cd9938 with delete_type TABLET_DATA_DELETED (replaced by tablet 53b4116db0a44dfbb4c1ec5b2acebc67) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:37.912227 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 042f8bf13a7a4712b5bdc04671cd9938 with delete_type TABLET_DATA_DELETED (replaced by tablet 53b4116db0a44dfbb4c1ec5b2acebc67) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:37.912781 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 14f367adb60744c8a593d3b9d4711823
I20251211 20:52:37.913604 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_72e3939e6805448ba5d1f87e6dad3d3a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:37.913812 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 14f367adb60744c8a593d3b9d4711823 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:37.914132 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_72e3939e6805448ba5d1f87e6dad3d3a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:37.915828 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 14f367adb60744c8a593d3b9d4711823 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:37.917304 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 2197822e0f11484f9679c4858626ac2c
I20251211 20:52:37.917850 591 catalog_manager.cc:6724] ReplaceTablet: tablet 2197822e0f11484f9679c4858626ac2c deleted and replaced by tablet 16d2ff8925e342729a71c93db378d031
I20251211 20:52:37.918210 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_f21e4bd4ff9846c8b37a5723d0fc3c3e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:37.918682 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 2197822e0f11484f9679c4858626ac2c with delete_type TABLET_DATA_DELETED (replaced by tablet 16d2ff8925e342729a71c93db378d031) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:37.918748 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f21e4bd4ff9846c8b37a5723d0fc3c3e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:37.920981 1160 tablet_bootstrap.cc:492] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:37.921109 1160 ts_tablet_manager.cc:1403] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.041s user 0.003s sys 0.006s
I20251211 20:52:37.921487 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 2197822e0f11484f9679c4858626ac2c with delete_type TABLET_DATA_DELETED (replaced by tablet 16d2ff8925e342729a71c93db378d031) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:37.921845 1160 raft_consensus.cc:359] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.922103 1160 raft_consensus.cc:385] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.922155 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 2ef524747034491c99f4f9767acd3c9d
I20251211 20:52:37.922241 1160 raft_consensus.cc:740] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.922976 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_491ce822e76742a1882861a98eff45b5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:37.923350 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ef524747034491c99f4f9767acd3c9d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:37.923628 1160 consensus_queue.cc:260] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.924389 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 2d849c62f7434099a16f62c2a2fc5e85
I20251211 20:52:37.924866 597 catalog_manager.cc:6724] ReplaceTablet: tablet 2d849c62f7434099a16f62c2a2fc5e85 deleted and replaced by tablet 3dfea28c44d84c68850db65a30d52ee6
I20251211 20:52:37.925205 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c0c0556e591f4b1eb2f215439d385a47" } new_table_name: "default.Table_686a26915b42421380037144900dd97c" modify_external_catalogs: true
I20251211 20:52:37.926510 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ef524747034491c99f4f9767acd3c9d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:37.927354 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 2d849c62f7434099a16f62c2a2fc5e85 with delete_type TABLET_DATA_DELETED (replaced by tablet 3dfea28c44d84c68850db65a30d52ee6) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:37.927520 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 2d849c62f7434099a16f62c2a2fc5e85 with delete_type TABLET_DATA_DELETED (replaced by tablet 3dfea28c44d84c68850db65a30d52ee6) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:37.927826 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_491ce822e76742a1882861a98eff45b5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:37.928093 1160 ts_tablet_manager.cc:1434] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20251211 20:52:37.928362 1160 tablet_bootstrap.cc:492] T 8feee01fc0aa4b32be926ff2b617b894 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:37.929591 1163 ts_tablet_manager.cc:1929] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:37.932724 589 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3b2c6064c5eb4c0f8979c485f5d85a75 (table default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f]). Expected version 1 got 0
I20251211 20:52:37.932960 589 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 8feee01fc0aa4b32be926ff2b617b894 (table default.Table_e6f3a819faad48aabfad3a235c3c1b95 [id=18f8d722faaa4293a112525ff0c96216]). Expected version 4 got 3
I20251211 20:52:37.933122 589 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 8b793241ee3b42b1a38e9f2f05e1173b (table default.Table_58802d3fc36742ffb050db7bb7f0c064 [id=279f39d718ba45318f52f95e8aa13652]). Expected version 1 got 0
I20251211 20:52:37.933252 589 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 1843dc4548134f29969d46fbc760f5d7 (table default.Table_99917cb4c70840e19004ad381746d77e [id=6abb187bfd214e91b2e11267065a2841]). Expected version 1 got 0
I20251211 20:52:37.933928 595 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_12f41f708f1949598d6176d89777aa69 [id=3309205faabf4a42acadc43766e4f57f] alter complete (version 1)
I20251211 20:52:37.934207 1160 tablet_bootstrap.cc:654] T 8feee01fc0aa4b32be926ff2b617b894 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.935539 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_264b707b3aba4b1a93ee5b8131af5192" } modify_external_catalogs: true
I20251211 20:52:37.935772 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_264b707b3aba4b1a93ee5b8131af5192" } modify_external_catalogs: true
I20251211 20:52:37.936892 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet a5eeb56709f046d9be92ac600ebc330e
I20251211 20:52:37.937602 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_33ba9c5d5329493ba57d012d9344420d" } new_table_name: "default.Table_da0851457f78469c9a4b783e05c6a58e" modify_external_catalogs: true
I20251211 20:52:37.938210 883 tablet_service.cc:1552] Processing DeleteTablet for tablet a5eeb56709f046d9be92ac600ebc330e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:37.939950 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 3c578d81e8dd41aaa0f707c4e3fed225 with delete_type TABLET_DATA_DELETED (replaced by tablet b115d737d7fc483d9f5a1f040b511bae) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:37.940160 888 tablet_service.cc:1552] Processing DeleteTablet for tablet bc32a778e5424850b23ccaf74bb9df07 with delete_type TABLET_DATA_DELETED (replaced by tablet 6a206abd6ea2420089cb07714e0d19c0) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:37.940526 883 tablet_service.cc:1552] Processing DeleteTablet for tablet d762b4afb5ff447dac3468393826205a with delete_type TABLET_DATA_DELETED (replaced by tablet 2197822e0f11484f9679c4858626ac2c) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:37.943396 751 tablet_service.cc:1552] Processing DeleteTablet for tablet fb030a62796248869d46b682656eaeb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:37.943535 756 tablet_service.cc:1552] Processing DeleteTablet for tablet a5eeb56709f046d9be92ac600ebc330e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:37.943737 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 25454e67437541d9ac2bfab29bad55f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:37.945225 1164 ts_tablet_manager.cc:1929] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20251211 20:52:37.945421 589 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 3b2c6064c5eb4c0f8979c485f5d85a75 (table default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 3b2c6064c5eb4c0f8979c485f5d85a75 consensus state has no leader
I20251211 20:52:37.946911 2077 raft_consensus.cc:493] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:37.947155 2077 raft_consensus.cc:515] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.948787 2077 leader_election.cc:290] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:37.948284 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25454e67437541d9ac2bfab29bad55f9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:37.949631 711 leader_election.cc:343] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:37.949944 1164 log.cc:1199] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/05385f20e88c46108e09a78814811209
I20251211 20:52:37.950834 1164 ts_tablet_manager.cc:1950] T 05385f20e88c46108e09a78814811209 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:37.951707 589 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 9f8aa28c1c9b4fc58dfc128952c180ba (table default.Table_bf7d402a36be44cfb6b79041906d26a1 [id=0327e686e7ed4475a3e8cda8721a3a3a]). Expected version 1 got 0
I20251211 20:52:37.952445 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 05385f20e88c46108e09a78814811209 (table default.Table_2a5ff353268344dba84653a0fd41e4a0 [id=c06795252c09401b99e58c702574ac05]) successfully deleted
I20251211 20:52:37.953352 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d with delete_type TABLET_DATA_DELETED (replaced by tablet e4647105ab57454d883ebfcde7bde891) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:37.953627 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 822abda3c85848728205b71847ef7029 with delete_type TABLET_DATA_DELETED (replaced by tablet f9a3fdf17f274c15ac0bdfa9c174f649) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:37.955925 883 tablet_service.cc:1505] Processing CreateTablet for tablet e89d4d76bf64453892a4d0b55fcc3454 (DEFAULT_TABLE table=default.Table_bcd4dcb9828242a18cc8ca757e2180ba [id=01bba75f854d4c338d6c09a446dd7499]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.956468 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e89d4d76bf64453892a4d0b55fcc3454. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.958221 2077 consensus_queue.cc:1048] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251211 20:52:37.960354 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 042f8bf13a7a4712b5bdc04671cd9938 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 042f8bf13a7a4712b5bdc04671cd9938 already in progress: creating tablet
I20251211 20:52:37.963991 754 tablet_service.cc:1505] Processing CreateTablet for tablet e89d4d76bf64453892a4d0b55fcc3454 (DEFAULT_TABLE table=default.Table_bcd4dcb9828242a18cc8ca757e2180ba [id=01bba75f854d4c338d6c09a446dd7499]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:37.964087 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 14f367adb60744c8a593d3b9d4711823 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14f367adb60744c8a593d3b9d4711823 already in progress: creating tablet
I20251211 20:52:37.964433 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e89d4d76bf64453892a4d0b55fcc3454. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.965086 1160 tablet_bootstrap.cc:492] T 8feee01fc0aa4b32be926ff2b617b894 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:37.965215 1160 ts_tablet_manager.cc:1403] T 8feee01fc0aa4b32be926ff2b617b894 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.037s user 0.002s sys 0.001s
I20251211 20:52:37.966121 1160 raft_consensus.cc:359] T 8feee01fc0aa4b32be926ff2b617b894 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.966284 1160 raft_consensus.cc:385] T 8feee01fc0aa4b32be926ff2b617b894 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.966327 1160 raft_consensus.cc:740] T 8feee01fc0aa4b32be926ff2b617b894 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.966813 1160 consensus_queue.cc:260] T 8feee01fc0aa4b32be926ff2b617b894 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.968178 1160 ts_tablet_manager.cc:1434] T 8feee01fc0aa4b32be926ff2b617b894 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:37.970093 592 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 8feee01fc0aa4b32be926ff2b617b894 (table default.Table_e6f3a819faad48aabfad3a235c3c1b95 [id=18f8d722faaa4293a112525ff0c96216]). Expected version 4 got 0
I20251211 20:52:37.970749 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_d04571b1151e4e89bfbf3148cbcfd269" } new_table_name: "default.Table_7fe80ff5483a4b0292091119980f6639" modify_external_catalogs: true
I20251211 20:52:37.971230 1160 tablet_bootstrap.cc:492] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:37.973106 888 tablet_service.cc:1505] Processing CreateTablet for tablet c97d410272a947de8b5d7fb40fe4777d (DEFAULT_TABLE table=default.Table_c3617a747d9d4e4294df0b575520d39f [id=dd5c06c61d104b689ab9b674620e4fb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.973521 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c97d410272a947de8b5d7fb40fe4777d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.980161 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_d0cd234de1cb4bb485da3d10b527eb65"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:37.980970 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d0cd234de1cb4bb485da3d10b527eb65 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:37.983273 754 tablet_service.cc:1505] Processing CreateTablet for tablet c97d410272a947de8b5d7fb40fe4777d (DEFAULT_TABLE table=default.Table_c3617a747d9d4e4294df0b575520d39f [id=dd5c06c61d104b689ab9b674620e4fb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.983779 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c97d410272a947de8b5d7fb40fe4777d. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:37.988112 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 2197822e0f11484f9679c4858626ac2c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2197822e0f11484f9679c4858626ac2c already in progress: creating tablet
I20251211 20:52:37.989306 1168 tablet_bootstrap.cc:492] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:37.989506 1168 ts_tablet_manager.cc:1403] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.114s user 0.010s sys 0.022s
I20251211 20:52:37.990322 1160 tablet_bootstrap.cc:654] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.990327 1168 raft_consensus.cc:359] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.990587 1168 raft_consensus.cc:385] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:37.990669 1168 raft_consensus.cc:740] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.990890 1168 consensus_queue.cc:260] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:37.991372 1168 ts_tablet_manager.cc:1434] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20251211 20:52:37.991660 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 2ef524747034491c99f4f9767acd3c9d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2ef524747034491c99f4f9767acd3c9d already in progress: creating tablet
I20251211 20:52:37.991850 1168 tablet_bootstrap.cc:492] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:37.992436 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 2d849c62f7434099a16f62c2a2fc5e85 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d849c62f7434099a16f62c2a2fc5e85 already in progress: creating tablet
I20251211 20:52:37.992728 1164 tablet_replica.cc:333] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:37.992990 1164 raft_consensus.cc:2243] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:37.994155 1164 raft_consensus.cc:2272] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:37.994310 1168 tablet_bootstrap.cc:654] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:37.995003 1164 ts_tablet_manager.cc:1916] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:37.996006 888 tablet_service.cc:1505] Processing CreateTablet for tablet d35dd69f888747ec89c7a5f5f32cd409 (DEFAULT_TABLE table=default.Table_89da60cc38f54ac48f12f3b5988d48ad [id=e40a35fac2d7424ca101a4f6b4af3a2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.996209 742 tablet_service.cc:1505] Processing CreateTablet for tablet d35dd69f888747ec89c7a5f5f32cd409 (DEFAULT_TABLE table=default.Table_89da60cc38f54ac48f12f3b5988d48ad [id=e40a35fac2d7424ca101a4f6b4af3a2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.996385 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d35dd69f888747ec89c7a5f5f32cd409. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.996574 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d35dd69f888747ec89c7a5f5f32cd409. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.002575 1696 raft_consensus.cc:493] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:38.002789 1696 raft_consensus.cc:515] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.003504 1696 leader_election.cc:290] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:38.003893 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "52ef1707f6f2425aafd454f303af0333" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:38.004463 711 leader_election.cc:343] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:38.004685 742 tablet_service.cc:1505] Processing CreateTablet for tablet 04491faa6f594e128e51d8c266b33759 (DEFAULT_TABLE table=default.Table_3f8e5876c38a4c5aac82efa6e26de84d [id=f51019a5e161417b8dbca90195fdc761]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.005076 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04491faa6f594e128e51d8c266b33759. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.009526 1164 ts_tablet_manager.cc:1929] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20251211 20:52:38.014581 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet b61c4f0cb15940a988c5acb6879d265a
I20251211 20:52:38.012424 1160 tablet_bootstrap.cc:492] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:38.015764 1160 ts_tablet_manager.cc:1403] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.047s user 0.003s sys 0.000s
I20251211 20:52:38.015760 756 tablet_service.cc:1505] Processing CreateTablet for tablet 41d17a49e22746c0904c8b96626262a6 (DEFAULT_TABLE table=default.Table_b8ccf690baa544a8a6f41b0c6eea6b38 [id=2ee7c64dbda8471ea684bd515d4f2f4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.016173 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41d17a49e22746c0904c8b96626262a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.016242 590 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 8feee01fc0aa4b32be926ff2b617b894 (table default.Table_e6f3a819faad48aabfad3a235c3c1b95 [id=18f8d722faaa4293a112525ff0c96216]). Expected version 4 got 0
I20251211 20:52:38.016515 1160 raft_consensus.cc:359] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.016752 1160 raft_consensus.cc:385] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:38.016814 1160 raft_consensus.cc:740] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:38.017043 1160 consensus_queue.cc:260] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.019089 590 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet c8e17645a26947ab8c01f9b093fa422c (table default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5]). Expected version 2 got 0
I20251211 20:52:38.019747 891 tablet_service.cc:1552] Processing DeleteTablet for tablet b61c4f0cb15940a988c5acb6879d265a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:38.019771 751 tablet_service.cc:1552] Processing DeleteTablet for tablet b61c4f0cb15940a988c5acb6879d265a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:38.020818 883 tablet_service.cc:1505] Processing CreateTablet for tablet 41d17a49e22746c0904c8b96626262a6 (DEFAULT_TABLE table=default.Table_b8ccf690baa544a8a6f41b0c6eea6b38 [id=2ee7c64dbda8471ea684bd515d4f2f4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.021239 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41d17a49e22746c0904c8b96626262a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.021333 914 raft_consensus.cc:3060] T 8feee01fc0aa4b32be926ff2b617b894 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:38.022945 888 tablet_service.cc:1505] Processing CreateTablet for tablet 04491faa6f594e128e51d8c266b33759 (DEFAULT_TABLE table=default.Table_3f8e5876c38a4c5aac82efa6e26de84d [id=f51019a5e161417b8dbca90195fdc761]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.023387 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04491faa6f594e128e51d8c266b33759. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.024771 891 tablet_service.cc:1505] Processing CreateTablet for tablet 46b225ef696a45088a4816d18fa2e500 (DEFAULT_TABLE table=default.Table_b36aed5743884aa7bca0cb01b2af1bac [id=440072d0cbe248c381c8086eb62f1fcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.025269 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46b225ef696a45088a4816d18fa2e500. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.026599 1160 ts_tablet_manager.cc:1434] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.011s user 0.001s sys 0.000s
I20251211 20:52:38.027110 1160 tablet_bootstrap.cc:492] T 3fc173c396ff4af5a8589cff18165e85 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:38.030354 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f94b656e3c1844509fdae36e33b263e8 consensus state has no leader
W20251211 20:52:38.030993 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b52377c8ffd741a598e5328a00d876fe consensus state has no leader
I20251211 20:52:38.036201 1160 tablet_bootstrap.cc:654] T 3fc173c396ff4af5a8589cff18165e85 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:38.038081 891 tablet_service.cc:1505] Processing CreateTablet for tablet 0302cc8613934c43a0650f081cf98797 (DEFAULT_TABLE table=default.Table_bcb5c15cfcc2448a9f814e5ff5665f93 [id=ed371e073250439fa44330c7101e9387]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.038576 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0302cc8613934c43a0650f081cf98797. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:37.957252 983 consensus_peers.cc:597] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3eac21a07ba34e1fa275e25ee9dcabb1. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:38.039188 983 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 81974.5us
I20251211 20:52:38.042757 590 master_service.cc:954] ReplaceTablet: received request to replace tablet 97ada0641165444a998db26796793507 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:37.958853 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25454e67437541d9ac2bfab29bad55f9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:38.044413 1164 log.cc:1199] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/a5eeb56709f046d9be92ac600ebc330e
I20251211 20:52:37.984702 1023 tablet_service.cc:1505] Processing CreateTablet for tablet 9c5c937cc7db4ed18fe03f4a8a99a66a (DEFAULT_TABLE table=default.Table_68b9d0d7258a4b5caa1779a4769bdd7f [id=06f467911b6f45849e99f0b6fe615163]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.045259 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c5c937cc7db4ed18fe03f4a8a99a66a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.047408 742 tablet_service.cc:1505] Processing CreateTablet for tablet 0302cc8613934c43a0650f081cf98797 (DEFAULT_TABLE table=default.Table_bcb5c15cfcc2448a9f814e5ff5665f93 [id=ed371e073250439fa44330c7101e9387]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:37.984773 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 7d4e63f54d8443ae8322d4659a165e76 (DEFAULT_TABLE table=default.Table_35338ef5e24d4a1c9972400654a56d83 [id=ad4931c1fa584e83bae93e064eca9621]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.047843 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0302cc8613934c43a0650f081cf98797. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.048099 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d4e63f54d8443ae8322d4659a165e76. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.049451 754 tablet_service.cc:1505] Processing CreateTablet for tablet 46b225ef696a45088a4816d18fa2e500 (DEFAULT_TABLE table=default.Table_b36aed5743884aa7bca0cb01b2af1bac [id=440072d0cbe248c381c8086eb62f1fcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.049785 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46b225ef696a45088a4816d18fa2e500. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.985245 1025 tablet_service.cc:1505] Processing CreateTablet for tablet 88e4282d370e4df3a7dcab7819dbac45 (DEFAULT_TABLE table=default.Table_95bf875841aa45c8bf54c51b5839eb51 [id=71660a3234e74d639bca2b039ff420c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.051918 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88e4282d370e4df3a7dcab7819dbac45. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.053223 879 tablet_service.cc:1505] Processing CreateTablet for tablet 871406797f6b4961be75eec0fc21e61d (DEFAULT_TABLE table=default.Table_f89cac674abe45038710fc4b675afbc8 [id=687d8fab1ee24655b68058860cdd4355]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.055552 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 871406797f6b4961be75eec0fc21e61d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:37.985381 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet cce44b80c97948e1af7ac68e77cccfd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:37.985431 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet a2058b67ddf1413db338bfa2f64e7a77 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:37.985800 1012 tablet_service.cc:1505] Processing CreateTablet for tablet 745ce63314834c8abdecaa367133a068 (DEFAULT_TABLE table=default.Table_0040a1e388eb4e58a352a1a3884345c4 [id=c2fb490b2d5d44f793304b8939018ca2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.056336 1012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 745ce63314834c8abdecaa367133a068. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.058120 1168 tablet_bootstrap.cc:492] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:38.058247 1168 ts_tablet_manager.cc:1403] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.067s user 0.002s sys 0.003s
I20251211 20:52:38.058949 756 tablet_service.cc:1505] Processing CreateTablet for tablet 871406797f6b4961be75eec0fc21e61d (DEFAULT_TABLE table=default.Table_f89cac674abe45038710fc4b675afbc8 [id=687d8fab1ee24655b68058860cdd4355]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.059136 1168 raft_consensus.cc:359] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.059276 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 871406797f6b4961be75eec0fc21e61d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.059289 1168 raft_consensus.cc:385] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:38.059383 1168 raft_consensus.cc:740] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:37.986078 1019 tablet_service.cc:1552] Processing DeleteTablet for tablet c91ff29847d54e55b19321652a857e9f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:37.986315 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 098054406a1d459b824a094f50e71e7e with delete_type TABLET_DATA_DELETED (replaced by tablet aa50ddb2789b4be3bfa9aedef9287109) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:37.986588 1028 tablet_service.cc:1552] Processing DeleteTablet for tablet 3eac21a07ba34e1fa275e25ee9dcabb1 with delete_type TABLET_DATA_DELETED (replaced by tablet 2085f871ea0a4caebbf2b1459b8c1e3b) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:37.986815 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 042f8bf13a7a4712b5bdc04671cd9938 with delete_type TABLET_DATA_DELETED (replaced by tablet 53b4116db0a44dfbb4c1ec5b2acebc67) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:37.987100 1016 tablet_service.cc:1552] Processing DeleteTablet for tablet 14f367adb60744c8a593d3b9d4711823 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:37.987155 1011 tablet_service.cc:1552] Processing DeleteTablet for tablet 2197822e0f11484f9679c4858626ac2c with delete_type TABLET_DATA_DELETED (replaced by tablet 16d2ff8925e342729a71c93db378d031) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:37.987388 1014 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ef524747034491c99f4f9767acd3c9d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:38.062242 1813 raft_consensus.cc:493] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:38.062382 1813 raft_consensus.cc:515] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.062978 1813 leader_election.cc:290] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:38.064077 847 leader_election.cc:343] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 28ef13be3b84410d9c786c43502a3478
I20251211 20:52:38.068702 1164 ts_tablet_manager.cc:1950] T a5eeb56709f046d9be92ac600ebc330e P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:38.070756 889 tablet_service.cc:1505] Processing CreateTablet for tablet 16fbe528e228447a83e61852f002df60 (DEFAULT_TABLE table=default.Table_9fa0ba597863437e94957c10e3a68b4b [id=24012f9fe7eb4ba4ac585e3dea3b1425]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.071171 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16fbe528e228447a83e61852f002df60. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.004189 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "52ef1707f6f2425aafd454f303af0333" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:38.075009 879 tablet_service.cc:1505] Processing CreateTablet for tablet 2718aa08ba3b4a70a1caa67d0023ec4a (DEFAULT_TABLE table=default.Table_f762d4c1a3c44c63abb1c9cd599ac9ed [id=b1509c25961f4fb1b49866b4410d326d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.075234 881 tablet_service.cc:1505] Processing CreateTablet for tablet b61c4f0cb15940a988c5acb6879d265a (DEFAULT_TABLE table=default.Table_983c3bbea3c54b88bebbe065e4660218 [id=0b757258a3304c1e99d0c1e8fc8749d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.075506 1047 raft_consensus.cc:2468] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:38.075790 885 tablet_service.cc:1505] Processing CreateTablet for tablet 2e4ff869b0104de5b05035ac880f90d5 (DEFAULT_TABLE table=default.Table_5d7c1dcaff1a420c95fb2907b7cce4b8 [id=d3274b7f43b6447f887db4c3c885fe2e]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:38.080292 847 leader_election.cc:343] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 28ef13be3b84410d9c786c43502a3478
I20251211 20:52:38.037624 1166 tablet_bootstrap.cc:492] T e4647105ab57454d883ebfcde7bde891 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
W20251211 20:52:38.038410 2076 log.cc:927] Time spent T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7: Append to log took a long time: real 0.055s user 0.000s sys 0.000s
I20251211 20:52:38.080462 847 leader_election.cc:304] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:38.080698 595 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 334890f3874147608b9e091baf3c355e
I20251211 20:52:38.080788 1166 ts_tablet_manager.cc:1403] T e4647105ab57454d883ebfcde7bde891 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.297s user 0.004s sys 0.004s
I20251211 20:52:38.081105 1813 raft_consensus.cc:2749] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:38.081302 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5bcc94ec9630484cb7a3e8bf6d4f378a consensus state has no leader
I20251211 20:52:38.081553 1166 raft_consensus.cc:359] T e4647105ab57454d883ebfcde7bde891 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.081717 1166 raft_consensus.cc:385] T e4647105ab57454d883ebfcde7bde891 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:38.081758 1166 raft_consensus.cc:740] T e4647105ab57454d883ebfcde7bde891 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
W20251211 20:52:38.081871 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5b3e8ec3faf445ac9b746e65dc4dbfe0 consensus state has no leader
I20251211 20:52:38.081990 1166 consensus_queue.cc:260] T e4647105ab57454d883ebfcde7bde891 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.082203 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_142734af8412498fa19dd25ad1ef2c2e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:38.082796 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_142734af8412498fa19dd25ad1ef2c2e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.072472 751 tablet_service.cc:1505] Processing CreateTablet for tablet 16fbe528e228447a83e61852f002df60 (DEFAULT_TABLE table=default.Table_9fa0ba597863437e94957c10e3a68b4b [id=24012f9fe7eb4ba4ac585e3dea3b1425]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.083324 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16fbe528e228447a83e61852f002df60. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.084044 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_25ddd02b2a6640c4a6898d1796553a8b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:38.084376 1163 log.cc:1199] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/92a71dff4ea8445d8484fc2747471148
W20251211 20:52:38.084528 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_25ddd02b2a6640c4a6898d1796553a8b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.084757 1163 ts_tablet_manager.cc:1950] T 92a71dff4ea8445d8484fc2747471148 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:38.085035 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 727) took 1964 ms. Trace:
I20251211 20:52:38.085101 1163 rpcz_store.cc:276] 1211 20:52:36.120440 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:36.121139 (+ 699us) service_pool.cc:225] Handling call
1211 20:52:37.788139 (+1667000us) ts_tablet_manager.cc:1151] Deleting tablet 92a71dff4ea8445d8484fc2747471148
1211 20:52:37.788150 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:37.806672 (+ 18522us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:37.929476 (+122804us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:38.085021 (+155545us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1666692}
I20251211 20:52:38.085407 1163 tablet_replica.cc:333] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:38.085952 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_dfb3a2820f32459e98c153c4566d3812" } new_table_name: "default.Table_bd9633c55bd242c18aac3a7f2c18a329" modify_external_catalogs: true
W20251211 20:52:38.072551 710 leader_election.cc:343] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:38.087831 710 leader_election.cc:304] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:38.088565 710 leader_election.cc:304] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:38.088665 1165 ts_tablet_manager.cc:1929] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:38.088743 1165 log.cc:1199] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/6a526a6d1bc84cefad2c5d897223292e
I20251211 20:52:38.088860 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_d3590e8f04c34c71b7054d4680e080a6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:38.089337 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d3590e8f04c34c71b7054d4680e080a6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.089437 2086 raft_consensus.cc:2749] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:38.059568 1168 consensus_queue.cc:260] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.089951 2086 raft_consensus.cc:2804] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:38.090039 2086 raft_consensus.cc:493] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:38.090113 2086 raft_consensus.cc:3060] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:38.090175 1168 ts_tablet_manager.cc:1434] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.032s user 0.000s sys 0.002s
I20251211 20:52:38.090453 1168 tablet_bootstrap.cc:492] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:38.090492 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_ed9d95ed154245de80d2cdc5f1eb5bd3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:38.090929 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ed9d95ed154245de80d2cdc5f1eb5bd3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.091625 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 2d849c62f7434099a16f62c2a2fc5e85 with delete_type TABLET_DATA_DELETED (replaced by tablet 3dfea28c44d84c68850db65a30d52ee6) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:38.092010 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet a5eeb56709f046d9be92ac600ebc330e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:38.092299 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet b529b38d754f46ba8cb60b19e9705256 with delete_type TABLET_DATA_DELETED (replaced by tablet 84bb3583857848459e0e865283de623d) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:38.092572 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet e6ba12f94f5f4d688f4222a4c2fb5436 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:38.092664 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_109eac21e7ee45e9a1ad808c49b10998"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:38.092824 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 14e2650f8ec1482ea9511b989fdc963c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:38.093145 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_109eac21e7ee45e9a1ad808c49b10998 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.093276 1027 tablet_service.cc:1505] Processing CreateTablet for tablet e89d4d76bf64453892a4d0b55fcc3454 (DEFAULT_TABLE table=default.Table_bcd4dcb9828242a18cc8ca757e2180ba [id=01bba75f854d4c338d6c09a446dd7499]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.093639 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e89d4d76bf64453892a4d0b55fcc3454. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.093848 2086 raft_consensus.cc:515] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.094229 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_7684c7e006504ea0a1ef81dc2eecaa8d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:38.094691 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7684c7e006504ea0a1ef81dc2eecaa8d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.095364 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25454e67437541d9ac2bfab29bad55f9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:38.095880 711 leader_election.cc:343] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:38.095561 2086 leader_election.cc:290] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:38.072870 756 tablet_service.cc:1505] Processing CreateTablet for tablet 2718aa08ba3b4a70a1caa67d0023ec4a (DEFAULT_TABLE table=default.Table_f762d4c1a3c44c63abb1c9cd599ac9ed [id=b1509c25961f4fb1b49866b4410d326d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.096707 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2718aa08ba3b4a70a1caa67d0023ec4a. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:38.097743 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8675e2a18f0f4a0e95a87005cf82d8b4 consensus state has no leader
I20251211 20:52:38.076246 757 tablet_service.cc:1505] Processing CreateTablet for tablet 2e4ff869b0104de5b05035ac880f90d5 (DEFAULT_TABLE table=default.Table_5d7c1dcaff1a420c95fb2907b7cce4b8 [id=d3274b7f43b6447f887db4c3c885fe2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.098470 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e4ff869b0104de5b05035ac880f90d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.103396 1166 ts_tablet_manager.cc:1434] T e4647105ab57454d883ebfcde7bde891 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.023s user 0.001s sys 0.001s
I20251211 20:52:38.076821 746 tablet_service.cc:1505] Processing CreateTablet for tablet b61c4f0cb15940a988c5acb6879d265a (DEFAULT_TABLE table=default.Table_983c3bbea3c54b88bebbe065e4660218 [id=0b757258a3304c1e99d0c1e8fc8749d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.103739 1166 tablet_bootstrap.cc:492] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:38.103956 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b61c4f0cb15940a988c5acb6879d265a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.105024 1166 tablet_bootstrap.cc:654] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:38.106706 754 tablet_service.cc:1505] Processing CreateTablet for tablet f1a765c46fc34ce2899b45ae3fa1d9a2 (DEFAULT_TABLE table=default.Table_e9b10d4bc5db49289ea753470f1585a6 [id=26d30d9b3c3d46fd9fbffa0f0124a63e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.107223 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1a765c46fc34ce2899b45ae3fa1d9a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.071928 2077 raft_consensus.cc:493] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:38.108028 2077 raft_consensus.cc:515] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.108453 888 tablet_service.cc:1505] Processing CreateTablet for tablet f1a765c46fc34ce2899b45ae3fa1d9a2 (DEFAULT_TABLE table=default.Table_e9b10d4bc5db49289ea753470f1585a6 [id=26d30d9b3c3d46fd9fbffa0f0124a63e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.108595 2077 leader_election.cc:290] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:38.074748 1696 raft_consensus.cc:493] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:38.109177 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4201da4241aa48dfb670a713e4453ca5" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:38.109228 1696 raft_consensus.cc:515] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:38.109694 711 leader_election.cc:343] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:38.110399 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2dde7f28e61d41f48015ba6e4f3757f2" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:38.110469 1696 leader_election.cc:290] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:38.112888 711 leader_election.cc:343] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:38.109089 751 tablet_service.cc:1505] Processing CreateTablet for tablet 243d0d396aca438a997e26246ca1ea21 (DEFAULT_TABLE table=default.Table_1f11bf68fb18474c9a3cce63119e7e5f [id=713c1b5f22c24b918617f807b319b1c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.114555 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 243d0d396aca438a997e26246ca1ea21. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.115286 589 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 9f8aa28c1c9b4fc58dfc128952c180ba (table default.Table_bf7d402a36be44cfb6b79041906d26a1 [id=0327e686e7ed4475a3e8cda8721a3a3a]). Expected version 1 got 0
I20251211 20:52:38.116966 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_6310a6161f1f4def90affd0ea6fc67d2" } modify_external_catalogs: true
I20251211 20:52:38.117172 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_6310a6161f1f4def90affd0ea6fc67d2" } modify_external_catalogs: true
I20251211 20:52:38.118294 757 tablet_service.cc:1505] Processing CreateTablet for tablet d15eff49af284a38bb1babdb231a35df (DEFAULT_TABLE table=default.Table_df5fb8b06973455d8c2b6a335e1f1d95 [id=a32b6267d2f949e4bcea7f499e5df00d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.118716 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d15eff49af284a38bb1babdb231a35df. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.120666 891 tablet_service.cc:1505] Processing CreateTablet for tablet d15eff49af284a38bb1babdb231a35df (DEFAULT_TABLE table=default.Table_df5fb8b06973455d8c2b6a335e1f1d95 [id=a32b6267d2f949e4bcea7f499e5df00d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.123700 590 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet e4647105ab57454d883ebfcde7bde891 (table default.Table_c817dec9a7674f699b3366fc302ead57 [id=d555d1fc6aef4a21b4d3f2804264d030]). Expected version 1 got 0
I20251211 20:52:38.129129 595 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 9f8aa28c1c9b4fc58dfc128952c180ba (table default.Table_bf7d402a36be44cfb6b79041906d26a1 [id=0327e686e7ed4475a3e8cda8721a3a3a]). Expected version 1 got 0
I20251211 20:52:38.130437 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_63e0e08eb63c40b9aece6be45b6d4944" } modify_external_catalogs: true
I20251211 20:52:38.130710 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_63e0e08eb63c40b9aece6be45b6d4944" } modify_external_catalogs: true
I20251211 20:52:38.132488 757 tablet_service.cc:1505] Processing CreateTablet for tablet f1630b5d35284a998934a30f86b6fcb7 (DEFAULT_TABLE table=default.Table_2eeb4b33f8424175a9e15a54e38099a2 [id=1b0b021139bc4dc4b08a90403f2d58d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.132745 1163 raft_consensus.cc:2243] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:38.132792 877 tablet_service.cc:1505] Processing CreateTablet for tablet f1630b5d35284a998934a30f86b6fcb7 (DEFAULT_TABLE table=default.Table_2eeb4b33f8424175a9e15a54e38099a2 [id=1b0b021139bc4dc4b08a90403f2d58d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.132968 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1630b5d35284a998934a30f86b6fcb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.133025 1163 raft_consensus.cc:2272] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:38.133565 1163 ts_tablet_manager.cc:1916] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:38.137146 886 tablet_service.cc:1505] Processing CreateTablet for tablet 9563684e423d41eda6ff7f9985a7b1c1 (DEFAULT_TABLE table=default.Table_91c76a3c00864c3fa3d653b3f6e0f64f [id=d991ee6b2aa54096a3b1676618b6cb22]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.139436 1163 ts_tablet_manager.cc:1929] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:38.140388 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_d3590e8f04c34c71b7054d4680e080a6" } modify_external_catalogs: true
I20251211 20:52:38.140609 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_d3590e8f04c34c71b7054d4680e080a6" } modify_external_catalogs: true
I20251211 20:52:38.141433 754 tablet_service.cc:1505] Processing CreateTablet for tablet 9563684e423d41eda6ff7f9985a7b1c1 (DEFAULT_TABLE table=default.Table_91c76a3c00864c3fa3d653b3f6e0f64f [id=d991ee6b2aa54096a3b1676618b6cb22]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.141909 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9563684e423d41eda6ff7f9985a7b1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.148921 884 tablet_service.cc:1505] Processing CreateTablet for tablet 52b1bf7f19584d5297b56e5aacb98560 (DEFAULT_TABLE table=default.Table_b929f7410fd44316b3a3571d02066b45 [id=e9dcc3d777524e70b9653ec6a2df7da0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.150725 746 tablet_service.cc:1505] Processing CreateTablet for tablet 52b1bf7f19584d5297b56e5aacb98560 (DEFAULT_TABLE table=default.Table_b929f7410fd44316b3a3571d02066b45 [id=e9dcc3d777524e70b9653ec6a2df7da0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.151218 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52b1bf7f19584d5297b56e5aacb98560. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.156070 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 87863daf77304a399318cdf56da9f9bb (table default.Table_28b13655ab2b47e68f2bd01d618a2b58 [id=9b92dd859a6641b8ade2b49515d1e810]) successfully deleted
I20251211 20:52:38.157245 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 92a71dff4ea8445d8484fc2747471148 (table default.Table_c7e035d673434c588da10ea5a480f541 [id=6a07ca569d504b8292feabcab2e3f70c]) successfully deleted
W20251211 20:52:38.159307 710 leader_election.cc:343] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 4201da4241aa48dfb670a713e4453ca5
I20251211 20:52:38.159749 710 leader_election.cc:304] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:38.160295 2077 raft_consensus.cc:2749] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:38.161392 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet ccc3ddd4b7f74f11a3835997cb7ae8d0
I20251211 20:52:38.162174 880 tablet_service.cc:1552] Processing DeleteTablet for tablet ccc3ddd4b7f74f11a3835997cb7ae8d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:38.163093 742 tablet_service.cc:1552] Processing DeleteTablet for tablet ccc3ddd4b7f74f11a3835997cb7ae8d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:38.165799 595 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 43bc633d320b491fa91cda46fae6cf5d
I20251211 20:52:38.166108 880 tablet_service.cc:1505] Processing CreateTablet for tablet fdb2b65b027f4abf9b55080c95ab27ae (DEFAULT_TABLE table=default.Table_cb402f83e7b24e308e46a798daa7db02 [id=d7bc4f722b6943438cffc3f0a7e5015a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.168071 883 tablet_service.cc:1505] Processing CreateTablet for tablet 243d0d396aca438a997e26246ca1ea21 (DEFAULT_TABLE table=default.Table_1f11bf68fb18474c9a3cce63119e7e5f [id=713c1b5f22c24b918617f807b319b1c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.168907 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet a471a9accbfc4875bd1d7802f5620808
I20251211 20:52:38.168885 890 tablet_service.cc:1505] Processing CreateTablet for tablet 0beff1cfdb1e49908ee4e6756b8b7e65 (DEFAULT_TABLE table=default.Table_97f780d6dd39409dbb5e5ff7e4fd61fd [id=adef6ee1143c43e495691dc186c25241]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.169531 894 tablet_service.cc:1552] Processing DeleteTablet for tablet a471a9accbfc4875bd1d7802f5620808 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:38.169674 1168 tablet_bootstrap.cc:654] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:38.170111 746 tablet_service.cc:1505] Processing CreateTablet for tablet fdb2b65b027f4abf9b55080c95ab27ae (DEFAULT_TABLE table=default.Table_cb402f83e7b24e308e46a798daa7db02 [id=d7bc4f722b6943438cffc3f0a7e5015a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.170212 742 tablet_service.cc:1552] Processing DeleteTablet for tablet a471a9accbfc4875bd1d7802f5620808 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:38.170485 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdb2b65b027f4abf9b55080c95ab27ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.172857 742 tablet_service.cc:1505] Processing CreateTablet for tablet 0beff1cfdb1e49908ee4e6756b8b7e65 (DEFAULT_TABLE table=default.Table_97f780d6dd39409dbb5e5ff7e4fd61fd [id=adef6ee1143c43e495691dc186c25241]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.173511 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0beff1cfdb1e49908ee4e6756b8b7e65. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.174947 1165 ts_tablet_manager.cc:1950] T 6a526a6d1bc84cefad2c5d897223292e P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:38.176090 1165 tablet_replica.cc:333] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:38.176252 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 6a526a6d1bc84cefad2c5d897223292e (table default.Table_254d66657cb14d0f8bbf713f794806f6 [id=f96a54c77528480ebf36ecc5741f7e17]) successfully deleted
I20251211 20:52:38.183189 1165 raft_consensus.cc:2243] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:38.180457 2168 tablet.cc:1722] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20251211 20:52:38.184288 1165 raft_consensus.cc:2272] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:38.185161 1165 ts_tablet_manager.cc:1916] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:38.185118 889 tablet_service.cc:1505] Processing CreateTablet for tablet 6212b3b4425543168837946a54d60834 (DEFAULT_TABLE table=default.Table_4abd8d4223574788846f1682e0e522ff [id=2d489cd7d73e403d84bf10960e6a2dfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.192024 2086 raft_consensus.cc:493] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:38.192188 2086 raft_consensus.cc:515] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.192768 2086 leader_election.cc:290] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:38.194375 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb030a62796248869d46b682656eaeb0" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:38.195529 711 leader_election.cc:343] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:38.198207 747 tablet_service.cc:1505] Processing CreateTablet for tablet 6212b3b4425543168837946a54d60834 (DEFAULT_TABLE table=default.Table_4abd8d4223574788846f1682e0e522ff [id=2d489cd7d73e403d84bf10960e6a2dfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.200057 1168 tablet_bootstrap.cc:492] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:38.200241 1168 ts_tablet_manager.cc:1403] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.110s user 0.002s sys 0.007s
I20251211 20:52:38.200990 1168 raft_consensus.cc:359] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.201192 1168 raft_consensus.cc:385] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:38.201258 1168 raft_consensus.cc:740] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:38.201458 1168 consensus_queue.cc:260] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.202024 1168 ts_tablet_manager.cc:1434] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251211 20:52:38.204774 894 tablet_service.cc:1505] Processing CreateTablet for tablet 5b3e8ec3faf445ac9b746e65dc4dbfe0 (DEFAULT_TABLE table=default.Table_7fe80ff5483a4b0292091119980f6639 [id=3c3462a0c09748cea0d2ffb544ae2f13]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.205026 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6212b3b4425543168837946a54d60834. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.205798 1168 tablet_bootstrap.cc:492] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:38.207311 1168 tablet_bootstrap.cc:654] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:38.209522 893 tablet_service.cc:1505] Processing CreateTablet for tablet 6888d2a2c6574c93ae0475faf07609f2 (DEFAULT_TABLE table=default.Table_bd71cd88b0354c88aa03799a4362f1b1 [id=2f4a7d71bf18442fa8f3ed7421512ff2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.211740 2093 raft_consensus.cc:493] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:38.211863 2093 raft_consensus.cc:515] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.212380 2093 leader_election.cc:290] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:38.212965 746 tablet_service.cc:1505] Processing CreateTablet for tablet 6888d2a2c6574c93ae0475faf07609f2 (DEFAULT_TABLE table=default.Table_bd71cd88b0354c88aa03799a4362f1b1 [id=2f4a7d71bf18442fa8f3ed7421512ff2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.213320 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6888d2a2c6574c93ae0475faf07609f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.215152 878 tablet_service.cc:1505] Processing CreateTablet for tablet 9db471247b674e34acf4239a8b55e1d6 (DEFAULT_TABLE table=default.Table_7e3bb905c8ec4e96809e6b7d4e44a3b7 [id=8212122882be451f9b5f5c6880bf3dce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.216224 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb112b6431af42a6a7b328dfdefbc2a2" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:38.211534 589 catalog_manager.cc:5654] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:38.218806 742 tablet_service.cc:1505] Processing CreateTablet for tablet 9db471247b674e34acf4239a8b55e1d6 (DEFAULT_TABLE table=default.Table_7e3bb905c8ec4e96809e6b7d4e44a3b7 [id=8212122882be451f9b5f5c6880bf3dce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.219645 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9db471247b674e34acf4239a8b55e1d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.219849 754 tablet_service.cc:1505] Processing CreateTablet for tablet 5b3e8ec3faf445ac9b746e65dc4dbfe0 (DEFAULT_TABLE table=default.Table_7fe80ff5483a4b0292091119980f6639 [id=3c3462a0c09748cea0d2ffb544ae2f13]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:38.219976 711 leader_election.cc:343] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:38.220173 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b3e8ec3faf445ac9b746e65dc4dbfe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.219722 589 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 9f8aa28c1c9b4fc58dfc128952c180ba (table default.Table_bf7d402a36be44cfb6b79041906d26a1 [id=0327e686e7ed4475a3e8cda8721a3a3a]). Expected version 1 got 0
I20251211 20:52:38.221124 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0040a1e388eb4e58a352a1a3884345c4" } modify_external_catalogs: true
I20251211 20:52:38.221308 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0040a1e388eb4e58a352a1a3884345c4" } modify_external_catalogs: true
I20251211 20:52:38.223371 875 tablet_service.cc:1505] Processing CreateTablet for tablet 757ec913dff840f992a41a38b05f8d41 (DEFAULT_TABLE table=default.Table_a05c8b486c1940c29db8cbb1ea1dfb17 [id=9e0f07ff85c4413eb6efb4223190223c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.227777 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 745ce63314834c8abdecaa367133a068
I20251211 20:52:38.228729 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 745ce63314834c8abdecaa367133a068 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:38.229584 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 745ce63314834c8abdecaa367133a068 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:38.232203 2093 raft_consensus.cc:493] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:38.232327 2093 raft_consensus.cc:515] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.232378 595 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_3fda0e3909b04d63817a6851c0354909" } new_table_name: "default.Table_34bd5155aa3b4ef7818022ba9346c074" modify_external_catalogs: true
I20251211 20:52:38.232836 2093 leader_election.cc:290] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:38.233233 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_52e650ce0b824a41a39fcd4f1b8ba455"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:38.233768 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_52e650ce0b824a41a39fcd4f1b8ba455 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.140149 1023 tablet_service.cc:1505] Processing CreateTablet for tablet c97d410272a947de8b5d7fb40fe4777d (DEFAULT_TABLE table=default.Table_c3617a747d9d4e4294df0b575520d39f [id=dd5c06c61d104b689ab9b674620e4fb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.235132 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c97d410272a947de8b5d7fb40fe4777d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.235085 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_3357d689c6fa4890bcaa97d99990c9fb" } new_table_name: "default.Table_18b21232dfa74378b27defe672bb2574" modify_external_catalogs: true
I20251211 20:52:38.236351 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f807596fa6a4e66b89566b5d5dd1080" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:38.236099 747 tablet_service.cc:1505] Processing CreateTablet for tablet 757ec913dff840f992a41a38b05f8d41 (DEFAULT_TABLE table=default.Table_a05c8b486c1940c29db8cbb1ea1dfb17 [id=9e0f07ff85c4413eb6efb4223190223c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.236860 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_d5a441b1f9d049169084913c94813ab3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:38.237390 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d5a441b1f9d049169084913c94813ab3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:38.237964 711 leader_election.cc:343] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:38.239105 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 757ec913dff840f992a41a38b05f8d41. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.150692 1012 tablet_service.cc:1505] Processing CreateTablet for tablet d35dd69f888747ec89c7a5f5f32cd409 (DEFAULT_TABLE table=default.Table_89da60cc38f54ac48f12f3b5988d48ad [id=e40a35fac2d7424ca101a4f6b4af3a2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.239997 1012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d35dd69f888747ec89c7a5f5f32cd409. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.240713 887 tablet_service.cc:1505] Processing CreateTablet for tablet 516f7181aaac48cdadf9fbc08d0d3f57 (DEFAULT_TABLE table=default.Table_87f092378445464ab48c155fae469f29 [id=0f029363985443529615cdd1f8bbd78d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.151069 1014 tablet_service.cc:1505] Processing CreateTablet for tablet 04491faa6f594e128e51d8c266b33759 (DEFAULT_TABLE table=default.Table_3f8e5876c38a4c5aac82efa6e26de84d [id=f51019a5e161417b8dbca90195fdc761]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.229749 589 catalog_manager.cc:5654] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:38.244012 1014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04491faa6f594e128e51d8c266b33759. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.151261 1011 tablet_service.cc:1552] Processing DeleteTablet for tablet b61c4f0cb15940a988c5acb6879d265a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:38.151549 1016 tablet_service.cc:1505] Processing CreateTablet for tablet 41d17a49e22746c0904c8b96626262a6 (DEFAULT_TABLE table=default.Table_b8ccf690baa544a8a6f41b0c6eea6b38 [id=2ee7c64dbda8471ea684bd515d4f2f4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.248169 1016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41d17a49e22746c0904c8b96626262a6. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:38.249136 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 497c10742f01476db885bd70c4d2c491 consensus state has no leader
I20251211 20:52:38.152351 1026 tablet_service.cc:1505] Processing CreateTablet for tablet 46b225ef696a45088a4816d18fa2e500 (DEFAULT_TABLE table=default.Table_b36aed5743884aa7bca0cb01b2af1bac [id=440072d0cbe248c381c8086eb62f1fcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.251518 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46b225ef696a45088a4816d18fa2e500. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.152554 1028 tablet_service.cc:1552] Processing DeleteTablet for tablet 4f807596fa6a4e66b89566b5d5dd1080 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:38.251758 595 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e3f711f02af843adb4f5c3c750924e33 consensus state has no leader
I20251211 20:52:38.152853 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 0302cc8613934c43a0650f081cf98797 (DEFAULT_TABLE table=default.Table_bcb5c15cfcc2448a9f814e5ff5665f93 [id=ed371e073250439fa44330c7101e9387]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.153156 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 871406797f6b4961be75eec0fc21e61d (DEFAULT_TABLE table=default.Table_f89cac674abe45038710fc4b675afbc8 [id=687d8fab1ee24655b68058860cdd4355]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.153427 1013 tablet_service.cc:1505] Processing CreateTablet for tablet 2718aa08ba3b4a70a1caa67d0023ec4a (DEFAULT_TABLE table=default.Table_f762d4c1a3c44c63abb1c9cd599ac9ed [id=b1509c25961f4fb1b49866b4410d326d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.153735 1024 tablet_service.cc:1505] Processing CreateTablet for tablet 16fbe528e228447a83e61852f002df60 (DEFAULT_TABLE table=default.Table_9fa0ba597863437e94957c10e3a68b4b [id=24012f9fe7eb4ba4ac585e3dea3b1425]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.154042 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 2e4ff869b0104de5b05035ac880f90d5 (DEFAULT_TABLE table=default.Table_5d7c1dcaff1a420c95fb2907b7cce4b8 [id=d3274b7f43b6447f887db4c3c885fe2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.154393 1029 tablet_service.cc:1505] Processing CreateTablet for tablet b61c4f0cb15940a988c5acb6879d265a (DEFAULT_TABLE table=default.Table_983c3bbea3c54b88bebbe065e4660218 [id=0b757258a3304c1e99d0c1e8fc8749d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.154459 2100 tablet.cc:1722] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20251211 20:52:38.154706 1018 tablet_service.cc:1505] Processing CreateTablet for tablet f1a765c46fc34ce2899b45ae3fa1d9a2 (DEFAULT_TABLE table=default.Table_e9b10d4bc5db49289ea753470f1585a6 [id=26d30d9b3c3d46fd9fbffa0f0124a63e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.253916 589 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 8229b704e7464b30b7d2902a938ce33b (table default.Table_b008891c4e894b02aa0db32d1740371a [id=8fe47ac33eb2461784b859e64fbd3e24]). Expected version 5 got 0
I20251211 20:52:38.257009 757 tablet_service.cc:1505] Processing CreateTablet for tablet 516f7181aaac48cdadf9fbc08d0d3f57 (DEFAULT_TABLE table=default.Table_87f092378445464ab48c155fae469f29 [id=0f029363985443529615cdd1f8bbd78d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.257413 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 516f7181aaac48cdadf9fbc08d0d3f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.257664 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 871406797f6b4961be75eec0fc21e61d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.258131 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet a5eeb56709f046d9be92ac600ebc330e (table default.Table_279bb02bdbd743159babc9a4e6498437 [id=f78e1da479af4d23a552af1c4fc55f81]) successfully deleted
I20251211 20:52:38.258963 1164 tablet_replica.cc:333] T 3c578d81e8dd41aaa0f707c4e3fed225 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:38.259959 1164 raft_consensus.cc:2243] T 3c578d81e8dd41aaa0f707c4e3fed225 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:38.260893 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16fbe528e228447a83e61852f002df60. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.260725 1164 raft_consensus.cc:2272] T 3c578d81e8dd41aaa0f707c4e3fed225 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:38.263692 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1a765c46fc34ce2899b45ae3fa1d9a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.263934 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 243d0d396aca438a997e26246ca1ea21. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.264434 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52b1bf7f19584d5297b56e5aacb98560. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.264662 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6888d2a2c6574c93ae0475faf07609f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.264882 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b61c4f0cb15940a988c5acb6879d265a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.266706 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b61c4f0cb15940a988c5acb6879d265a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.267696 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e4ff869b0104de5b05035ac880f90d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.267867 875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 757ec913dff840f992a41a38b05f8d41. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.270008 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdb2b65b027f4abf9b55080c95ab27ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.270215 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2718aa08ba3b4a70a1caa67d0023ec4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.270972 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b3e8ec3faf445ac9b746e65dc4dbfe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.271803 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 516f7181aaac48cdadf9fbc08d0d3f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.273350 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e4ff869b0104de5b05035ac880f90d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.273648 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0beff1cfdb1e49908ee4e6756b8b7e65. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.275682 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9db471247b674e34acf4239a8b55e1d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.275952 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9563684e423d41eda6ff7f9985a7b1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.276016 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0302cc8613934c43a0650f081cf98797. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.279891 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1630b5d35284a998934a30f86b6fcb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.280140 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1a765c46fc34ce2899b45ae3fa1d9a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.284627 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2718aa08ba3b4a70a1caa67d0023ec4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.288527 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6212b3b4425543168837946a54d60834. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.262106 589 catalog_manager.cc:5654] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:38.298166 589 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 1a77e3e744ff42ccb68897704b4c2f1d (table default.Table_d70f80161dcd4063815a0e079c5821de [id=85ab3a18cda143e5a7f72e49e6671318]). Expected version 1 got 0
I20251211 20:52:38.298930 589 catalog_manager.cc:5654] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:38.299495 589 catalog_manager.cc:5654] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:38.300047 589 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet c7a2610f20df4c1ca81eae2fc7dba6bb (table default.Table_59072235d93b41c0b5cd5e3debcb8ddf [id=9a15a8a075af4969ae2c8b3be1bfed78]). Expected version 1 got 0
I20251211 20:52:38.302397 911 raft_consensus.cc:3060] T 40722aef4e194ff094d271a079bda9d9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:38.302615 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d15eff49af284a38bb1babdb231a35df. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.307803 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_9b8d474fab284933871f1ed0993d1d61"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:38.308357 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9b8d474fab284933871f1ed0993d1d61 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.309547 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_2a8eb0e48b1e42d59c215333429cac46" } new_table_name: "default.Table_5e5286ea174048f8ba9abd14daf14bcf" modify_external_catalogs: true
I20251211 20:52:38.314020 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb112b6431af42a6a7b328dfdefbc2a2" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:38.314365 1047 raft_consensus.cc:2468] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:38.314679 740 tablet_service.cc:1505] Processing CreateTablet for tablet affe191cba57401f9c58211f2d06b80d (DEFAULT_TABLE table=default.Table_7730b7f512954bcab4c83c9f272d9a59 [id=2783ef72c8944e8188a8646565a6df13]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.315095 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet affe191cba57401f9c58211f2d06b80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.315647 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_6063375588754775b43c219ddee022de" } new_table_name: "default.Table_6e9f3ca164384cc8ac4501deb3f0c6aa" modify_external_catalogs: true
I20251211 20:52:38.315734 1813 raft_consensus.cc:493] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:38.315876 1813 raft_consensus.cc:515] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.316428 1813 leader_election.cc:290] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:38.316988 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_08f40dd56ad24729abbd776dabf07b8f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:38.317505 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_08f40dd56ad24729abbd776dabf07b8f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:38.317605 847 leader_election.cc:343] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 6dfa7480e2ab49b9b8a938da0dc6cf0d
W20251211 20:52:38.318383 847 leader_election.cc:343] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 6dfa7480e2ab49b9b8a938da0dc6cf0d
I20251211 20:52:38.318485 847 leader_election.cc:304] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:38.318696 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f516a45c0526407c96ee48db2945eb80" } new_table_name: "default.Table_f196fe07bc5d407d895f5d36dd956786" modify_external_catalogs: true
I20251211 20:52:38.318961 1813 raft_consensus.cc:2749] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:38.319948 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b7a123d140e64879b3fb2606998af217"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:38.320669 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b7a123d140e64879b3fb2606998af217 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.321468 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_d5a441b1f9d049169084913c94813ab3" } modify_external_catalogs: true
I20251211 20:52:38.321671 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_d5a441b1f9d049169084913c94813ab3" } modify_external_catalogs: true
I20251211 20:52:38.154906 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2dde7f28e61d41f48015ba6e4f3757f2" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:38.323894 1042 raft_consensus.cc:2468] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:38.155037 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 243d0d396aca438a997e26246ca1ea21 (DEFAULT_TABLE table=default.Table_1f11bf68fb18474c9a3cce63119e7e5f [id=713c1b5f22c24b918617f807b319b1c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.324563 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 243d0d396aca438a997e26246ca1ea21. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.326977 883 tablet_service.cc:1505] Processing CreateTablet for tablet affe191cba57401f9c58211f2d06b80d (DEFAULT_TABLE table=default.Table_7730b7f512954bcab4c83c9f272d9a59 [id=2783ef72c8944e8188a8646565a6df13]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.155444 1021 tablet_service.cc:1505] Processing CreateTablet for tablet f1630b5d35284a998934a30f86b6fcb7 (DEFAULT_TABLE table=default.Table_2eeb4b33f8424175a9e15a54e38099a2 [id=1b0b021139bc4dc4b08a90403f2d58d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.328012 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1630b5d35284a998934a30f86b6fcb7. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:38.329015 711 consensus_peers.cc:597] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:38.329540 1043 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f807596fa6a4e66b89566b5d5dd1080" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:38.329717 1043 raft_consensus.cc:2468] T 4f807596fa6a4e66b89566b5d5dd1080 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:38.330186 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet affe191cba57401f9c58211f2d06b80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.155557 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 9563684e423d41eda6ff7f9985a7b1c1 (DEFAULT_TABLE table=default.Table_91c76a3c00864c3fa3d653b3f6e0f64f [id=d991ee6b2aa54096a3b1676618b6cb22]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.331979 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9563684e423d41eda6ff7f9985a7b1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.333439 886 tablet_service.cc:1505] Processing CreateTablet for tablet 2d06c480594a44948fe004f73857f3bd (DEFAULT_TABLE table=default.Table_6cce646a13f6405c96fd61f193add5fa [id=1ea52ab9ffc74cb18c6e24e4f4d1cf88]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:38.334880 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3fc173c396ff4af5a8589cff18165e85 consensus state has no leader
I20251211 20:52:38.336464 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_ec0bf98cd1ce42ceacacde30208d5d04"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:38.337042 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ec0bf98cd1ce42ceacacde30208d5d04 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.338593 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_cbecf49ce5e044049547838a5d6ce009"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:38.338682 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d06c480594a44948fe004f73857f3bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.157802 2129 raft_consensus.cc:493] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251211 20:52:38.339172 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cbecf49ce5e044049547838a5d6ce009 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.339107 2129 raft_consensus.cc:515] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.339803 2129 leader_election.cc:290] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:38.340413 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2197822e0f11484f9679c4858626ac2c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:38.340899 983 leader_election.cc:343] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:38.344305 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet df457783b2534b3ea23bb1fd529ce268 consensus state has no leader
I20251211 20:52:38.345376 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2197822e0f11484f9679c4858626ac2c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:38.345618 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_f2301327ddd04d278201a1cb0da3afe8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:38.345803 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b6ff995fa24746b8a8d1cfe5a656893f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:38.346217 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f2301327ddd04d278201a1cb0da3afe8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:38.346357 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b6ff995fa24746b8a8d1cfe5a656893f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.155028 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "25454e67437541d9ac2bfab29bad55f9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:38.182749 1166 tablet_bootstrap.cc:492] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:38.347859 1166 ts_tablet_manager.cc:1403] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.244s user 0.003s sys 0.000s
W20251211 20:52:38.347960 982 leader_election.cc:343] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:38.348067 982 leader_election.cc:304] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:38.348554 2129 raft_consensus.cc:2749] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:38.348680 1166 raft_consensus.cc:359] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.155400 1015 tablet_service.cc:1505] Processing CreateTablet for tablet d15eff49af284a38bb1babdb231a35df (DEFAULT_TABLE table=default.Table_df5fb8b06973455d8c2b6a335e1f1d95 [id=a32b6267d2f949e4bcea7f499e5df00d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.348882 1166 raft_consensus.cc:385] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:38.348927 1166 raft_consensus.cc:740] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:38.349170 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d15eff49af284a38bb1babdb231a35df. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.349128 1166 consensus_queue.cc:260] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.349684 1166 ts_tablet_manager.cc:1434] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:38.349946 1166 tablet_bootstrap.cc:492] T 676faec7f3d1488f99b220f8e968e9a0 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:38.351353 1166 tablet_bootstrap.cc:654] T 676faec7f3d1488f99b220f8e968e9a0 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:38.224758 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb030a62796248869d46b682656eaeb0" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:38.352465 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_fd043ec094f94011986fc8f5a9f32f1a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:38.353060 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fd043ec094f94011986fc8f5a9f32f1a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.356244 1045 raft_consensus.cc:2468] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:38.356429 1168 tablet_bootstrap.cc:492] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:38.356642 1168 ts_tablet_manager.cc:1403] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.151s user 0.003s sys 0.000s
I20251211 20:52:38.139566 1163 log.cc:1199] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/25454e67437541d9ac2bfab29bad55f9
I20251211 20:52:38.357678 1168 raft_consensus.cc:359] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.357848 1168 raft_consensus.cc:385] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:38.357900 1168 raft_consensus.cc:740] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:38.358074 1168 consensus_queue.cc:260] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.358546 1168 ts_tablet_manager.cc:1434] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:38.358809 1168 tablet_bootstrap.cc:492] T f2d6c9781b4d4c37bc08703347963d5d P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:38.359130 1813 raft_consensus.cc:493] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:38.359256 1813 raft_consensus.cc:515] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.359275 746 tablet_service.cc:1505] Processing CreateTablet for tablet 2d06c480594a44948fe004f73857f3bd (DEFAULT_TABLE table=default.Table_6cce646a13f6405c96fd61f193add5fa [id=1ea52ab9ffc74cb18c6e24e4f4d1cf88]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.359745 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d06c480594a44948fe004f73857f3bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.359869 1813 leader_election.cc:290] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:38.361119 847 leader_election.cc:343] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: bc32a778e5424850b23ccaf74bb9df07
W20251211 20:52:38.361838 847 leader_election.cc:343] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: bc32a778e5424850b23ccaf74bb9df07
I20251211 20:52:38.361938 847 leader_election.cc:304] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:38.362368 1813 raft_consensus.cc:2749] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:38.362921 1163 ts_tablet_manager.cc:1950] T 25454e67437541d9ac2bfab29bad55f9 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:38.363786 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 730) took 2243 ms. Trace:
I20251211 20:52:38.363930 1163 rpcz_store.cc:276] 1211 20:52:36.120774 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:36.121552 (+ 778us) service_pool.cc:225] Handling call
1211 20:52:38.085364 (+1963812us) ts_tablet_manager.cc:1151] Deleting tablet 25454e67437541d9ac2bfab29bad55f9
1211 20:52:38.085374 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:38.136644 (+ 51270us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:38.139315 (+ 2671us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:38.363765 (+224450us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":47136,"tablet-delete.queue_time_us":1963649}
I20251211 20:52:38.364571 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 732) took 2228 ms. Trace:
I20251211 20:52:38.364715 1163 rpcz_store.cc:276] 1211 20:52:36.136140 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:36.136342 (+ 202us) service_pool.cc:225] Handling call
1211 20:52:38.364425 (+2228083us) ts_tablet_manager.cc:1151] Deleting tablet 10746162da4640e6a2a891467690b770
1211 20:52:38.364437 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:38.364549 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2227831}
I20251211 20:52:38.365113 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 733) took 2228 ms. Trace:
I20251211 20:52:38.365237 1163 rpcz_store.cc:276] 1211 20:52:36.136295 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:36.136627 (+ 332us) service_pool.cc:225] Handling call
1211 20:52:38.365021 (+2228394us) ts_tablet_manager.cc:1151] Deleting tablet 2f7b1bc3dd1f4496b3ac6e71652cf012
1211 20:52:38.365030 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:38.365101 (+ 71us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2228230}
I20251211 20:52:38.365967 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 743) took 1850 ms. Trace:
I20251211 20:52:38.366122 1163 rpcz_store.cc:276] 1211 20:52:36.515920 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:36.516641 (+ 721us) service_pool.cc:225] Handling call
1211 20:52:38.365732 (+1849091us) ts_tablet_manager.cc:1151] Deleting tablet e8db5a11a69e4554bc689c38ab57b28a
1211 20:52:38.365769 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:38.365931 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1738259}
I20251211 20:52:38.366513 1163 tablet_replica.cc:333] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
W20251211 20:52:38.367980 711 consensus_peers.cc:597] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:38.368584 710 leader_election.cc:304] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:38.369041 2093 raft_consensus.cc:2804] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:38.369112 2093 raft_consensus.cc:493] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:38.369164 2093 raft_consensus.cc:3060] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:38.369920 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 66cdc6f297b24cfba3a3a0ae9f45d12d consensus state has no leader
I20251211 20:52:38.370340 595 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_e6f3a819faad48aabfad3a235c3c1b95 [id=18f8d722faaa4293a112525ff0c96216] alter complete (version 4)
I20251211 20:52:38.368631 1165 ts_tablet_manager.cc:1929] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:38.372730 1165 log.cc:1199] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/6c64fd8dd49b4e5d91c1ffb0fc2559e0
I20251211 20:52:38.372769 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_da0851457f78469c9a4b783e05c6a58e" } modify_external_catalogs: true
I20251211 20:52:38.372972 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_da0851457f78469c9a4b783e05c6a58e" } modify_external_catalogs: true
I20251211 20:52:38.373200 1165 ts_tablet_manager.cc:1950] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:38.373179 2093 raft_consensus.cc:515] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.373512 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 783) took 1130 ms. Trace:
I20251211 20:52:38.373591 1165 rpcz_store.cc:276] 1211 20:52:37.242947 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.244684 (+ 1737us) service_pool.cc:225] Handling call
1211 20:52:38.176044 (+931360us) ts_tablet_manager.cc:1151] Deleting tablet 6c64fd8dd49b4e5d91c1ffb0fc2559e0
1211 20:52:38.176053 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:38.230013 (+ 53960us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:38.234504 (+ 4491us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:38.373485 (+138981us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":907466}
I20251211 20:52:38.374079 2093 leader_election.cc:290] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:38.374683 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 6c64fd8dd49b4e5d91c1ffb0fc2559e0 (table default.Table_cc48404bbbf24a7caae7641a8d42c247 [id=048a032339b3462ba676bcb44f4e2082]) successfully deleted
I20251211 20:52:38.374984 1166 tablet_bootstrap.cc:492] T 676faec7f3d1488f99b220f8e968e9a0 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:38.375131 1166 ts_tablet_manager.cc:1403] T 676faec7f3d1488f99b220f8e968e9a0 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.001s
I20251211 20:52:38.375129 2170 tablet.cc:1722] T 8feee01fc0aa4b32be926ff2b617b894 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:38.375730 741 tablet_service.cc:1505] Processing CreateTablet for tablet b52377c8ffd741a598e5328a00d876fe (DEFAULT_TABLE table=default.Table_686a26915b42421380037144900dd97c [id=8a1b0c9d641b4a3c83d1ca48576d110d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.375950 1166 raft_consensus.cc:359] T 676faec7f3d1488f99b220f8e968e9a0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.376098 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b52377c8ffd741a598e5328a00d876fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.376097 1166 raft_consensus.cc:385] T 676faec7f3d1488f99b220f8e968e9a0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:38.376138 1166 raft_consensus.cc:740] T 676faec7f3d1488f99b220f8e968e9a0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:38.376427 1166 consensus_queue.cc:260] T 676faec7f3d1488f99b220f8e968e9a0 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.376960 1166 ts_tablet_manager.cc:1434] T 676faec7f3d1488f99b220f8e968e9a0 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:38.377203 1166 tablet_bootstrap.cc:492] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:38.378568 1166 tablet_bootstrap.cc:654] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:38.379851 739 tablet_service.cc:1505] Processing CreateTablet for tablet 2e800d8cc5ea4535a065d7d9243326b4 (DEFAULT_TABLE table=default.Table_083c478bedec45a5bf5203e4ed333860 [id=8baf0ea87d734d58896adbd0d6ef99c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.380218 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e800d8cc5ea4535a065d7d9243326b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.380268 1168 tablet_bootstrap.cc:654] T f2d6c9781b4d4c37bc08703347963d5d P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:38.381318 889 tablet_service.cc:1505] Processing CreateTablet for tablet b52377c8ffd741a598e5328a00d876fe (DEFAULT_TABLE table=default.Table_686a26915b42421380037144900dd97c [id=8a1b0c9d641b4a3c83d1ca48576d110d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.383874 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb112b6431af42a6a7b328dfdefbc2a2" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:38.384290 745 tablet_service.cc:1505] Processing CreateTablet for tablet 4776c456b4d74c90b1fc71d57b62a461 (DEFAULT_TABLE table=default.Table_f520281d794144b5b124b2a218ef3fea [id=796b07624b41435c8c16b117648688ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.384629 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4776c456b4d74c90b1fc71d57b62a461. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:38.386513 711 leader_election.cc:343] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:38.388147 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9739da338096425a8cd7aa3038740fa6" } modify_external_catalogs: true
I20251211 20:52:38.388334 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9739da338096425a8cd7aa3038740fa6" } modify_external_catalogs: true
I20251211 20:52:38.389945 1166 tablet_bootstrap.cc:492] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:38.390081 1166 ts_tablet_manager.cc:1403] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20251211 20:52:38.390849 1166 raft_consensus.cc:359] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.391008 1166 raft_consensus.cc:385] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:38.391055 1166 raft_consensus.cc:740] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:38.391544 1166 consensus_queue.cc:260] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.392692 753 tablet_service.cc:1552] Processing DeleteTablet for tablet b631e6b3048e428f901af1b5b0b1ab3c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:38.393101 750 tablet_service.cc:1552] Processing DeleteTablet for tablet bb112b6431af42a6a7b328dfdefbc2a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:38.393317 752 tablet_service.cc:1552] Processing DeleteTablet for tablet a2058b67ddf1413db338bfa2f64e7a77 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:38.393512 748 tablet_service.cc:1552] Processing DeleteTablet for tablet 52ef1707f6f2425aafd454f303af0333 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:38.393693 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 92a71dff4ea8445d8484fc2747471148 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:38.393864 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 4201da4241aa48dfb670a713e4453ca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:38.394032 738 tablet_service.cc:1552] Processing DeleteTablet for tablet 0b267c7feaac43f6aacb7229759c7ec3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:38.396507 711 consensus_peers.cc:597] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:38.397241 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ca767602e99a4dd9a13c30726a3465fa" } modify_external_catalogs: true
I20251211 20:52:38.397457 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ca767602e99a4dd9a13c30726a3465fa" } modify_external_catalogs: true
I20251211 20:52:38.398017 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 25454e67437541d9ac2bfab29bad55f9 (table default.Table_b0e00b07650849958a1a4486eeb3c660 [id=73346680c9a043cea75e13865bdc7bed]) successfully deleted
W20251211 20:52:38.398422 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 10746162da4640e6a2a891467690b770 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 10746162da4640e6a2a891467690b770 already in progress: creating tablet
W20251211 20:52:38.398782 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 2f7b1bc3dd1f4496b3ac6e71652cf012 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f7b1bc3dd1f4496b3ac6e71652cf012 already in progress: creating tablet
I20251211 20:52:38.398836 593 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet bfdc8fa544894f43921414d9246fb273
I20251211 20:52:38.399415 877 tablet_service.cc:1505] Processing CreateTablet for tablet 877aa237fe43452cbdbb4dac46f613ae (DEFAULT_TABLE table=default.Table_fb5ab0d4096b4494b7ca5a55deb992c2 [id=9c0ed678cf214ff285e736b8c3feb0b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.400056 1166 ts_tablet_manager.cc:1434] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
W20251211 20:52:38.400059 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet e8db5a11a69e4554bc689c38ab57b28a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e8db5a11a69e4554bc689c38ab57b28a already in progress: creating tablet
I20251211 20:52:38.400439 1166 tablet_bootstrap.cc:492] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:38.405407 1168 tablet_bootstrap.cc:492] T f2d6c9781b4d4c37bc08703347963d5d P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:38.405628 1168 ts_tablet_manager.cc:1403] T f2d6c9781b4d4c37bc08703347963d5d P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.047s user 0.002s sys 0.002s
I20251211 20:52:38.406366 1168 raft_consensus.cc:359] T f2d6c9781b4d4c37bc08703347963d5d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.406608 1168 raft_consensus.cc:385] T f2d6c9781b4d4c37bc08703347963d5d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:38.406690 1168 raft_consensus.cc:740] T f2d6c9781b4d4c37bc08703347963d5d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:38.406906 1168 consensus_queue.cc:260] T f2d6c9781b4d4c37bc08703347963d5d P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.407420 1168 ts_tablet_manager.cc:1434] T f2d6c9781b4d4c37bc08703347963d5d P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:38.408002 1168 tablet_bootstrap.cc:492] T e4b5a7553110424d901788c7a2ecdce6 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:38.409715 1168 tablet_bootstrap.cc:654] T e4b5a7553110424d901788c7a2ecdce6 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:38.410509 1813 raft_consensus.cc:493] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:38.410647 1813 raft_consensus.cc:515] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.411226 1813 leader_election.cc:290] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:38.411327 593 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet c8e17645a26947ab8c01f9b093fa422c (table default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5]). Expected version 2 got 0
I20251211 20:52:38.411598 593 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 676faec7f3d1488f99b220f8e968e9a0 (table default.Table_e523eadf39e648adba47da28461e85e3 [id=a2bbfccfe3974c378521d5ef2333df86]). Expected version 1 got 0
W20251211 20:52:38.412336 847 leader_election.cc:343] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 822abda3c85848728205b71847ef7029
I20251211 20:52:38.412992 2168 tablet.cc:1717] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919: Skipping requested alter to schema version 4, tablet already version 4
W20251211 20:52:38.413064 847 leader_election.cc:343] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 822abda3c85848728205b71847ef7029
I20251211 20:52:38.413164 847 leader_election.cc:304] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:38.413745 1813 raft_consensus.cc:2749] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:38.414503 581 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 33fd09ea06ee47908b85911e4ee79708 (table default.Table_3c3cd48c8fe5411e96cdf811d492925d [id=30620a01c27f456bac05fa7c02a2cb4d], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 33fd09ea06ee47908b85911e4ee79708 consensus state has no leader
I20251211 20:52:38.416211 888 tablet_service.cc:1505] Processing CreateTablet for tablet bfb694657a0a44a0a3a3325cd171a78e (DEFAULT_TABLE table=default.Table_86e88e19fbd645d3bd5a919328a0969c [id=cccf9f896f034dda92ea20dfbd569ae1]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:38.417397 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 48833.5us
I20251211 20:52:38.417730 710 leader_election.cc:304] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:38.418051 2093 raft_consensus.cc:2804] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:38.418144 2093 raft_consensus.cc:493] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:38.418242 2093 raft_consensus.cc:3060] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:38.420851 2093 raft_consensus.cc:515] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.421556 2093 leader_election.cc:290] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:38.422915 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2dde7f28e61d41f48015ba6e4f3757f2" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:38.423851 1166 tablet_bootstrap.cc:654] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:38.424463 894 tablet_service.cc:1505] Processing CreateTablet for tablet 8675e2a18f0f4a0e95a87005cf82d8b4 (DEFAULT_TABLE table=default.Table_bd9633c55bd242c18aac3a7f2c18a329 [id=767f50bee5464316a18f73e64bab817a]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:38.424815 711 leader_election.cc:343] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:38.425248 739 tablet_service.cc:1505] Processing CreateTablet for tablet 938a0d8a090f490b816fdf2b8f1a51b8 (DEFAULT_TABLE table=default.Table_4d6139c7976d414fbdcbdb96787d186b [id=9ffcb594267a4b0eb8d792dbe46d585f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.425652 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 938a0d8a090f490b816fdf2b8f1a51b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.429934 710 leader_election.cc:304] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:38.430269 2086 raft_consensus.cc:2804] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:38.430363 2086 raft_consensus.cc:493] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:38.430419 2086 raft_consensus.cc:3060] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:38.430935 741 tablet_service.cc:1505] Processing CreateTablet for tablet bfb694657a0a44a0a3a3325cd171a78e (DEFAULT_TABLE table=default.Table_86e88e19fbd645d3bd5a919328a0969c [id=cccf9f896f034dda92ea20dfbd569ae1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.431293 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfb694657a0a44a0a3a3325cd171a78e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.432909 2086 raft_consensus.cc:515] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.433472 2086 leader_election.cc:290] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:38.434059 710 leader_election.cc:343] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Illegal state: Tablet not RUNNING: STOPPED
I20251211 20:52:38.434285 710 leader_election.cc:304] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:38.434973 2086 raft_consensus.cc:2749] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20251211 20:52:38.435070 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f807596fa6a4e66b89566b5d5dd1080" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:38.435670 711 leader_election.cc:343] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:38.436558 1166 tablet_bootstrap.cc:492] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:38.436512 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0ec526100c24487b808171d9734bcfb7" } modify_external_catalogs: true
I20251211 20:52:38.436789 1166 ts_tablet_manager.cc:1403] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.036s user 0.006s sys 0.000s
I20251211 20:52:38.437244 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0ec526100c24487b808171d9734bcfb7" } modify_external_catalogs: true
I20251211 20:52:38.437958 1166 raft_consensus.cc:359] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.438669 757 tablet_service.cc:1505] Processing CreateTablet for tablet 8675e2a18f0f4a0e95a87005cf82d8b4 (DEFAULT_TABLE table=default.Table_bd9633c55bd242c18aac3a7f2c18a329 [id=767f50bee5464316a18f73e64bab817a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.438468 2135 raft_consensus.cc:493] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6f8d784fd1cd492bbe3809403ec34919)
I20251211 20:52:38.438897 1166 raft_consensus.cc:385] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:38.438959 1166 raft_consensus.cc:740] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:38.438944 2135 raft_consensus.cc:515] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.439114 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8675e2a18f0f4a0e95a87005cf82d8b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.439543 2135 leader_election.cc:290] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:38.439704 1166 consensus_queue.cc:260] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.439808 597 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet c8e17645a26947ab8c01f9b093fa422c (table default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5]). Expected version 2 got 0
I20251211 20:52:38.440546 754 tablet_service.cc:1505] Processing CreateTablet for tablet 877aa237fe43452cbdbb4dac46f613ae (DEFAULT_TABLE table=default.Table_fb5ab0d4096b4494b7ca5a55deb992c2 [id=9c0ed678cf214ff285e736b8c3feb0b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.441222 1166 ts_tablet_manager.cc:1434] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251211 20:52:38.441569 1166 tablet_bootstrap.cc:492] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:38.442236 982 leader_election.cc:343] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 7474a08c51cf4346ae9a518963d56b7a
W20251211 20:52:38.443126 983 leader_election.cc:343] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Not found: Tablet not found: 7474a08c51cf4346ae9a518963d56b7a
I20251211 20:52:38.443222 983 leader_election.cc:304] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:38.443534 2135 raft_consensus.cc:2749] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:52:38.444976 881 tablet_service.cc:1505] Processing CreateTablet for tablet df457783b2534b3ea23bb1fd529ce268 (DEFAULT_TABLE table=default.Table_6e9f3ca164384cc8ac4501deb3f0c6aa [id=342c2469d6e74e3e9217236e3aa863d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.445400 1166 tablet_bootstrap.cc:654] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:38.446904 1168 tablet_bootstrap.cc:492] T e4b5a7553110424d901788c7a2ecdce6 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:38.447085 1168 ts_tablet_manager.cc:1403] T e4b5a7553110424d901788c7a2ecdce6 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.039s user 0.006s sys 0.002s
I20251211 20:52:38.447499 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 877aa237fe43452cbdbb4dac46f613ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.449224 1015 tablet_service.cc:1505] Processing CreateTablet for tablet 52b1bf7f19584d5297b56e5aacb98560 (DEFAULT_TABLE table=default.Table_b929f7410fd44316b3a3571d02066b45 [id=e9dcc3d777524e70b9653ec6a2df7da0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.449746 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52b1bf7f19584d5297b56e5aacb98560. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.449832 884 tablet_service.cc:1505] Processing CreateTablet for tablet 938a0d8a090f490b816fdf2b8f1a51b8 (DEFAULT_TABLE table=default.Table_4d6139c7976d414fbdcbdb96787d186b [id=9ffcb594267a4b0eb8d792dbe46d585f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.450337 880 tablet_service.cc:1505] Processing CreateTablet for tablet 4776c456b4d74c90b1fc71d57b62a461 (DEFAULT_TABLE table=default.Table_f520281d794144b5b124b2a218ef3fea [id=796b07624b41435c8c16b117648688ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.451596 879 tablet_service.cc:1505] Processing CreateTablet for tablet 2e800d8cc5ea4535a065d7d9243326b4 (DEFAULT_TABLE table=default.Table_083c478bedec45a5bf5203e4ed333860 [id=8baf0ea87d734d58896adbd0d6ef99c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.451766 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_bc230026e7c34f16ad80ed668889d340" } new_table_name: "default.Table_68ccb2052d824ccf92f7162d2961fcb1" modify_external_catalogs: true
I20251211 20:52:38.454119 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b158d0ce6ac74ce9a3a5f207a42ec4f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:38.455220 1168 raft_consensus.cc:359] T e4b5a7553110424d901788c7a2ecdce6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.455497 1168 raft_consensus.cc:385] T e4b5a7553110424d901788c7a2ecdce6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:38.455615 1168 raft_consensus.cc:740] T e4b5a7553110424d901788c7a2ecdce6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
W20251211 20:52:38.455971 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b158d0ce6ac74ce9a3a5f207a42ec4f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.456001 1168 consensus_queue.cc:260] T e4b5a7553110424d901788c7a2ecdce6 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.456609 1168 ts_tablet_manager.cc:1434] T e4b5a7553110424d901788c7a2ecdce6 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20251211 20:52:38.458550 890 tablet_service.cc:1505] Processing CreateTablet for tablet 2cf2f91730fa480b92063e50489bd37e (DEFAULT_TABLE table=default.Table_b94ece9030c84d37a4ed8d28c2a12a91 [id=2026c3e019fb4c85bb9911107c1cd648]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.458701 1168 tablet_bootstrap.cc:492] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:38.460855 1168 tablet_bootstrap.cc:654] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:38.460985 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_25841fca409b44329301efe7768c649b" } new_table_name: "default.Table_031de0f4cc7744bc9e320261f8b46696" modify_external_catalogs: true
I20251211 20:52:38.462411 746 tablet_service.cc:1505] Processing CreateTablet for tablet df457783b2534b3ea23bb1fd529ce268 (DEFAULT_TABLE table=default.Table_6e9f3ca164384cc8ac4501deb3f0c6aa [id=342c2469d6e74e3e9217236e3aa863d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.462857 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df457783b2534b3ea23bb1fd529ce268. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.464310 1160 tablet_bootstrap.cc:492] T 3fc173c396ff4af5a8589cff18165e85 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:38.464855 1160 ts_tablet_manager.cc:1403] T 3fc173c396ff4af5a8589cff18165e85 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.438s user 0.004s sys 0.003s
I20251211 20:52:38.465728 1160 raft_consensus.cc:359] T 3fc173c396ff4af5a8589cff18165e85 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:38.465981 1160 raft_consensus.cc:385] T 3fc173c396ff4af5a8589cff18165e85 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:38.466065 1160 raft_consensus.cc:740] T 3fc173c396ff4af5a8589cff18165e85 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:38.466332 1160 consensus_queue.cc:260] T 3fc173c396ff4af5a8589cff18165e85 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:38.467103 1160 ts_tablet_manager.cc:1434] T 3fc173c396ff4af5a8589cff18165e85 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:38.469089 757 tablet_service.cc:1552] Processing DeleteTablet for tablet e4b5a7553110424d901788c7a2ecdce6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:38.471218 885 tablet_service.cc:1505] Processing CreateTablet for tablet fb29f983c4524a49bdc5c0c1144ad708 (DEFAULT_TABLE table=default.Table_7809ac85fdfe4c1d9c58afb54edecb27 [id=c4b0bfff99f64888beb70d9154d2f156]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.475739 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 4f807596fa6a4e66b89566b5d5dd1080 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:38.481057 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b52377c8ffd741a598e5328a00d876fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.483058 597 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 8feee01fc0aa4b32be926ff2b617b894 (table default.Table_e6f3a819faad48aabfad3a235c3c1b95 [id=18f8d722faaa4293a112525ff0c96216]). Expected version 4 got 0
I20251211 20:52:38.484534 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 28ef13be3b84410d9c786c43502a3478 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:38.485036 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 877aa237fe43452cbdbb4dac46f613ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.480183 741 tablet_service.cc:1505] Processing CreateTablet for tablet 2cf2f91730fa480b92063e50489bd37e (DEFAULT_TABLE table=default.Table_b94ece9030c84d37a4ed8d28c2a12a91 [id=2026c3e019fb4c85bb9911107c1cd648]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.486665 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cf2f91730fa480b92063e50489bd37e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.487903 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_4b444339f63048179e9b53a3aec9173c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:38.488631 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4b444339f63048179e9b53a3aec9173c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.489092 1164 ts_tablet_manager.cc:1916] T 3c578d81e8dd41aaa0f707c4e3fed225 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:38.490002 1021 tablet_service.cc:1505] Processing CreateTablet for tablet fdb2b65b027f4abf9b55080c95ab27ae (DEFAULT_TABLE table=default.Table_cb402f83e7b24e308e46a798daa7db02 [id=d7bc4f722b6943438cffc3f0a7e5015a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.490412 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdb2b65b027f4abf9b55080c95ab27ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.492264 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 5bcc94ec9630484cb7a3e8bf6d4f378a
I20251211 20:52:38.493122 739 tablet_service.cc:1552] Processing DeleteTablet for tablet 5bcc94ec9630484cb7a3e8bf6d4f378a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:38.494073 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 5bcc94ec9630484cb7a3e8bf6d4f378a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:38.497260 1096 heartbeater.cc:344] Connected to a master server at 127.0.104.61:40749
I20251211 20:52:38.497571 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8675e2a18f0f4a0e95a87005cf82d8b4. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:38.497692 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 63631us
I20251211 20:52:38.497933 710 leader_election.cc:304] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:38.499003 2086 raft_consensus.cc:2804] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:38.499086 2086 raft_consensus.cc:493] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:38.499146 2086 raft_consensus.cc:3060] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:38.499804 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df457783b2534b3ea23bb1fd529ce268. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.499823 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2dde7f28e61d41f48015ba6e4f3757f2" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:38.499984 1042 raft_consensus.cc:3060] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:38.500702 1043 raft_consensus.cc:1275] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Refusing update from remote peer 6f8d784fd1cd492bbe3809403ec34919: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251211 20:52:38.501780 2086 raft_consensus.cc:515] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.502405 2086 leader_election.cc:290] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:38.502542 2086 raft_consensus.cc:493] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:38.502662 2086 raft_consensus.cc:515] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.503122 2086 leader_election.cc:290] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:38.504269 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb030a62796248869d46b682656eaeb0" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:38.505144 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 938a0d8a090f490b816fdf2b8f1a51b8. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:38.505120 711 leader_election.cc:343] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:38.507287 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b267c7feaac43f6aacb7229759c7ec3" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:38.507854 711 leader_election.cc:343] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:38.508234 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet a471a9accbfc4875bd1d7802f5620808 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:38.508607 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4776c456b4d74c90b1fc71d57b62a461. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.509491 2100 tablet.cc:1717] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7: Skipping requested alter to schema version 4, tablet already version 4
I20251211 20:52:38.510551 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 0beff1cfdb1e49908ee4e6756b8b7e65 (DEFAULT_TABLE table=default.Table_97f780d6dd39409dbb5e5ff7e4fd61fd [id=adef6ee1143c43e495691dc186c25241]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.510915 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0beff1cfdb1e49908ee4e6756b8b7e65. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.511741 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 5b3e8ec3faf445ac9b746e65dc4dbfe0 (DEFAULT_TABLE table=default.Table_7fe80ff5483a4b0292091119980f6639 [id=3c3462a0c09748cea0d2ffb544ae2f13]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.511962 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e800d8cc5ea4535a065d7d9243326b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.512042 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b3e8ec3faf445ac9b746e65dc4dbfe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.510551 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 6212b3b4425543168837946a54d60834 (DEFAULT_TABLE table=default.Table_4abd8d4223574788846f1682e0e522ff [id=2d489cd7d73e403d84bf10960e6a2dfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.512375 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6212b3b4425543168837946a54d60834. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.514415 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cf2f91730fa480b92063e50489bd37e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.514705 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f807596fa6a4e66b89566b5d5dd1080" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:38.514858 1048 raft_consensus.cc:3060] T 4f807596fa6a4e66b89566b5d5dd1080 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:38.500460 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb112b6431af42a6a7b328dfdefbc2a2" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:38.516661 1045 raft_consensus.cc:3060] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:38.374135 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 785) took 1001 ms. Trace:
I20251211 20:52:38.519853 1165 rpcz_store.cc:276] 1211 20:52:37.373056 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.379674 (+ 6618us) service_pool.cc:225] Handling call
1211 20:52:38.374017 (+994343us) ts_tablet_manager.cc:1151] Deleting tablet 80df719919a44aba9b4e3c62d6830cf3
1211 20:52:38.374027 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:38.374120 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":978448}
I20251211 20:52:38.520498 1048 raft_consensus.cc:2468] T 4f807596fa6a4e66b89566b5d5dd1080 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
W20251211 20:52:38.520635 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 80df719919a44aba9b4e3c62d6830cf3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 80df719919a44aba9b4e3c62d6830cf3 already in progress: creating tablet
I20251211 20:52:38.521140 1165 tablet_replica.cc:333] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:38.521389 1165 raft_consensus.cc:2243] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:38.521622 1165 raft_consensus.cc:2272] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:38.522215 1165 ts_tablet_manager.cc:1916] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:38.524490 1163 raft_consensus.cc:2243] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:38.524710 1163 raft_consensus.cc:2272] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:38.525235 1163 ts_tablet_manager.cc:1916] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:38.526640 1042 raft_consensus.cc:2468] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:38.538064 2093 consensus_queue.cc:1048] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:38.539433 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb030a62796248869d46b682656eaeb0" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:38.539538 2168 tablet.cc:1722] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:38.539671 1042 raft_consensus.cc:3060] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:38.540016 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfb694657a0a44a0a3a3325cd171a78e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.543803 1045 raft_consensus.cc:2468] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:38.545352 1042 raft_consensus.cc:2468] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:38.548231 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9d7fd7d87cf049c28685144bb909d0c3" } new_table_name: "default.Table_ed6ce5986e224e49a69922db6e940db6" modify_external_catalogs: true
I20251211 20:52:38.549831 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_0e243018f79e4e358c4f844e6ba51c35"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:38.550411 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0e243018f79e4e358c4f844e6ba51c35 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.551640 914 raft_consensus.cc:3060] T 3fc173c396ff4af5a8589cff18165e85 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:38.551710 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_6fbcf571523d4693b48035d60c6322ad" } new_table_name: "default.Table_6f8c133c7a364b93af2c83d8911a0ac3" modify_external_catalogs: true
I20251211 20:52:38.549876 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b267c7feaac43f6aacb7229759c7ec3" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:38.557854 2170 tablet.cc:1722] T 40722aef4e194ff094d271a079bda9d9 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:38.558249 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 75dd170c5828409fa69bd68fdf28a72b
I20251211 20:52:38.559180 1015 tablet_service.cc:1552] Processing DeleteTablet for tablet ccc3ddd4b7f74f11a3835997cb7ae8d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:38.561058 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 75dd170c5828409fa69bd68fdf28a72b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:38.562729 2170 tablet.cc:1722] T 8feee01fc0aa4b32be926ff2b617b894 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20251211 20:52:38.564006 1015 tablet_service.cc:1505] Processing CreateTablet for tablet 6888d2a2c6574c93ae0475faf07609f2 (DEFAULT_TABLE table=default.Table_bd71cd88b0354c88aa03799a4362f1b1 [id=2f4a7d71bf18442fa8f3ed7421512ff2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.564494 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6888d2a2c6574c93ae0475faf07609f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.567303 741 tablet_service.cc:1505] Processing CreateTablet for tablet fb29f983c4524a49bdc5c0c1144ad708 (DEFAULT_TABLE table=default.Table_7809ac85fdfe4c1d9c58afb54edecb27 [id=c4b0bfff99f64888beb70d9154d2f156]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.567775 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb29f983c4524a49bdc5c0c1144ad708. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.567936 710 leader_election.cc:304] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:38.568182 2093 raft_consensus.cc:2804] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:38.568269 2093 raft_consensus.cc:697] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:38.568466 2093 consensus_queue.cc:237] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.570578 746 tablet_service.cc:1505] Processing CreateTablet for tablet 2a276bc5e8a3458fb63a2a6a652aa28d (DEFAULT_TABLE table=default.Table_47ff5fcc1088481592447a200ff8432f [id=b4cbc8ef524d4b72bc1c109aa2dd09e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.570952 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a276bc5e8a3458fb63a2a6a652aa28d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.571339 877 tablet_service.cc:1505] Processing CreateTablet for tablet 174902fb98a5492dbc29423797cb5a2b (DEFAULT_TABLE table=default.Table_f5a4d53f860a4ab48ac66679cd67c7eb [id=db7d2c4d61b246f2984075c1787b24ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.565797 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 9db471247b674e34acf4239a8b55e1d6 (DEFAULT_TABLE table=default.Table_7e3bb905c8ec4e96809e6b7d4e44a3b7 [id=8212122882be451f9b5f5c6880bf3dce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.571727 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 174902fb98a5492dbc29423797cb5a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.571895 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9db471247b674e34acf4239a8b55e1d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.575752 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 745ce63314834c8abdecaa367133a068 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:38.575904 889 tablet_service.cc:1505] Processing CreateTablet for tablet 2a276bc5e8a3458fb63a2a6a652aa28d (DEFAULT_TABLE table=default.Table_47ff5fcc1088481592447a200ff8432f [id=b4cbc8ef524d4b72bc1c109aa2dd09e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.576124 739 tablet_service.cc:1505] Processing CreateTablet for tablet 174902fb98a5492dbc29423797cb5a2b (DEFAULT_TABLE table=default.Table_f5a4d53f860a4ab48ac66679cd67c7eb [id=db7d2c4d61b246f2984075c1787b24ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.576367 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a276bc5e8a3458fb63a2a6a652aa28d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.576350 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 757ec913dff840f992a41a38b05f8d41 (DEFAULT_TABLE table=default.Table_a05c8b486c1940c29db8cbb1ea1dfb17 [id=9e0f07ff85c4413eb6efb4223190223c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.576715 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 757ec913dff840f992a41a38b05f8d41. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.577178 710 leader_election.cc:304] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:38.577344 710 raft_consensus.cc:2686] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251211 20:52:38.579833 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb29f983c4524a49bdc5c0c1144ad708. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.580051 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 174902fb98a5492dbc29423797cb5a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.580065 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 516f7181aaac48cdadf9fbc08d0d3f57 (DEFAULT_TABLE table=default.Table_87f092378445464ab48c155fae469f29 [id=0f029363985443529615cdd1f8bbd78d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.580397 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 516f7181aaac48cdadf9fbc08d0d3f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.584216 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 75dd170c5828409fa69bd68fdf28a72b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:38.584403 1160 tablet_bootstrap.cc:492] T 5bd801124df84a6c96f4912993f1bdfd P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:38.584650 2086 consensus_queue.cc:1048] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:38.585434 710 leader_election.cc:304] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:38.585916 1160 tablet_bootstrap.cc:654] T 5bd801124df84a6c96f4912993f1bdfd P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:38.589270 1021 tablet_service.cc:1505] Processing CreateTablet for tablet b52377c8ffd741a598e5328a00d876fe (DEFAULT_TABLE table=default.Table_686a26915b42421380037144900dd97c [id=8a1b0c9d641b4a3c83d1ca48576d110d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.589768 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b52377c8ffd741a598e5328a00d876fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.587653 2086 raft_consensus.cc:493] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:38.593250 2086 raft_consensus.cc:515] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.593861 2086 leader_election.cc:290] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:38.590111 2168 tablet.cc:1722] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:38.595873 1168 tablet_bootstrap.cc:492] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:38.596009 1168 ts_tablet_manager.cc:1403] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.137s user 0.004s sys 0.013s
I20251211 20:52:38.596798 1168 raft_consensus.cc:359] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.596956 1168 raft_consensus.cc:385] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:38.597003 1168 raft_consensus.cc:740] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:38.597831 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 2e800d8cc5ea4535a065d7d9243326b4 (DEFAULT_TABLE table=default.Table_083c478bedec45a5bf5203e4ed333860 [id=8baf0ea87d734d58896adbd0d6ef99c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.598317 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e800d8cc5ea4535a065d7d9243326b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.599802 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 449921ea74f04a9c88eb9e094e8226c8
I20251211 20:52:38.601274 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 449921ea74f04a9c88eb9e094e8226c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:38.603684 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 449921ea74f04a9c88eb9e094e8226c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:38.605391 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 877aa237fe43452cbdbb4dac46f613ae (DEFAULT_TABLE table=default.Table_fb5ab0d4096b4494b7ca5a55deb992c2 [id=9c0ed678cf214ff285e736b8c3feb0b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.605888 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 877aa237fe43452cbdbb4dac46f613ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.605926 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "92a71dff4ea8445d8484fc2747471148" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:38.607069 711 leader_election.cc:343] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:38.609540 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_df9af0c629bc44728c89b7ae449b7ea1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:38.610307 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_df9af0c629bc44728c89b7ae449b7ea1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.611932 1021 tablet_service.cc:1505] Processing CreateTablet for tablet 4776c456b4d74c90b1fc71d57b62a461 (DEFAULT_TABLE table=default.Table_f520281d794144b5b124b2a218ef3fea [id=796b07624b41435c8c16b117648688ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.612300 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4776c456b4d74c90b1fc71d57b62a461. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.612509 2086 raft_consensus.cc:2804] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:38.612632 2086 raft_consensus.cc:697] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:38.612854 2086 consensus_queue.cc:237] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.615911 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet 793f543d6b614ad7b3ab8e717db2ceaf with delete_type TABLET_DATA_DELETED (replaced by tablet 06b8f8e46e524545aeab6c931efaf116) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:38.616600 1024 tablet_service.cc:1505] Processing CreateTablet for tablet 938a0d8a090f490b816fdf2b8f1a51b8 (DEFAULT_TABLE table=default.Table_4d6139c7976d414fbdcbdb96787d186b [id=9ffcb594267a4b0eb8d792dbe46d585f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.616976 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 938a0d8a090f490b816fdf2b8f1a51b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.617779 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 6029c1fd563d4023a8cf7cfb8ffa0876
I20251211 20:52:38.618733 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 6029c1fd563d4023a8cf7cfb8ffa0876 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:38.621451 2187 tablet.cc:1722] T 3fc173c396ff4af5a8589cff18165e85 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:38.623149 884 tablet_service.cc:1505] Processing CreateTablet for tablet 56a51729b1824027baf7d6d1d5645846 (DEFAULT_TABLE table=default.Table_fde09a02a1664bc5bad79b2e0036b42a [id=3c1a33a40f334749b4b79833bfc6063a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.623646 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56a51729b1824027baf7d6d1d5645846. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.623798 891 tablet_service.cc:1505] Processing CreateTablet for tablet 5e47af616e1b4d2db3780c1d67cdcb47 (DEFAULT_TABLE table=default.Table_d87f970fdabb4d10871f55bb8193bd4d [id=052edcbde60c41449d468c8a449fc3f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.624184 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e47af616e1b4d2db3780c1d67cdcb47. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.624678 741 tablet_service.cc:1505] Processing CreateTablet for tablet 56a51729b1824027baf7d6d1d5645846 (DEFAULT_TABLE table=default.Table_fde09a02a1664bc5bad79b2e0036b42a [id=3c1a33a40f334749b4b79833bfc6063a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.625052 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56a51729b1824027baf7d6d1d5645846. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.626320 1024 tablet_service.cc:1505] Processing CreateTablet for tablet bfb694657a0a44a0a3a3325cd171a78e (DEFAULT_TABLE table=default.Table_86e88e19fbd645d3bd5a919328a0969c [id=cccf9f896f034dda92ea20dfbd569ae1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.626674 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfb694657a0a44a0a3a3325cd171a78e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.628211 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9aa05c766d874348a9e0eb2c016058d3" } new_table_name: "default.Table_80e3542d21b64eca991b2e81ac80ac7c" modify_external_catalogs: true
I20251211 20:52:38.634361 1019 tablet_service.cc:1505] Processing CreateTablet for tablet affe191cba57401f9c58211f2d06b80d (DEFAULT_TABLE table=default.Table_7730b7f512954bcab4c83c9f272d9a59 [id=2783ef72c8944e8188a8646565a6df13]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:38.634433 592 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d6923cfd826d495291d9ac8e0b3b44c6 consensus state has no leader
I20251211 20:52:38.634933 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet affe191cba57401f9c58211f2d06b80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.639123 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 6029c1fd563d4023a8cf7cfb8ffa0876 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:38.639916 1013 tablet_service.cc:1505] Processing CreateTablet for tablet 8675e2a18f0f4a0e95a87005cf82d8b4 (DEFAULT_TABLE table=default.Table_bd9633c55bd242c18aac3a7f2c18a329 [id=767f50bee5464316a18f73e64bab817a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.640053 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_96bab30f95d5434ab659bb63869eb23d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:38.640295 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8675e2a18f0f4a0e95a87005cf82d8b4. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:38.640643 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_96bab30f95d5434ab659bb63869eb23d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.653297 741 tablet_service.cc:1505] Processing CreateTablet for tablet 5e47af616e1b4d2db3780c1d67cdcb47 (DEFAULT_TABLE table=default.Table_d87f970fdabb4d10871f55bb8193bd4d [id=052edcbde60c41449d468c8a449fc3f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.653570 746 tablet_service.cc:1505] Processing CreateTablet for tablet ff9e65a8c1a245ef9b79b77848cafca0 (DEFAULT_TABLE table=default.Table_72c2c33862a149418a3c31c07caf8a37 [id=3e233f1f255847a5830bb85f65370aaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.653745 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e47af616e1b4d2db3780c1d67cdcb47. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.653975 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff9e65a8c1a245ef9b79b77848cafca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.654522 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 2cf2f91730fa480b92063e50489bd37e (DEFAULT_TABLE table=default.Table_b94ece9030c84d37a4ed8d28c2a12a91 [id=2026c3e019fb4c85bb9911107c1cd648]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.654894 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cf2f91730fa480b92063e50489bd37e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.659190 1166 tablet_bootstrap.cc:492] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:38.659322 1166 ts_tablet_manager.cc:1403] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.218s user 0.002s sys 0.014s
I20251211 20:52:38.660121 1166 raft_consensus.cc:359] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.660372 1166 raft_consensus.cc:385] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:38.660787 1166 raft_consensus.cc:740] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:38.661156 1013 tablet_service.cc:1505] Processing CreateTablet for tablet df457783b2534b3ea23bb1fd529ce268 (DEFAULT_TABLE table=default.Table_6e9f3ca164384cc8ac4501deb3f0c6aa [id=342c2469d6e74e3e9217236e3aa863d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.661294 1166 consensus_queue.cc:260] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.661767 1166 ts_tablet_manager.cc:1434] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:38.661813 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df457783b2534b3ea23bb1fd529ce268. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.662225 1166 tablet_bootstrap.cc:492] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:38.665668 589 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 3fc173c396ff4af5a8589cff18165e85 (table default.Table_5e5286ea174048f8ba9abd14daf14bcf [id=a455f6ef64ae44f78972d14b2ea9654c]). Expected version 1 got 0
I20251211 20:52:38.666047 589 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 40722aef4e194ff094d271a079bda9d9 (table default.Table_12f41f708f1949598d6176d89777aa69 [id=3309205faabf4a42acadc43766e4f57f]). Expected version 1 got 0
I20251211 20:52:38.663928 1166 tablet_bootstrap.cc:654] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:38.667307 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 81896.1us
I20251211 20:52:38.670466 885 tablet_service.cc:1505] Processing CreateTablet for tablet ff9e65a8c1a245ef9b79b77848cafca0 (DEFAULT_TABLE table=default.Table_72c2c33862a149418a3c31c07caf8a37 [id=3e233f1f255847a5830bb85f65370aaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.670943 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff9e65a8c1a245ef9b79b77848cafca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.671017 710 leader_election.cc:304] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:38.671716 1168 consensus_queue.cc:260] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.673336 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 2d06c480594a44948fe004f73857f3bd (DEFAULT_TABLE table=default.Table_6cce646a13f6405c96fd61f193add5fa [id=1ea52ab9ffc74cb18c6e24e4f4d1cf88]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.673715 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d06c480594a44948fe004f73857f3bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.678462 2019 raft_consensus.cc:2804] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:38.678617 2019 raft_consensus.cc:697] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:38.678777 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 5bcc94ec9630484cb7a3e8bf6d4f378a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:38.678977 2019 consensus_queue.cc:237] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.679207 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 75dd170c5828409fa69bd68fdf28a72b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:38.679858 1029 tablet_service.cc:1505] Processing CreateTablet for tablet fb29f983c4524a49bdc5c0c1144ad708 (DEFAULT_TABLE table=default.Table_7809ac85fdfe4c1d9c58afb54edecb27 [id=c4b0bfff99f64888beb70d9154d2f156]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.680234 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb29f983c4524a49bdc5c0c1144ad708. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.681861 1168 ts_tablet_manager.cc:1434] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.086s user 0.002s sys 0.000s
I20251211 20:52:38.682566 1168 tablet_bootstrap.cc:492] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:38.683148 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b2974f16034c44cf85ff85b5095cf6a0 consensus state has no leader
I20251211 20:52:38.687135 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet e4b5a7553110424d901788c7a2ecdce6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:38.688457 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 56a51729b1824027baf7d6d1d5645846 (DEFAULT_TABLE table=default.Table_fde09a02a1664bc5bad79b2e0036b42a [id=3c1a33a40f334749b4b79833bfc6063a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.688848 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56a51729b1824027baf7d6d1d5645846. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.688874 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_6b058d9ed449475e986f56f95463634a" } modify_external_catalogs: true
I20251211 20:52:38.689064 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_6b058d9ed449475e986f56f95463634a" } modify_external_catalogs: true
W20251211 20:52:38.688866 710 leader_election.cc:343] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Illegal state: Tablet not RUNNING: STOPPED
I20251211 20:52:38.691202 710 leader_election.cc:304] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:38.691416 2019 raft_consensus.cc:2749] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:38.692819 2100 tablet.cc:1722] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:38.693913 1160 tablet_bootstrap.cc:492] T 5bd801124df84a6c96f4912993f1bdfd P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:38.694126 1160 ts_tablet_manager.cc:1403] T 5bd801124df84a6c96f4912993f1bdfd P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.110s user 0.001s sys 0.001s
I20251211 20:52:38.695928 754 tablet_service.cc:1505] Processing CreateTablet for tablet c8bb92fc25124ba4be887c453abc7d9a (DEFAULT_TABLE table=default.Table_f2eaa62388894072a390cc4c167f3385 [id=ac6774893a6a4f82a7c9630bbe502fe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.696363 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8bb92fc25124ba4be887c453abc7d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.701876 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet 6029c1fd563d4023a8cf7cfb8ffa0876 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:38.703143 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 174902fb98a5492dbc29423797cb5a2b (DEFAULT_TABLE table=default.Table_f5a4d53f860a4ab48ac66679cd67c7eb [id=db7d2c4d61b246f2984075c1787b24ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.705456 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 174902fb98a5492dbc29423797cb5a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.708036 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_948151057bf94b75a251b99e3906f4a4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:38.708657 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_948151057bf94b75a251b99e3906f4a4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.710170 2019 raft_consensus.cc:493] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:38.710379 2019 raft_consensus.cc:515] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.711279 2019 leader_election.cc:290] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:38.705483 1166 tablet_bootstrap.cc:492] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:38.711776 1166 ts_tablet_manager.cc:1403] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.050s user 0.003s sys 0.000s
I20251211 20:52:38.712061 741 tablet_service.cc:1505] Processing CreateTablet for tablet e95c63b4d99a4dd0b2c0fa384ca9d552 (DEFAULT_TABLE table=default.Table_d65f65d608a346968cc2f35b9f55912e [id=88e8b1b4524846e18f5a638dee69a688]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.712451 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e95c63b4d99a4dd0b2c0fa384ca9d552. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.702530 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 2a276bc5e8a3458fb63a2a6a652aa28d (DEFAULT_TABLE table=default.Table_47ff5fcc1088481592447a200ff8432f [id=b4cbc8ef524d4b72bc1c109aa2dd09e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.712872 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a276bc5e8a3458fb63a2a6a652aa28d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.715353 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet b631e6b3048e428f901af1b5b0b1ab3c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:38.716578 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fbc2d4aea084492c8270ad1d87096e58 consensus state has no leader
W20251211 20:52:38.717398 2120 lock_manager.cc:585] Waited 1 seconds to obtain row lock on key '<redacted>' (key hash 3711972704387067618) tablet 00000000000000000000000000000000 cur holder 0x617000060f00
I20251211 20:52:38.720750 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_9e7258165cbf458c83d7cd4874a8755f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:38.721396 594 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9e7258165cbf458c83d7cd4874a8755f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.712555 1166 raft_consensus.cc:359] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.722385 1166 raft_consensus.cc:385] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:38.722466 1166 raft_consensus.cc:740] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:38.723085 1010 tablet_service.cc:1505] Processing CreateTablet for tablet ff9e65a8c1a245ef9b79b77848cafca0 (DEFAULT_TABLE table=default.Table_72c2c33862a149418a3c31c07caf8a37 [id=3e233f1f255847a5830bb85f65370aaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.723488 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff9e65a8c1a245ef9b79b77848cafca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.728003 1024 tablet_service.cc:1505] Processing CreateTablet for tablet e95c63b4d99a4dd0b2c0fa384ca9d552 (DEFAULT_TABLE table=default.Table_d65f65d608a346968cc2f35b9f55912e [id=88e8b1b4524846e18f5a638dee69a688]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.728482 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e95c63b4d99a4dd0b2c0fa384ca9d552. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.730089 1168 tablet_bootstrap.cc:654] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:38.732683 888 tablet_service.cc:1505] Processing CreateTablet for tablet aab26df748724fe5b2bb7fad9e3bfc0c (DEFAULT_TABLE table=default.Table_37849576518b4320a3bac34baa096242 [id=df6255d66f334ef2b4406f4b845cf980]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.733134 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aab26df748724fe5b2bb7fad9e3bfc0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.733260 884 tablet_service.cc:1505] Processing CreateTablet for tablet e95c63b4d99a4dd0b2c0fa384ca9d552 (DEFAULT_TABLE table=default.Table_d65f65d608a346968cc2f35b9f55912e [id=88e8b1b4524846e18f5a638dee69a688]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.733548 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f737a7bfb0f44196abdc831bb22d5285" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:38.733707 877 tablet_service.cc:1505] Processing CreateTablet for tablet c8bb92fc25124ba4be887c453abc7d9a (DEFAULT_TABLE table=default.Table_f2eaa62388894072a390cc4c167f3385 [id=ac6774893a6a4f82a7c9630bbe502fe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.733979 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8bb92fc25124ba4be887c453abc7d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.735954 1166 consensus_queue.cc:260] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.739861 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e95c63b4d99a4dd0b2c0fa384ca9d552. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.741240 2135 raft_consensus.cc:493] T e4647105ab57454d883ebfcde7bde891 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:38.741395 2135 raft_consensus.cc:515] T e4647105ab57454d883ebfcde7bde891 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.741554 961 heartbeater.cc:344] Connected to a master server at 127.0.104.61:40749
I20251211 20:52:38.741986 2135 leader_election.cc:290] T e4647105ab57454d883ebfcde7bde891 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:38.742931 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e4647105ab57454d883ebfcde7bde891" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:38.743566 982 leader_election.cc:343] T e4647105ab57454d883ebfcde7bde891 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:38.744342 2019 raft_consensus.cc:493] T f2d6c9781b4d4c37bc08703347963d5d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:38.744415 1166 ts_tablet_manager.cc:1434] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.033s user 0.002s sys 0.000s
I20251211 20:52:38.744586 2019 raft_consensus.cc:515] T f2d6c9781b4d4c37bc08703347963d5d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.744962 1166 tablet_bootstrap.cc:492] T bfb52935296443c58190d4d06e2e6e9c P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:38.745309 2019 leader_election.cc:290] T f2d6c9781b4d4c37bc08703347963d5d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:38.746248 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_862fff1ce3564d939f033f3f2fe049ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:38.746677 1166 tablet_bootstrap.cc:654] T bfb52935296443c58190d4d06e2e6e9c P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:38.746995 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_862fff1ce3564d939f033f3f2fe049ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.747304 1043 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f737a7bfb0f44196abdc831bb22d5285" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:38.747473 1043 raft_consensus.cc:2468] T f737a7bfb0f44196abdc831bb22d5285 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:38.750464 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f2d6c9781b4d4c37bc08703347963d5d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:38.750962 1163 ts_tablet_manager.cc:1929] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:38.751112 1163 log.cc:1199] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/0b267c7feaac43f6aacb7229759c7ec3
I20251211 20:52:38.751075 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e4647105ab57454d883ebfcde7bde891" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:38.751183 711 leader_election.cc:343] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:38.751435 1163 ts_tablet_manager.cc:1950] T 0b267c7feaac43f6aacb7229759c7ec3 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:38.751772 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 742) took 2235 ms. Trace:
I20251211 20:52:38.753124 1163 rpcz_store.cc:276] 1211 20:52:36.515842 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:36.515876 (+ 34us) service_pool.cc:225] Handling call
1211 20:52:38.366448 (+1850572us) ts_tablet_manager.cc:1151] Deleting tablet 0b267c7feaac43f6aacb7229759c7ec3
1211 20:52:38.366457 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:38.609740 (+243283us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:38.750847 (+141107us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:38.751754 (+ 907us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1738555}
I20251211 20:52:38.753458 2101 consensus_queue.cc:1048] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:38.753644 1163 tablet_replica.cc:333] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:38.753825 1163 raft_consensus.cc:2243] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:38.753782 1018 tablet_service.cc:1505] Processing CreateTablet for tablet aab26df748724fe5b2bb7fad9e3bfc0c (DEFAULT_TABLE table=default.Table_37849576518b4320a3bac34baa096242 [id=df6255d66f334ef2b4406f4b845cf980]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.753892 741 tablet_service.cc:1505] Processing CreateTablet for tablet aab26df748724fe5b2bb7fad9e3bfc0c (DEFAULT_TABLE table=default.Table_37849576518b4320a3bac34baa096242 [id=df6255d66f334ef2b4406f4b845cf980]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.754000 1163 raft_consensus.cc:2272] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:38.754158 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aab26df748724fe5b2bb7fad9e3bfc0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.754202 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aab26df748724fe5b2bb7fad9e3bfc0c. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:38.754403 710 leader_election.cc:343] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 92a71dff4ea8445d8484fc2747471148
I20251211 20:52:38.754534 1163 ts_tablet_manager.cc:1916] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:38.754503 710 leader_election.cc:304] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:38.755678 2101 raft_consensus.cc:2749] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:38.756210 710 leader_election.cc:304] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:38.757539 2101 raft_consensus.cc:2804] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:38.757611 2101 raft_consensus.cc:493] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:38.757663 2101 raft_consensus.cc:3060] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:38.760952 2101 raft_consensus.cc:515] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.761018 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 0b267c7feaac43f6aacb7229759c7ec3 (table default.Table_a95581dfcc7a448ea004e1b657388be9 [id=32aa02b009a64faca225b65dadd3b479]) successfully deleted
I20251211 20:52:38.761530 2101 leader_election.cc:290] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:38.764864 2101 consensus_queue.cc:1048] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251211 20:52:38.751994 983 leader_election.cc:343] T e4647105ab57454d883ebfcde7bde891 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:38.767724 983 leader_election.cc:304] T e4647105ab57454d883ebfcde7bde891 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:38.768149 2135 raft_consensus.cc:2749] T e4647105ab57454d883ebfcde7bde891 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:38.768597 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f00128ba84c245d9a0e35f26bc842b3e has no consensus state
W20251211 20:52:38.768824 711 consensus_peers.cc:597] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:38.769057 711 leader_election.cc:343] T f2d6c9781b4d4c37bc08703347963d5d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:38.770239 1160 raft_consensus.cc:359] T 5bd801124df84a6c96f4912993f1bdfd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.770413 1160 raft_consensus.cc:385] T 5bd801124df84a6c96f4912993f1bdfd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:38.770465 1160 raft_consensus.cc:740] T 5bd801124df84a6c96f4912993f1bdfd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:38.770648 1160 consensus_queue.cc:260] T 5bd801124df84a6c96f4912993f1bdfd P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.771107 1160 ts_tablet_manager.cc:1434] T 5bd801124df84a6c96f4912993f1bdfd P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.077s user 0.001s sys 0.001s
I20251211 20:52:38.773860 590 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3fc173c396ff4af5a8589cff18165e85 (table default.Table_5e5286ea174048f8ba9abd14daf14bcf [id=a455f6ef64ae44f78972d14b2ea9654c]). Expected version 1 got 0
I20251211 20:52:38.775960 1015 tablet_service.cc:1505] Processing CreateTablet for tablet 5e47af616e1b4d2db3780c1d67cdcb47 (DEFAULT_TABLE table=default.Table_d87f970fdabb4d10871f55bb8193bd4d [id=052edcbde60c41449d468c8a449fc3f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.776399 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e47af616e1b4d2db3780c1d67cdcb47. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.777429 746 tablet_service.cc:1505] Processing CreateTablet for tablet d6923cfd826d495291d9ac8e0b3b44c6 (DEFAULT_TABLE table=default.Table_68ccb2052d824ccf92f7162d2961fcb1 [id=9ac9591d4f04482d926710fdf6e797c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.777849 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6923cfd826d495291d9ac8e0b3b44c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.778263 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f737a7bfb0f44196abdc831bb22d5285" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:38.780354 711 leader_election.cc:343] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:38.783731 1165 ts_tablet_manager.cc:1929] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:38.783862 1165 log.cc:1199] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/2dde7f28e61d41f48015ba6e4f3757f2
I20251211 20:52:38.784142 1165 ts_tablet_manager.cc:1950] T 2dde7f28e61d41f48015ba6e4f3757f2 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:38.784849 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 789) took 1322 ms. Trace:
I20251211 20:52:38.784935 1165 rpcz_store.cc:276] 1211 20:52:37.461941 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.461999 (+ 58us) service_pool.cc:225] Handling call
1211 20:52:38.521080 (+1059081us) ts_tablet_manager.cc:1151] Deleting tablet 2dde7f28e61d41f48015ba6e4f3757f2
1211 20:52:38.521094 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:38.673377 (+152283us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:38.750445 (+ 77068us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:38.784828 (+ 34383us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1012193}
I20251211 20:52:38.785449 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 2dde7f28e61d41f48015ba6e4f3757f2 (table default.Table_8b66a5217c494c8ebfa10a2b986e5e78 [id=499af44163bd46e7a93b9edcdd695ae7]) successfully deleted
I20251211 20:52:38.785859 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 790) took 1308 ms. Trace:
I20251211 20:52:38.785929 1165 rpcz_store.cc:276] 1211 20:52:37.477178 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.477539 (+ 361us) service_pool.cc:225] Handling call
1211 20:52:38.785737 (+1308198us) ts_tablet_manager.cc:1151] Deleting tablet 32f8d104f6a948d1a97a83769ea82449
1211 20:52:38.785748 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:38.785842 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1267682}
W20251211 20:52:38.786350 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 32f8d104f6a948d1a97a83769ea82449 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 32f8d104f6a948d1a97a83769ea82449 already in progress: creating tablet
I20251211 20:52:38.786700 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 797) took 1228 ms. Trace:
I20251211 20:52:38.786768 1165 rpcz_store.cc:276] 1211 20:52:37.558371 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.558433 (+ 62us) service_pool.cc:225] Handling call
1211 20:52:38.786599 (+1228166us) ts_tablet_manager.cc:1151] Deleting tablet cce44b80c97948e1af7ac68e77cccfd6
1211 20:52:38.786609 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:38.786684 (+ 75us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1227953}
W20251211 20:52:38.787132 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet cce44b80c97948e1af7ac68e77cccfd6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cce44b80c97948e1af7ac68e77cccfd6 already in progress: creating tablet
I20251211 20:52:38.787428 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 798) took 1199 ms. Trace:
I20251211 20:52:38.787863 595 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5] alter complete (version 2)
I20251211 20:52:38.790372 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f2d6c9781b4d4c37bc08703347963d5d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:38.791635 2170 tablet.cc:1717] T 8feee01fc0aa4b32be926ff2b617b894 P eb0f1933d9a74fccaa4618047fc05c41: Skipping requested alter to schema version 4, tablet already version 4
I20251211 20:52:38.791719 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 449921ea74f04a9c88eb9e094e8226c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:38.792824 1168 tablet_bootstrap.cc:492] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:38.792845 1164 ts_tablet_manager.cc:1929] T 3c578d81e8dd41aaa0f707c4e3fed225 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:38.792968 1168 ts_tablet_manager.cc:1403] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.110s user 0.003s sys 0.002s
I20251211 20:52:38.793716 1168 raft_consensus.cc:359] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.793900 1168 raft_consensus.cc:385] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:38.793954 1168 raft_consensus.cc:740] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:38.794175 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_18ec995eeab648dd8f0399b55e571ece"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:38.794780 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_18ec995eeab648dd8f0399b55e571ece in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.795231 1021 tablet_service.cc:1505] Processing CreateTablet for tablet c8bb92fc25124ba4be887c453abc7d9a (DEFAULT_TABLE table=default.Table_f2eaa62388894072a390cc4c167f3385 [id=ac6774893a6a4f82a7c9630bbe502fe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.796454 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8bb92fc25124ba4be887c453abc7d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.797591 1029 tablet_service.cc:1505] Processing CreateTablet for tablet d6923cfd826d495291d9ac8e0b3b44c6 (DEFAULT_TABLE table=default.Table_68ccb2052d824ccf92f7162d2961fcb1 [id=9ac9591d4f04482d926710fdf6e797c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.797986 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6923cfd826d495291d9ac8e0b3b44c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.801295 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f737a7bfb0f44196abdc831bb22d5285" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:38.801563 1048 raft_consensus.cc:3060] T f737a7bfb0f44196abdc831bb22d5285 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:38.787493 1165 rpcz_store.cc:276] 1211 20:52:37.588191 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.588380 (+ 189us) service_pool.cc:225] Handling call
1211 20:52:38.787347 (+1198967us) ts_tablet_manager.cc:1151] Deleting tablet 87863daf77304a399318cdf56da9f9bb
1211 20:52:38.787357 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:38.787413 (+ 56us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1198756}
W20251211 20:52:38.802492 581 catalog_manager.cc:4952] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 87863daf77304a399318cdf56da9f9bb because the tablet was not found. No further retry: Not found: Tablet not found: 87863daf77304a399318cdf56da9f9bb
I20251211 20:52:38.803033 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 799) took 1139 ms. Trace:
I20251211 20:52:38.803172 1165 rpcz_store.cc:276] 1211 20:52:37.663110 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.663271 (+ 161us) service_pool.cc:225] Handling call
1211 20:52:38.802881 (+1139610us) ts_tablet_manager.cc:1151] Deleting tablet c91ff29847d54e55b19321652a857e9f
1211 20:52:38.802895 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:38.803009 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1124111}
I20251211 20:52:38.803361 885 tablet_service.cc:1505] Processing CreateTablet for tablet b3ccde2b054841769a4c85f19f80f20a (DEFAULT_TABLE table=default.Table_7e3cf73eaea24e34b5dd291f8a738947 [id=e3ff2e8147aa41a99206b3961c29d751]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:38.803797 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet c91ff29847d54e55b19321652a857e9f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c91ff29847d54e55b19321652a857e9f already in progress: creating tablet
I20251211 20:52:38.804248 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 800) took 1141 ms. Trace:
I20251211 20:52:38.804366 1165 rpcz_store.cc:276] 1211 20:52:37.663220 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.663419 (+ 199us) service_pool.cc:225] Handling call
1211 20:52:38.804123 (+1140704us) ts_tablet_manager.cc:1151] Deleting tablet 098054406a1d459b824a094f50e71e7e
1211 20:52:38.804135 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:38.804230 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1125291}
W20251211 20:52:38.804931 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 098054406a1d459b824a094f50e71e7e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 098054406a1d459b824a094f50e71e7e already in progress: creating tablet
I20251211 20:52:38.805262 1165 tablet_replica.cc:333] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:38.805480 1165 raft_consensus.cc:2243] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:38.805740 1165 raft_consensus.cc:2272] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:38.806030 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3ccde2b054841769a4c85f19f80f20a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.806406 1165 ts_tablet_manager.cc:1916] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:38.811146 2100 tablet.cc:1722] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:38.814098 588 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet bfb52935296443c58190d4d06e2e6e9c (table default.Table_3721b5725e8c4297a392d3986a081128 [id=383392d835da4c50848068d83261cecc]). Expected version 7 got 0
I20251211 20:52:38.814687 891 tablet_service.cc:1505] Processing CreateTablet for tablet d6923cfd826d495291d9ac8e0b3b44c6 (DEFAULT_TABLE table=default.Table_68ccb2052d824ccf92f7162d2961fcb1 [id=9ac9591d4f04482d926710fdf6e797c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.815150 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6923cfd826d495291d9ac8e0b3b44c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.817984 1163 ts_tablet_manager.cc:1929] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:38.820655 1160 tablet_bootstrap.cc:492] T 97ada0641165444a998db26796793507 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:38.822365 1160 tablet_bootstrap.cc:654] T 97ada0641165444a998db26796793507 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:38.809746 746 tablet_service.cc:1505] Processing CreateTablet for tablet b3ccde2b054841769a4c85f19f80f20a (DEFAULT_TABLE table=default.Table_7e3cf73eaea24e34b5dd291f8a738947 [id=e3ff2e8147aa41a99206b3961c29d751]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:38.819005 710 leader_election.cc:343] T f2d6c9781b4d4c37bc08703347963d5d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:38.828204 710 leader_election.cc:304] T f2d6c9781b4d4c37bc08703347963d5d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:38.828306 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3ccde2b054841769a4c85f19f80f20a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.833397 589 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_5e5286ea174048f8ba9abd14daf14bcf [id=a455f6ef64ae44f78972d14b2ea9654c] alter complete (version 1)
I20251211 20:52:38.833771 589 master_service.cc:954] ReplaceTablet: received request to replace tablet 7452dd736b0440fcb4626858e794676f from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:38.822888 1168 consensus_queue.cc:260] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.834663 1168 ts_tablet_manager.cc:1434] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.042s user 0.001s sys 0.001s
I20251211 20:52:38.834992 1168 tablet_bootstrap.cc:492] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:38.836628 1168 tablet_bootstrap.cc:654] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:38.846961 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 80c566822fa94ada983aa7266df4bf9d
I20251211 20:52:38.853224 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_df5fb8b06973455d8c2b6a335e1f1d95" } modify_external_catalogs: true
I20251211 20:52:38.853521 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_df5fb8b06973455d8c2b6a335e1f1d95" } modify_external_catalogs: true
I20251211 20:52:38.845377 1048 raft_consensus.cc:2468] T f737a7bfb0f44196abdc831bb22d5285 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:38.856093 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 80c566822fa94ada983aa7266df4bf9d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:38.856841 1164 log.cc:1199] T 3c578d81e8dd41aaa0f707c4e3fed225 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/3c578d81e8dd41aaa0f707c4e3fed225
I20251211 20:52:38.857415 1164 ts_tablet_manager.cc:1950] T 3c578d81e8dd41aaa0f707c4e3fed225 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:38.836930 2093 raft_consensus.cc:2749] T f2d6c9781b4d4c37bc08703347963d5d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:38.837670 2019 consensus_queue.cc:1048] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:38.856657 2086 consensus_queue.cc:1048] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:38.857601 710 leader_election.cc:304] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:38.858814 2086 raft_consensus.cc:2804] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:38.858973 2086 raft_consensus.cc:697] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:38.859314 2086 consensus_queue.cc:237] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.859688 1163 log.cc:1199] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/7474a08c51cf4346ae9a518963d56b7a
I20251211 20:52:38.860702 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_d65f65d608a346968cc2f35b9f55912e" } modify_external_catalogs: true
I20251211 20:52:38.860855 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_d65f65d608a346968cc2f35b9f55912e" } modify_external_catalogs: true
I20251211 20:52:38.854611 595 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_bcd4dcb9828242a18cc8ca757e2180ba" } new_table_name: "default.Table_fba99384e9f14160b3e3504f8d1d0b07" modify_external_catalogs: true
W20251211 20:52:38.863310 711 consensus_peers.cc:597] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:38.855038 593 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 5bd801124df84a6c96f4912993f1bdfd (table default.Table_80e3542d21b64eca991b2e81ac80ac7c [id=44f2f2bf53af4b038ced294f20db9e48]). Expected version 2 got 0
I20251211 20:52:38.855321 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_e54af82590454fdf941b0462c30738ad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:38.865051 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e54af82590454fdf941b0462c30738ad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.866858 594 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 5bd801124df84a6c96f4912993f1bdfd (table default.Table_80e3542d21b64eca991b2e81ac80ac7c [id=44f2f2bf53af4b038ced294f20db9e48]). Expected version 2 got 0
I20251211 20:52:38.867105 594 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 1a77e3e744ff42ccb68897704b4c2f1d (table default.Table_d70f80161dcd4063815a0e079c5821de [id=85ab3a18cda143e5a7f72e49e6671318]). Expected version 1 got 0
I20251211 20:52:38.858819 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_2198fd77ec454943a0c0a392964c1fd6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:38.868106 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2198fd77ec454943a0c0a392964c1fd6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.859215 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c3617a747d9d4e4294df0b575520d39f" } modify_external_catalogs: true
I20251211 20:52:38.859284 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 3c578d81e8dd41aaa0f707c4e3fed225 (table default.Table_6d94053661f142b79292e19752df02df [id=c5be59a2c3934b3789641ceeb2d4d6e2]) successfully deleted
I20251211 20:52:38.869284 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c3617a747d9d4e4294df0b575520d39f" } modify_external_catalogs: true
I20251211 20:52:38.859593 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c6c5666f7b3044589afd38743a1a200c" } modify_external_catalogs: true
I20251211 20:52:38.870471 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c6c5666f7b3044589afd38743a1a200c" } modify_external_catalogs: true
I20251211 20:52:38.873873 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 80c566822fa94ada983aa7266df4bf9d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:38.874150 1163 ts_tablet_manager.cc:1950] T 7474a08c51cf4346ae9a518963d56b7a P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:38.874296 741 tablet_service.cc:1552] Processing DeleteTablet for tablet c74427cb9d6f4a679e43eae796ff8c3f with delete_type TABLET_DATA_DELETED (replaced by tablet 4e123b96b7534f07ae48a808f47e21a1) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:38.859503 1164 tablet_replica.cc:333] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:38.875834 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 746) took 2355 ms. Trace:
I20251211 20:52:38.875923 1163 rpcz_store.cc:276] 1211 20:52:36.520378 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:36.520647 (+ 269us) service_pool.cc:225] Handling call
1211 20:52:38.753592 (+2232945us) ts_tablet_manager.cc:1151] Deleting tablet 7474a08c51cf4346ae9a518963d56b7a
1211 20:52:38.753604 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:38.815130 (+ 61526us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:38.817893 (+ 2763us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:38.875809 (+ 57916us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2123875}
I20251211 20:52:38.876394 1163 tablet_replica.cc:333] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:38.876561 1164 raft_consensus.cc:2243] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:38.876720 1163 raft_consensus.cc:2243] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:38.876749 1164 raft_consensus.cc:2272] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:38.877277 1163 raft_consensus.cc:2272] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:38.877884 1163 ts_tablet_manager.cc:1916] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:38.878103 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 7474a08c51cf4346ae9a518963d56b7a (table default.Table_d65f65d608a346968cc2f35b9f55912e [id=88e8b1b4524846e18f5a638dee69a688]) successfully deleted
W20251211 20:52:38.878444 711 consensus_peers.cc:597] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:38.880743 754 tablet_service.cc:1505] Processing CreateTablet for tablet b2974f16034c44cf85ff85b5095cf6a0 (DEFAULT_TABLE table=default.Table_031de0f4cc7744bc9e320261f8b46696 [id=be36dabaa6d44325b4855b9f13bda847]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.881721 2093 consensus_queue.cc:1048] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:38.883059 589 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 1a77e3e744ff42ccb68897704b4c2f1d (table default.Table_d70f80161dcd4063815a0e079c5821de [id=85ab3a18cda143e5a7f72e49e6671318]). Expected version 1 got 0
I20251211 20:52:38.883255 589 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b072b759d1254859b0b8d401e3078792 (table default.Table_b56de1e85a1b42ea81308593884dadb8 [id=a137f0ddf52145a99c62444602ab3fdb]). Expected version 4 got 0
I20251211 20:52:38.883934 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2974f16034c44cf85ff85b5095cf6a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.884820 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0cc01b2a751d4591b9b227f46093e552" } new_table_name: "default.Table_4e8d74eff9674d67b5b0f54370b06598" modify_external_catalogs: true
I20251211 20:52:38.894240 877 tablet_service.cc:1505] Processing CreateTablet for tablet b2974f16034c44cf85ff85b5095cf6a0 (DEFAULT_TABLE table=default.Table_031de0f4cc7744bc9e320261f8b46696 [id=be36dabaa6d44325b4855b9f13bda847]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.895846 1168 tablet_bootstrap.cc:492] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:38.897718 1168 ts_tablet_manager.cc:1403] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.063s user 0.002s sys 0.002s
I20251211 20:52:38.899063 1168 raft_consensus.cc:359] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.904438 589 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 1a77e3e744ff42ccb68897704b4c2f1d (table default.Table_d70f80161dcd4063815a0e079c5821de [id=85ab3a18cda143e5a7f72e49e6671318]). Expected version 1 got 0
I20251211 20:52:38.904711 589 catalog_manager.cc:5654] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:38.905143 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2974f16034c44cf85ff85b5095cf6a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.905216 589 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 9f8aa28c1c9b4fc58dfc128952c180ba (table default.Table_bf7d402a36be44cfb6b79041906d26a1 [id=0327e686e7ed4475a3e8cda8721a3a3a]). Expected version 1 got 0
I20251211 20:52:38.899708 741 tablet_service.cc:1505] Processing CreateTablet for tablet ac8fb4234af54acc8b250a081e8aeecc (DEFAULT_TABLE table=default.Table_4e2d2e9566764ee9bc5edf65d0e693a2 [id=d3c56e51e2464bafb3819af545f21a3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.906002 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac8fb4234af54acc8b250a081e8aeecc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.878383 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 52ef1707f6f2425aafd454f303af0333 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:38.881640 1015 tablet_service.cc:1505] Processing CreateTablet for tablet b3ccde2b054841769a4c85f19f80f20a (DEFAULT_TABLE table=default.Table_7e3cf73eaea24e34b5dd291f8a738947 [id=e3ff2e8147aa41a99206b3961c29d751]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.907272 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3ccde2b054841769a4c85f19f80f20a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.907747 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_0d3816dfd69844518591138a3a844a9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:38.908376 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0d3816dfd69844518591138a3a844a9a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:38.908548 1168 raft_consensus.cc:385] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:38.908864 1168 raft_consensus.cc:740] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:38.905896 1021 tablet_service.cc:1552] Processing DeleteTablet for tablet 80c566822fa94ada983aa7266df4bf9d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:38.900107 888 tablet_service.cc:1505] Processing CreateTablet for tablet ac8fb4234af54acc8b250a081e8aeecc (DEFAULT_TABLE table=default.Table_4e2d2e9566764ee9bc5edf65d0e693a2 [id=d3c56e51e2464bafb3819af545f21a3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.912196 2086 raft_consensus.cc:493] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:38.912652 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac8fb4234af54acc8b250a081e8aeecc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.916019 1164 ts_tablet_manager.cc:1916] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:38.918951 885 tablet_service.cc:1505] Processing CreateTablet for tablet 8dc4e730951144d78b77280f6cce6d0e (DEFAULT_TABLE table=default.Table_890c9027415a44ccaf95c0adede51992 [id=7bc8a2b91fe54b91b6594b371fea7bd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.919348 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dc4e730951144d78b77280f6cce6d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.921538 889 tablet_service.cc:1505] Processing CreateTablet for tablet 39e054bb3dc54c3d89972e6844b580e1 (DEFAULT_TABLE table=default.Table_6b8536dae8ce48be97d778489d1da99c [id=50493a9594aa423cb79ed4f4235a5e4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.921945 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39e054bb3dc54c3d89972e6844b580e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.922518 746 tablet_service.cc:1505] Processing CreateTablet for tablet 8dc4e730951144d78b77280f6cce6d0e (DEFAULT_TABLE table=default.Table_890c9027415a44ccaf95c0adede51992 [id=7bc8a2b91fe54b91b6594b371fea7bd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.923152 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dc4e730951144d78b77280f6cce6d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.923899 1165 ts_tablet_manager.cc:1929] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20251211 20:52:38.924027 1165 log.cc:1199] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/3eac21a07ba34e1fa275e25ee9dcabb1
I20251211 20:52:38.924391 1165 ts_tablet_manager.cc:1950] T 3eac21a07ba34e1fa275e25ee9dcabb1 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:38.924729 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 802) took 1080 ms. Trace:
I20251211 20:52:38.924822 1165 rpcz_store.cc:276] 1211 20:52:37.844053 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.844139 (+ 86us) service_pool.cc:225] Handling call
1211 20:52:38.805220 (+961081us) ts_tablet_manager.cc:1151] Deleting tablet 3eac21a07ba34e1fa275e25ee9dcabb1
1211 20:52:38.805229 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:38.860248 (+ 55019us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:38.863287 (+ 3039us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:38.924713 (+ 61426us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":960899}
I20251211 20:52:38.925223 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 803) took 1013 ms. Trace:
I20251211 20:52:38.925354 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 3eac21a07ba34e1fa275e25ee9dcabb1 (table default.Table_c6c5666f7b3044589afd38743a1a200c [id=9700f9fe6e3044748e34954bef8a0038]) successfully deleted
I20251211 20:52:38.925351 1165 rpcz_store.cc:276] 1211 20:52:37.912074 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.912132 (+ 58us) service_pool.cc:225] Handling call
1211 20:52:38.925100 (+1012968us) ts_tablet_manager.cc:1151] Deleting tablet 042f8bf13a7a4712b5bdc04671cd9938
1211 20:52:38.925109 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:38.925207 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1012739}
I20251211 20:52:38.926139 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 39e054bb3dc54c3d89972e6844b580e1 (DEFAULT_TABLE table=default.Table_6b8536dae8ce48be97d778489d1da99c [id=50493a9594aa423cb79ed4f4235a5e4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.926538 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39e054bb3dc54c3d89972e6844b580e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.926924 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet e95c63b4d99a4dd0b2c0fa384ca9d552
I20251211 20:52:38.927060 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 8dc4e730951144d78b77280f6cce6d0e (DEFAULT_TABLE table=default.Table_890c9027415a44ccaf95c0adede51992 [id=7bc8a2b91fe54b91b6594b371fea7bd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.927814 1022 tablet_service.cc:1505] Processing CreateTablet for tablet b2974f16034c44cf85ff85b5095cf6a0 (DEFAULT_TABLE table=default.Table_031de0f4cc7744bc9e320261f8b46696 [id=be36dabaa6d44325b4855b9f13bda847]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.927838 884 tablet_service.cc:1552] Processing DeleteTablet for tablet e95c63b4d99a4dd0b2c0fa384ca9d552 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:38.932775 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2974f16034c44cf85ff85b5095cf6a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.933794 1021 tablet_service.cc:1505] Processing CreateTablet for tablet ac8fb4234af54acc8b250a081e8aeecc (DEFAULT_TABLE table=default.Table_4e2d2e9566764ee9bc5edf65d0e693a2 [id=d3c56e51e2464bafb3819af545f21a3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.934226 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac8fb4234af54acc8b250a081e8aeecc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.934396 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dc4e730951144d78b77280f6cce6d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.934407 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_d70f80161dcd4063815a0e079c5821de" } new_table_name: "default.Table_d17d0534f0544cef99924179cb1376b0" modify_external_catalogs: true
I20251211 20:52:38.934587 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet e95c63b4d99a4dd0b2c0fa384ca9d552 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:38.968596 1160 tablet_bootstrap.cc:492] T 97ada0641165444a998db26796793507 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:38.968773 1160 ts_tablet_manager.cc:1403] T 97ada0641165444a998db26796793507 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.148s user 0.003s sys 0.003s
I20251211 20:52:38.969561 1160 raft_consensus.cc:359] T 97ada0641165444a998db26796793507 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.969748 1160 raft_consensus.cc:385] T 97ada0641165444a998db26796793507 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:38.970111 1160 raft_consensus.cc:740] T 97ada0641165444a998db26796793507 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:38.970345 1160 consensus_queue.cc:260] T 97ada0641165444a998db26796793507 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.970834 1160 ts_tablet_manager.cc:1434] T 97ada0641165444a998db26796793507 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:38.973243 1160 tablet_bootstrap.cc:492] T 3cd7a465249246c28f07b8fb2e42675f P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:38.973480 1166 tablet_bootstrap.cc:492] T bfb52935296443c58190d4d06e2e6e9c P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:38.973659 1166 ts_tablet_manager.cc:1403] T bfb52935296443c58190d4d06e2e6e9c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.229s user 0.004s sys 0.008s
I20251211 20:52:38.974462 1166 raft_consensus.cc:359] T bfb52935296443c58190d4d06e2e6e9c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.974655 1166 raft_consensus.cc:385] T bfb52935296443c58190d4d06e2e6e9c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:38.974704 1166 raft_consensus.cc:740] T bfb52935296443c58190d4d06e2e6e9c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:38.974954 1160 tablet_bootstrap.cc:654] T 3cd7a465249246c28f07b8fb2e42675f P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:38.979717 1166 consensus_queue.cc:260] T bfb52935296443c58190d4d06e2e6e9c P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.927649 747 tablet_service.cc:1552] Processing DeleteTablet for tablet e95c63b4d99a4dd0b2c0fa384ca9d552 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:38.934399 711 consensus_peers.cc:597] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:38.991252 711 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 56891.4us
W20251211 20:52:38.991712 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 042f8bf13a7a4712b5bdc04671cd9938 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 042f8bf13a7a4712b5bdc04671cd9938 already in progress: creating tablet
I20251211 20:52:38.950663 2086 raft_consensus.cc:515] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.992705 2086 leader_election.cc:290] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:38.952653 739 tablet_service.cc:1505] Processing CreateTablet for tablet 39e054bb3dc54c3d89972e6844b580e1 (DEFAULT_TABLE table=default.Table_6b8536dae8ce48be97d778489d1da99c [id=50493a9594aa423cb79ed4f4235a5e4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:38.993149 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39e054bb3dc54c3d89972e6844b580e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:38.995754 1813 raft_consensus.cc:493] T 5bd801124df84a6c96f4912993f1bdfd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:38.991523 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 804) took 1075 ms. Trace:
I20251211 20:52:38.995941 1813 raft_consensus.cc:515] T 5bd801124df84a6c96f4912993f1bdfd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:38.995950 1165 rpcz_store.cc:276] 1211 20:52:37.915695 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.915752 (+ 57us) service_pool.cc:225] Handling call
1211 20:52:38.991386 (+1075634us) ts_tablet_manager.cc:1151] Deleting tablet 14f367adb60744c8a593d3b9d4711823
1211 20:52:38.991396 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:38.991505 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1075426}
W20251211 20:52:38.996440 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 14f367adb60744c8a593d3b9d4711823 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14f367adb60744c8a593d3b9d4711823 already in progress: creating tablet
I20251211 20:52:38.996585 1813 leader_election.cc:290] T 5bd801124df84a6c96f4912993f1bdfd P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:38.998211 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5bd801124df84a6c96f4912993f1bdfd" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:38.998525 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bfb52935296443c58190d4d06e2e6e9c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:38.999975 711 leader_election.cc:343] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:39.000214 711 consensus_peers.cc:597] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:39.000504 2086 raft_consensus.cc:493] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:39.000685 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 805) took 1079 ms. Trace:
I20251211 20:52:39.000636 2086 raft_consensus.cc:515] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.000761 1165 rpcz_store.cc:276] 1211 20:52:37.921326 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.921386 (+ 60us) service_pool.cc:225] Handling call
1211 20:52:39.000573 (+1079187us) ts_tablet_manager.cc:1151] Deleting tablet 2197822e0f11484f9679c4858626ac2c
1211 20:52:39.000584 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.000671 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1078930}
W20251211 20:52:39.001194 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 2197822e0f11484f9679c4858626ac2c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2197822e0f11484f9679c4858626ac2c already in progress: creating tablet
I20251211 20:52:39.001245 2086 leader_election.cc:290] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:39.001688 711 consensus_peers.cc:597] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:39.002130 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a2058b67ddf1413db338bfa2f64e7a77" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:39.002740 711 leader_election.cc:343] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.004014 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 806) took 1080 ms. Trace:
W20251211 20:52:39.004690 982 consensus_peers.cc:597] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 -> Peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Couldn't send request to peer 6f8d784fd1cd492bbe3809403ec34919. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3eac21a07ba34e1fa275e25ee9dcabb1. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:39.007575 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bfb52935296443c58190d4d06e2e6e9c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:39.008140 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a2058b67ddf1413db338bfa2f64e7a77" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:39.008284 1047 raft_consensus.cc:2468] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:39.008301 1048 raft_consensus.cc:2468] T bfb52935296443c58190d4d06e2e6e9c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:39.009557 2100 tablet.cc:1722] T 5bd801124df84a6c96f4912993f1bdfd P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20251211 20:52:39.017868 595 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e89d4d76bf64453892a4d0b55fcc3454 consensus state has no leader
I20251211 20:52:39.026274 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet d15eff49af284a38bb1babdb231a35df
I20251211 20:52:39.027542 1021 tablet_service.cc:1552] Processing DeleteTablet for tablet d15eff49af284a38bb1babdb231a35df with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:39.028326 595 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 8229b704e7464b30b7d2902a938ce33b (table default.Table_b008891c4e894b02aa0db32d1740371a [id=8fe47ac33eb2461784b859e64fbd3e24]). Expected version 5 got 0
I20251211 20:52:39.028656 595 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b072b759d1254859b0b8d401e3078792 (table default.Table_b56de1e85a1b42ea81308593884dadb8 [id=a137f0ddf52145a99c62444602ab3fdb]). Expected version 4 got 0
I20251211 20:52:39.033375 1164 ts_tablet_manager.cc:1929] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:39.033524 1164 log.cc:1199] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/bc32a778e5424850b23ccaf74bb9df07
I20251211 20:52:39.033872 1164 ts_tablet_manager.cc:1950] T bc32a778e5424850b23ccaf74bb9df07 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:39.034605 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 775) took 1094 ms. Trace:
I20251211 20:52:39.034705 1164 rpcz_store.cc:276] 1211 20:52:37.939810 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.940128 (+ 318us) service_pool.cc:225] Handling call
1211 20:52:38.859446 (+919318us) ts_tablet_manager.cc:1151] Deleting tablet bc32a778e5424850b23ccaf74bb9df07
1211 20:52:38.859459 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:38.987726 (+128267us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.033250 (+ 45524us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.034582 (+ 1332us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":919204}
I20251211 20:52:39.035084 1164 tablet_replica.cc:333] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:39.035247 1164 raft_consensus.cc:2243] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:39.035282 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet bc32a778e5424850b23ccaf74bb9df07 (table default.Table_e6252813479645a98ce2709dc7aba798 [id=db38368266194271a02ad265fd991439]) successfully deleted
I20251211 20:52:39.035398 1164 raft_consensus.cc:2272] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:39.037837 596 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_54d8f0fdaacf4be8b1043f2bae13c56e" } new_table_name: "default.Table_2aa7ff9186a04249abf8cd971cc0cb17" modify_external_catalogs: true
I20251211 20:52:39.039311 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_90271d921fcd4221a1ab532ae40b2f9f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:39.039975 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_90271d921fcd4221a1ab532ae40b2f9f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:39.041617 877 tablet_service.cc:1552] Processing DeleteTablet for tablet 97ada0641165444a998db26796793507 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:39.054392 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b9de49d730424e2b863b64535f2ba732"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:39.055011 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b9de49d730424e2b863b64535f2ba732 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:39.055510 1043 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5bd801124df84a6c96f4912993f1bdfd" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:39.057240 847 leader_election.cc:304] T 5bd801124df84a6c96f4912993f1bdfd P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:39.057617 888 tablet_service.cc:1552] Processing DeleteTablet for tablet d15eff49af284a38bb1babdb231a35df with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:39.057672 1813 raft_consensus.cc:2749] T 5bd801124df84a6c96f4912993f1bdfd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:39.058928 2135 raft_consensus.cc:493] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:39.059067 2135 raft_consensus.cc:515] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.060149 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b631e6b3048e428f901af1b5b0b1ab3c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:39.060724 983 leader_election.cc:343] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.061214 2135 leader_election.cc:290] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:39.065816 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet c97d410272a947de8b5d7fb40fe4777d
I20251211 20:52:39.066931 888 tablet_service.cc:1552] Processing DeleteTablet for tablet c97d410272a947de8b5d7fb40fe4777d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:39.067101 1160 tablet_bootstrap.cc:492] T 3cd7a465249246c28f07b8fb2e42675f P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:39.067144 1021 tablet_service.cc:1552] Processing DeleteTablet for tablet c97d410272a947de8b5d7fb40fe4777d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:39.067229 1160 ts_tablet_manager.cc:1403] T 3cd7a465249246c28f07b8fb2e42675f P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.094s user 0.003s sys 0.000s
I20251211 20:52:39.067749 597 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_5a2e3b6cc2734852af9a0a9b8f51a69b" } new_table_name: "default.Table_2cff6fab088c4d939b13b504c133cc12" modify_external_catalogs: true
I20251211 20:52:39.068013 1160 raft_consensus.cc:359] T 3cd7a465249246c28f07b8fb2e42675f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.068161 1160 raft_consensus.cc:385] T 3cd7a465249246c28f07b8fb2e42675f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:39.068204 1160 raft_consensus.cc:740] T 3cd7a465249246c28f07b8fb2e42675f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:39.068424 1160 consensus_queue.cc:260] T 3cd7a465249246c28f07b8fb2e42675f P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.068907 1160 ts_tablet_manager.cc:1434] T 3cd7a465249246c28f07b8fb2e42675f P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:39.080600 592 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet 2085f871ea0a4caebbf2b1459b8c1e3b
I20251211 20:52:38.909662 1168 consensus_queue.cc:260] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.083671 1165 rpcz_store.cc:276] 1211 20:52:37.924020 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.926427 (+ 2407us) service_pool.cc:225] Handling call
1211 20:52:39.003900 (+1077473us) ts_tablet_manager.cc:1151] Deleting tablet 2ef524747034491c99f4f9767acd3c9d
1211 20:52:39.003912 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.003997 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1077255}
I20251211 20:52:39.084098 1168 ts_tablet_manager.cc:1434] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.186s user 0.000s sys 0.002s
I20251211 20:52:39.084228 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_d15e45922b4340a59fce3b9dc10faf71"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:39.084417 1168 tablet_bootstrap.cc:492] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:39.084371 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 2ef524747034491c99f4f9767acd3c9d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2ef524747034491c99f4f9767acd3c9d already in progress: creating tablet
I20251211 20:52:39.084756 1165 tablet_replica.cc:333] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:39.084941 1165 raft_consensus.cc:2243] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:39.085280 1165 raft_consensus.cc:2272] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:39.085392 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d15e45922b4340a59fce3b9dc10faf71 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:39.085971 1168 tablet_bootstrap.cc:654] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:39.086284 1165 ts_tablet_manager.cc:1916] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:39.008939 2168 tablet.cc:1722] T 5bd801124df84a6c96f4912993f1bdfd P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:39.009446 2093 consensus_queue.cc:1048] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:39.009709 710 leader_election.cc:304] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:39.021196 2101 raft_consensus.cc:493] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:39.089504 2101 raft_consensus.cc:515] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.027321 739 tablet_service.cc:1552] Processing DeleteTablet for tablet d15eff49af284a38bb1babdb231a35df with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:39.061530 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b631e6b3048e428f901af1b5b0b1ab3c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:39.066946 747 tablet_service.cc:1552] Processing DeleteTablet for tablet c97d410272a947de8b5d7fb40fe4777d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:39.090255 776 raft_consensus.cc:2468] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 0.
I20251211 20:52:39.090636 2101 leader_election.cc:290] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:39.090778 2101 raft_consensus.cc:2804] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:39.090832 2101 raft_consensus.cc:493] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:39.090883 2101 raft_consensus.cc:3060] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.090821 982 leader_election.cc:304] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:39.089324 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 79638.8us
I20251211 20:52:39.091406 2135 raft_consensus.cc:2804] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:39.091483 2135 raft_consensus.cc:493] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:39.091513 710 leader_election.cc:304] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:39.091539 2135 raft_consensus.cc:3060] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.092289 2019 raft_consensus.cc:2804] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:39.092353 2019 raft_consensus.cc:493] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:39.092403 2019 raft_consensus.cc:3060] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.093212 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "72e23f3033e14c9db6de8539830bac3b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:39.093292 2101 raft_consensus.cc:515] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.093380 1048 raft_consensus.cc:2468] T 72e23f3033e14c9db6de8539830bac3b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:39.093758 710 leader_election.cc:304] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:39.093992 2093 raft_consensus.cc:2804] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:39.094059 2093 raft_consensus.cc:493] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:39.094110 2093 raft_consensus.cc:3060] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.094653 2019 raft_consensus.cc:515] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.094950 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a2058b67ddf1413db338bfa2f64e7a77" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:39.095104 1048 raft_consensus.cc:3060] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.095167 2019 leader_election.cc:290] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:39.096354 2093 raft_consensus.cc:515] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.096844 2093 leader_election.cc:290] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:39.097484 1048 raft_consensus.cc:2468] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:39.098230 2093 consensus_queue.cc:1048] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:39.098608 2101 leader_election.cc:290] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:39.098248 710 leader_election.cc:304] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:39.098683 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "72e23f3033e14c9db6de8539830bac3b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:39.098845 1047 raft_consensus.cc:3060] T 72e23f3033e14c9db6de8539830bac3b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.098886 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bfb52935296443c58190d4d06e2e6e9c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:39.099023 1048 raft_consensus.cc:3060] T bfb52935296443c58190d4d06e2e6e9c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.101558 1047 raft_consensus.cc:2468] T 72e23f3033e14c9db6de8539830bac3b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:39.101644 1048 raft_consensus.cc:2468] T bfb52935296443c58190d4d06e2e6e9c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:39.102098 710 leader_election.cc:304] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:39.102416 710 leader_election.cc:304] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:39.102641 2019 raft_consensus.cc:2804] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:39.102751 2019 raft_consensus.cc:697] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:39.102825 2101 raft_consensus.cc:2804] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:39.103017 2101 raft_consensus.cc:697] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:39.102979 2019 consensus_queue.cc:237] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.103201 2101 consensus_queue.cc:237] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.102649 2093 raft_consensus.cc:2804] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:39.104224 2093 raft_consensus.cc:697] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:39.104434 2093 consensus_queue.cc:237] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.108829 594 master_service.cc:954] ReplaceTablet: received request to replace tablet 3c578d81e8dd41aaa0f707c4e3fed225 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:39.110224 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ed9d95ed154245de80d2cdc5f1eb5bd3" } modify_external_catalogs: true
I20251211 20:52:39.110443 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ed9d95ed154245de80d2cdc5f1eb5bd3" } modify_external_catalogs: true
I20251211 20:52:39.111933 1163 ts_tablet_manager.cc:1929] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251211 20:52:39.112059 1163 log.cc:1199] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/c4e048ca28824329b924845ccf5067cf
I20251211 20:52:39.114576 1166 ts_tablet_manager.cc:1434] T bfb52935296443c58190d4d06e2e6e9c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.141s user 0.016s sys 0.020s
I20251211 20:52:39.114925 1166 tablet_bootstrap.cc:492] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:39.115875 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bfb52935296443c58190d4d06e2e6e9c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:39.116271 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "72e23f3033e14c9db6de8539830bac3b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:39.115882 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "72e23f3033e14c9db6de8539830bac3b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:39.115896 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a2058b67ddf1413db338bfa2f64e7a77" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:39.117970 2135 raft_consensus.cc:515] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:39.118146 711 leader_election.cc:343] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:39.118467 711 leader_election.cc:343] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:39.118709 711 leader_election.cc:343] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.118916 2135 leader_election.cc:290] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
W20251211 20:52:39.118945 711 leader_election.cc:343] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.119450 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b631e6b3048e428f901af1b5b0b1ab3c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:39.119627 776 raft_consensus.cc:3060] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.122128 776 raft_consensus.cc:2468] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 1.
I20251211 20:52:39.122563 982 leader_election.cc:304] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:39.122836 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b631e6b3048e428f901af1b5b0b1ab3c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:39.123333 983 leader_election.cc:343] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.124570 1166 tablet_bootstrap.cc:654] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:39.128561 1160 tablet_bootstrap.cc:492] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:39.130383 1160 tablet_bootstrap.cc:654] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:39.137992 2202 log.cc:927] Time spent T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Append to log took a long time: real 0.134s user 0.002s sys 0.000s
I20251211 20:52:39.143880 2135 raft_consensus.cc:2804] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:39.144181 2135 raft_consensus.cc:697] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 LEADER]: Becoming Leader. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Running, Role: LEADER
I20251211 20:52:39.144518 2135 consensus_queue.cc:237] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.151721 1166 tablet_bootstrap.cc:492] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
W20251211 20:52:39.153167 421 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 79954.6us
I20251211 20:52:39.153877 1166 ts_tablet_manager.cc:1403] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.039s user 0.000s sys 0.004s
I20251211 20:52:39.156996 1163 ts_tablet_manager.cc:1950] T c4e048ca28824329b924845ccf5067cf P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:39.157414 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 745) took 2637 ms. Trace:
I20251211 20:52:39.157570 1163 rpcz_store.cc:276] 1211 20:52:36.520289 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:36.520589 (+ 300us) service_pool.cc:225] Handling call
1211 20:52:38.876343 (+2355754us) ts_tablet_manager.cc:1151] Deleting tablet c4e048ca28824329b924845ccf5067cf
1211 20:52:38.876355 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:38.912275 (+ 35920us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.111804 (+199529us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.157398 (+ 45594us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2246524}
I20251211 20:52:39.158224 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 744) took 2638 ms. Trace:
I20251211 20:52:39.158242 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_beb19ebcfc3b426e9aa9f70a68445f72"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:39.158365 1163 rpcz_store.cc:276] 1211 20:52:36.520186 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:36.521359 (+ 1173us) service_pool.cc:225] Handling call
1211 20:52:39.158095 (+2636736us) ts_tablet_manager.cc:1151] Deleting tablet 76cfb26762d54593a150ec98125c6592
1211 20:52:39.158105 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.158209 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2527722}
I20251211 20:52:39.158483 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet c4e048ca28824329b924845ccf5067cf (table default.Table_f541a176465249d8aaba954bc23bcf01 [id=6b55ad645bec4a7daefc996978420295]) successfully deleted
W20251211 20:52:39.158782 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 76cfb26762d54593a150ec98125c6592 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 76cfb26762d54593a150ec98125c6592 already in progress: creating tablet
W20251211 20:52:39.158897 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_beb19ebcfc3b426e9aa9f70a68445f72 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:39.161763 1829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48310 (request call id 0) took 4685 ms. Trace:
I20251211 20:52:39.162009 2107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9230f94a6a1a49cead51aeea9f069087. Latest consensus state: current_term: 1 leader_uuid: "9230f94a6a1a49cead51aeea9f069087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } } }
I20251211 20:52:39.158761 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 751) took 2490 ms. Trace:
I20251211 20:52:39.162240 2107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [sys.catalog]: This master's current role is: FOLLOWER
I20251211 20:52:39.162233 1163 rpcz_store.cc:276] 1211 20:52:36.667956 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:36.668103 (+ 147us) service_pool.cc:225] Handling call
1211 20:52:39.158666 (+2490563us) ts_tablet_manager.cc:1151] Deleting tablet bbe3529c301a4a65b58b634a443e1fb6
1211 20:52:39.158674 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.158748 (+ 74us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2480312}
W20251211 20:52:39.162701 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet bbe3529c301a4a65b58b634a443e1fb6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bbe3529c301a4a65b58b634a443e1fb6 already in progress: creating tablet
I20251211 20:52:39.162950 2161 raft_consensus.cc:493] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9230f94a6a1a49cead51aeea9f069087)
I20251211 20:52:39.163172 2161 raft_consensus.cc:515] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:39.164382 2232 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20251211 20:52:39.165212 909 raft_consensus.cc:3060] T 5bd801124df84a6c96f4912993f1bdfd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.167191 2168 tablet.cc:1722] T 8229b704e7464b30b7d2902a938ce33b P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20251211 20:52:39.168094 2161 leader_election.cc:290] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1830b0c58cfe495989f3d911dda0409a (127.0.104.62:41123), 9230f94a6a1a49cead51aeea9f069087 (127.0.104.60:42087)
I20251211 20:52:39.168215 2100 tablet.cc:1722] T 8229b704e7464b30b7d2902a938ce33b P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20251211 20:52:39.160341 1166 raft_consensus.cc:359] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.172528 1166 raft_consensus.cc:385] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:39.172737 1166 raft_consensus.cc:740] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:39.172964 2101 raft_consensus.cc:493] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:39.173200 2101 raft_consensus.cc:515] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.174283 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c74427cb9d6f4a679e43eae796ff8c3f" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:39.174893 711 leader_election.cc:343] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.175393 2101 leader_election.cc:290] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:39.175673 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c74427cb9d6f4a679e43eae796ff8c3f" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:39.176172 1042 raft_consensus.cc:2468] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:39.176743 710 leader_election.cc:304] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:39.176983 2101 raft_consensus.cc:2804] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:39.177043 2101 raft_consensus.cc:493] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:39.177098 2101 raft_consensus.cc:3060] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.176160 1166 consensus_queue.cc:260] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.177503 1166 ts_tablet_manager.cc:1434] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.023s user 0.001s sys 0.001s
I20251211 20:52:39.177799 1166 tablet_bootstrap.cc:492] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:39.179458 1166 tablet_bootstrap.cc:654] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:39.179724 2101 raft_consensus.cc:515] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.181090 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c74427cb9d6f4a679e43eae796ff8c3f" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:39.182828 711 leader_election.cc:343] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.183231 1809 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" instance_seqno: 1765486340106790) as {username='slave'} at 127.0.104.1:55697; Asking this server to re-register.
I20251211 20:52:39.184703 2101 leader_election.cc:290] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:39.185220 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c74427cb9d6f4a679e43eae796ff8c3f" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:39.186133 1042 raft_consensus.cc:3060] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.188082 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 3cd7a465249246c28f07b8fb2e42675f with delete_type TABLET_DATA_DELETED (replaced by tablet 0763a475527546bb83cee8fb823893ac) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:39.189054 1042 raft_consensus.cc:2468] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:39.189373 1168 tablet_bootstrap.cc:492] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:39.189539 1168 ts_tablet_manager.cc:1403] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.105s user 0.003s sys 0.008s
I20251211 20:52:39.189460 710 leader_election.cc:304] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:39.189913 2101 raft_consensus.cc:2804] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:39.190039 2101 raft_consensus.cc:697] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:39.190289 2101 consensus_queue.cc:237] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.191629 1160 tablet_bootstrap.cc:492] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:39.191844 1160 ts_tablet_manager.cc:1403] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.063s user 0.002s sys 0.002s
I20251211 20:52:39.193954 2170 tablet.cc:1722] T 5bd801124df84a6c96f4912993f1bdfd P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:39.190337 1168 raft_consensus.cc:359] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.195216 1168 raft_consensus.cc:385] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:39.195302 1168 raft_consensus.cc:740] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:39.191387 824 heartbeater.cc:461] Registering TS with master...
I20251211 20:52:39.196139 1168 consensus_queue.cc:260] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.196632 1168 ts_tablet_manager.cc:1434] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20251211 20:52:39.196904 1168 tablet_bootstrap.cc:492] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:39.199878 1168 tablet_bootstrap.cc:654] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:39.205063 1809 ts_manager.cc:194] Registered new tserver with Master: 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:39.209671 1164 ts_tablet_manager.cc:1916] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:39.219945 2135 raft_consensus.cc:493] T 676faec7f3d1488f99b220f8e968e9a0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:39.220264 2135 raft_consensus.cc:515] T 676faec7f3d1488f99b220f8e968e9a0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.221136 2135 leader_election.cc:290] T 676faec7f3d1488f99b220f8e968e9a0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:39.221949 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "676faec7f3d1488f99b220f8e968e9a0" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:39.222401 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "676faec7f3d1488f99b220f8e968e9a0" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:39.229038 1160 raft_consensus.cc:359] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.229332 1160 raft_consensus.cc:385] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:39.229635 1160 raft_consensus.cc:740] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:39.229974 1160 consensus_queue.cc:260] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.232313 1160 ts_tablet_manager.cc:1434] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.040s user 0.002s sys 0.000s
I20251211 20:52:39.232679 1160 tablet_bootstrap.cc:492] T fb2d30147cd74d1893783be1e1694d1f P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:39.223055 983 leader_election.cc:343] T 676faec7f3d1488f99b220f8e968e9a0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.240160 1160 tablet_bootstrap.cc:654] T fb2d30147cd74d1893783be1e1694d1f P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:39.240238 982 leader_election.cc:343] T 676faec7f3d1488f99b220f8e968e9a0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.240545 982 leader_election.cc:304] T 676faec7f3d1488f99b220f8e968e9a0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:39.243244 2216 raft_consensus.cc:2749] T 676faec7f3d1488f99b220f8e968e9a0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:39.175160 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 752) took 2507 ms. Trace:
I20251211 20:52:39.249830 1163 rpcz_store.cc:276] 1211 20:52:36.668036 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:36.668168 (+ 132us) service_pool.cc:225] Handling call
1211 20:52:39.175011 (+2506843us) ts_tablet_manager.cc:1151] Deleting tablet 07e4336171714da8adc21977ce33ce50
1211 20:52:39.175023 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.175139 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2495665}
W20251211 20:52:39.250438 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 07e4336171714da8adc21977ce33ce50 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07e4336171714da8adc21977ce33ce50 already in progress: creating tablet
I20251211 20:52:39.250916 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 750) took 2583 ms. Trace:
I20251211 20:52:39.251021 1163 rpcz_store.cc:276] 1211 20:52:36.667874 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:36.668206 (+ 332us) service_pool.cc:225] Handling call
1211 20:52:39.250768 (+2582562us) ts_tablet_manager.cc:1151] Deleting tablet 3562b5effa124378beac7ccf0d27ee7e
1211 20:52:39.250780 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.250893 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2549775}
W20251211 20:52:39.251469 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 3562b5effa124378beac7ccf0d27ee7e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3562b5effa124378beac7ccf0d27ee7e already in progress: creating tablet
I20251211 20:52:39.252437 1163 tablet_replica.cc:333] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:39.252727 1163 raft_consensus.cc:2243] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:39.252949 1163 raft_consensus.cc:2272] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:39.253525 1163 ts_tablet_manager.cc:1916] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:39.257458 2168 tablet.cc:1722] T dd425601d89148eb95525f796a3abf1e P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:39.264720 1166 tablet_bootstrap.cc:492] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:39.264986 1166 ts_tablet_manager.cc:1403] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.087s user 0.000s sys 0.003s
I20251211 20:52:39.265880 1166 raft_consensus.cc:359] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.266156 1166 raft_consensus.cc:385] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:39.266253 1166 raft_consensus.cc:740] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:39.266563 1166 consensus_queue.cc:260] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.267153 1166 ts_tablet_manager.cc:1434] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251211 20:52:39.267496 1166 tablet_bootstrap.cc:492] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:39.269349 1166 tablet_bootstrap.cc:654] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:39.283162 1810 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" instance_seqno: 1765486340578987) as {username='slave'} at 127.0.104.2:53985; Asking this server to re-register.
I20251211 20:52:39.289209 961 heartbeater.cc:461] Registering TS with master...
I20251211 20:52:39.292572 1810 ts_manager.cc:194] Registered new tserver with Master: eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:39.294116 2170 tablet.cc:1722] T dd425601d89148eb95525f796a3abf1e P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:39.298496 1165 ts_tablet_manager.cc:1929] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:39.298738 1165 log.cc:1199] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/2d849c62f7434099a16f62c2a2fc5e85
I20251211 20:52:39.299204 1165 ts_tablet_manager.cc:1950] T 2d849c62f7434099a16f62c2a2fc5e85 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:39.300186 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 807) took 1373 ms. Trace:
I20251211 20:52:39.301735 1165 rpcz_store.cc:276] 1211 20:52:37.927218 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.927269 (+ 51us) service_pool.cc:225] Handling call
1211 20:52:39.084693 (+1157424us) ts_tablet_manager.cc:1151] Deleting tablet 2d849c62f7434099a16f62c2a2fc5e85
1211 20:52:39.084707 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.240091 (+155384us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.298400 (+ 58309us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.300164 (+ 1764us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1157189}
I20251211 20:52:39.310591 1809 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" instance_seqno: 1765486341071617) as {username='slave'} at 127.0.104.3:60211; Asking this server to re-register.
I20251211 20:52:39.318918 1160 tablet_bootstrap.cc:492] T fb2d30147cd74d1893783be1e1694d1f P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:39.319178 1160 ts_tablet_manager.cc:1403] T fb2d30147cd74d1893783be1e1694d1f P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.087s user 0.002s sys 0.001s
I20251211 20:52:39.320060 1160 raft_consensus.cc:359] T fb2d30147cd74d1893783be1e1694d1f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:39.330677 596 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f2d6c9781b4d4c37bc08703347963d5d consensus state has no leader
I20251211 20:52:39.337862 595 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_80e3542d21b64eca991b2e81ac80ac7c [id=44f2f2bf53af4b038ced294f20db9e48] alter complete (version 2)
I20251211 20:52:39.338282 595 master_service.cc:954] ReplaceTablet: received request to replace tablet f22e4d34cde24b37a025c1b5a1046b1b from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:39.338371 591 master_service.cc:954] ReplaceTablet: received request to replace tablet 28ef13be3b84410d9c786c43502a3478 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:39.338757 1165 tablet_replica.cc:333] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:39.338881 591 master_service.cc:954] ReplaceTablet: received request to replace tablet 48820386e41b47129a13bdd5768f021b from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:39.338940 1165 raft_consensus.cc:2243] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:39.338968 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_3826c483ba654865be6be42ae333a65b" } modify_external_catalogs: true
I20251211 20:52:39.339146 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_3826c483ba654865be6be42ae333a65b" } modify_external_catalogs: true
I20251211 20:52:39.305012 1168 tablet_bootstrap.cc:492] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:39.339475 1168 ts_tablet_manager.cc:1403] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.143s user 0.009s sys 0.017s
I20251211 20:52:39.276058 2100 tablet.cc:1722] T dd425601d89148eb95525f796a3abf1e P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:39.312258 1096 heartbeater.cc:461] Registering TS with master...
I20251211 20:52:39.312455 2135 raft_consensus.cc:493] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:39.342415 2135 raft_consensus.cc:515] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.342994 2135 leader_election.cc:290] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:39.343614 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 2d849c62f7434099a16f62c2a2fc5e85 (table default.Table_c504f8f3796f43b697488bd604268543 [id=2af76b9d32094a4ca656060a4478e08a]) successfully deleted
I20251211 20:52:39.344610 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "793f543d6b614ad7b3ab8e717db2ceaf" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:39.344754 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_655c466dc1b54183b7e91ddeca33ae5e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:39.345126 982 leader_election.cc:343] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.345280 2135 raft_consensus.cc:493] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251211 20:52:39.345368 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_655c466dc1b54183b7e91ddeca33ae5e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:39.345374 2135 raft_consensus.cc:515] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.345898 2135 leader_election.cc:290] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:39.346861 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_11a7f002cdb8441eabe140368c795751"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:39.347395 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_11a7f002cdb8441eabe140368c795751 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:39.347406 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e4b5a7553110424d901788c7a2ecdce6" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:39.347615 776 raft_consensus.cc:2468] T e4b5a7553110424d901788c7a2ecdce6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 0.
I20251211 20:52:39.347997 982 leader_election.cc:304] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:39.347986 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_ff22c4d4d5d34ac4bf51e3e4c97b2472"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:39.348383 2135 raft_consensus.cc:2804] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:39.348448 2135 raft_consensus.cc:493] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20251211 20:52:39.348470 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff22c4d4d5d34ac4bf51e3e4c97b2472 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:39.348521 2135 raft_consensus.cc:3060] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.351190 2135 raft_consensus.cc:515] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.353230 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e4b5a7553110424d901788c7a2ecdce6" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:39.353406 776 raft_consensus.cc:3060] T e4b5a7553110424d901788c7a2ecdce6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.355716 2135 leader_election.cc:290] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:39.356361 776 raft_consensus.cc:2468] T e4b5a7553110424d901788c7a2ecdce6 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 1.
I20251211 20:52:39.356904 982 leader_election.cc:304] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:39.357234 2135 raft_consensus.cc:2804] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:39.357581 2135 raft_consensus.cc:697] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 LEADER]: Becoming Leader. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Running, Role: LEADER
I20251211 20:52:39.357885 2135 consensus_queue.cc:237] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.360872 1168 raft_consensus.cc:359] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.361044 1168 raft_consensus.cc:385] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:39.361085 1168 raft_consensus.cc:740] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:39.361241 1168 consensus_queue.cc:260] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.361683 1168 ts_tablet_manager.cc:1434] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.022s user 0.001s sys 0.000s
I20251211 20:52:39.361967 1168 tablet_bootstrap.cc:492] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:39.363298 1168 tablet_bootstrap.cc:654] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:39.364791 2168 tablet.cc:1722] T 1a77e3e744ff42ccb68897704b4c2f1d P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:39.366822 2100 tablet.cc:1722] T 1a77e3e744ff42ccb68897704b4c2f1d P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:39.374892 1165 pending_rounds.cc:70] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919: Trying to abort 1 pending ops.
I20251211 20:52:39.374980 1165 pending_rounds.cc:77] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432125152497664 op_type: NO_OP noop_request { }
I20251211 20:52:39.375074 1165 raft_consensus.cc:2889] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:39.375170 1165 raft_consensus.cc:2272] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:39.381103 2135 raft_consensus.cc:493] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:39.381311 2135 raft_consensus.cc:515] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.382227 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cce44b80c97948e1af7ac68e77cccfd6" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:39.382722 982 leader_election.cc:343] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.344064 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "793f543d6b614ad7b3ab8e717db2ceaf" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:39.384363 1160 raft_consensus.cc:385] T fb2d30147cd74d1893783be1e1694d1f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:39.384459 1160 raft_consensus.cc:740] T fb2d30147cd74d1893783be1e1694d1f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:39.346886 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e4b5a7553110424d901788c7a2ecdce6" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:39.384601 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cce44b80c97948e1af7ac68e77cccfd6" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:39.384658 1160 consensus_queue.cc:260] T fb2d30147cd74d1893783be1e1694d1f P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.352437 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e4b5a7553110424d901788c7a2ecdce6" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:39.385205 1160 ts_tablet_manager.cc:1434] T fb2d30147cd74d1893783be1e1694d1f P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.066s user 0.001s sys 0.000s
I20251211 20:52:39.385535 1160 tablet_bootstrap.cc:492] T b072b759d1254859b0b8d401e3078792 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:39.386940 983 leader_election.cc:343] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.387074 983 leader_election.cc:304] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:39.387456 983 leader_election.cc:343] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.387494 1160 tablet_bootstrap.cc:654] T b072b759d1254859b0b8d401e3078792 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:39.387817 983 leader_election.cc:343] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.387909 983 leader_election.cc:304] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:39.388121 983 leader_election.cc:343] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.388386 2217 raft_consensus.cc:2749] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:39.388393 1809 ts_manager.cc:194] Registered new tserver with Master: e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:39.388346 607 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11f5f94732f049d0be4c97cebfffcb05" candidate_term: 2 candidate_status { last_received { term: 1 index: 848 } } ignore_live_leader: false dest_uuid: "9230f94a6a1a49cead51aeea9f069087" is_pre_election: true
I20251211 20:52:39.388540 2217 raft_consensus.cc:2749] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:39.388516 1350 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11f5f94732f049d0be4c97cebfffcb05" candidate_term: 2 candidate_status { last_received { term: 1 index: 848 } } ignore_live_leader: false dest_uuid: "1830b0c58cfe495989f3d911dda0409a" is_pre_election: true
I20251211 20:52:39.389647 1786 leader_election.cc:304] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f5f94732f049d0be4c97cebfffcb05; no voters: 1830b0c58cfe495989f3d911dda0409a, 9230f94a6a1a49cead51aeea9f069087
I20251211 20:52:39.390458 2161 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:52:39.392050 2135 leader_election.cc:290] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
W20251211 20:52:39.393649 597 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 72af55c8045b4f8f8e116a5782794d81 consensus state has no leader
I20251211 20:52:39.395720 1168 tablet_bootstrap.cc:492] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:39.396024 1168 ts_tablet_manager.cc:1403] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.034s user 0.006s sys 0.000s
I20251211 20:52:39.396831 1168 raft_consensus.cc:359] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.397076 1168 raft_consensus.cc:385] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:39.397164 1168 raft_consensus.cc:740] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:39.397436 1168 consensus_queue.cc:260] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.398092 1168 ts_tablet_manager.cc:1434] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:39.398399 1168 tablet_bootstrap.cc:492] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:39.400116 1168 tablet_bootstrap.cc:654] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:39.401849 1166 tablet_bootstrap.cc:492] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:39.402063 1166 ts_tablet_manager.cc:1403] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.135s user 0.000s sys 0.003s
I20251211 20:52:39.402882 1166 raft_consensus.cc:359] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.403133 1166 raft_consensus.cc:385] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:39.403234 1166 raft_consensus.cc:740] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:39.403498 1166 consensus_queue.cc:260] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.404189 1166 ts_tablet_manager.cc:1434] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:39.404564 1166 tablet_bootstrap.cc:492] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:39.406242 1166 tablet_bootstrap.cc:654] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:39.406672 1165 ts_tablet_manager.cc:1916] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:39.396498 1164 ts_tablet_manager.cc:1929] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:39.407696 1164 log.cc:1199] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/d762b4afb5ff447dac3468393826205a
I20251211 20:52:39.408032 1164 ts_tablet_manager.cc:1950] T d762b4afb5ff447dac3468393826205a P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:39.408798 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 773) took 1469 ms. Trace:
I20251211 20:52:39.408901 1164 rpcz_store.cc:276] 1211 20:52:37.939607 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.940485 (+ 878us) service_pool.cc:225] Handling call
1211 20:52:39.035031 (+1094546us) ts_tablet_manager.cc:1151] Deleting tablet d762b4afb5ff447dac3468393826205a
1211 20:52:39.035043 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.297052 (+262009us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.396382 (+ 99330us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.408773 (+ 12391us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1094394}
I20251211 20:52:39.409423 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet d762b4afb5ff447dac3468393826205a (table default.Table_f6f977edac0f45bea7e94de5bfa51ee4 [id=4a74b9a60cbb4198b55cee630b2f4bef]) successfully deleted
I20251211 20:52:39.409832 1164 tablet_replica.cc:333] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:39.410020 1164 raft_consensus.cc:2243] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:39.410171 1164 raft_consensus.cc:2272] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:39.410691 1164 ts_tablet_manager.cc:1916] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:39.411661 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_9b8d474fab284933871f1ed0993d1d61" } new_table_name: "default.Table_731826f26f9940b8afe425d694b642d9" modify_external_catalogs: true
I20251211 20:52:39.413267 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_4d0dab040d6c4dde86eadda5cf74061b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:39.413834 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4d0dab040d6c4dde86eadda5cf74061b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:39.415462 582 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet e3f711f02af843adb4f5c3c750924e33 (table default.Table_34bd5155aa3b4ef7818022ba9346c074 [id=83a61d23e80946439a14096ab3ab1bd9], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet e3f711f02af843adb4f5c3c750924e33 consensus state has no leader
I20251211 20:52:39.416589 594 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet e233aa9352f645669f1f7cb35fb1bbf2
I20251211 20:52:39.416955 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_72e3939e6805448ba5d1f87e6dad3d3a" } modify_external_catalogs: true
I20251211 20:52:39.417117 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_72e3939e6805448ba5d1f87e6dad3d3a" } modify_external_catalogs: true
I20251211 20:52:39.421444 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_cec07d512c8049e581be33a0214fad11"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:39.422039 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cec07d512c8049e581be33a0214fad11 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:39.422851 1164 ts_tablet_manager.cc:1929] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:39.422969 1164 log.cc:1199] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/6dfa7480e2ab49b9b8a938da0dc6cf0d
I20251211 20:52:39.423450 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0605520db16d4320bafd3ffe2e5395f0" } modify_external_catalogs: true
I20251211 20:52:39.423648 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_0605520db16d4320bafd3ffe2e5395f0" } modify_external_catalogs: true
I20251211 20:52:39.424285 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet e9ead34065e240eeb1f946319537d32a
I20251211 20:52:39.425009 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_bf0900ef3fb8447c834a43e509caa105" } modify_external_catalogs: true
I20251211 20:52:39.425187 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_bf0900ef3fb8447c834a43e509caa105" } modify_external_catalogs: true
I20251211 20:52:39.426083 747 tablet_service.cc:1552] Processing DeleteTablet for tablet e9ead34065e240eeb1f946319537d32a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:39.427202 888 tablet_service.cc:1552] Processing DeleteTablet for tablet e9ead34065e240eeb1f946319537d32a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:39.426560 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet e9ead34065e240eeb1f946319537d32a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:39.334946 1163 ts_tablet_manager.cc:1929] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20251211 20:52:39.440094 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4c7b74f370b042ee8575f821c07dc085 has no consensus state
I20251211 20:52:39.440217 1163 log.cc:1199] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/c74427cb9d6f4a679e43eae796ff8c3f
I20251211 20:52:39.440433 593 master_service.cc:954] ReplaceTablet: received request to replace tablet a295a24e81f14e5fa126a4a58ffd08cf from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:39.440559 1163 ts_tablet_manager.cc:1950] T c74427cb9d6f4a679e43eae796ff8c3f P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:39.441370 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 756) took 2580 ms. Trace:
I20251211 20:52:39.441565 1163 rpcz_store.cc:276] 1211 20:52:36.860547 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:36.860603 (+ 56us) service_pool.cc:225] Handling call
1211 20:52:39.252383 (+2391780us) ts_tablet_manager.cc:1151] Deleting tablet c74427cb9d6f4a679e43eae796ff8c3f
1211 20:52:39.252394 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.332000 (+ 79606us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.334839 (+ 2839us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.441348 (+106509us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2391495}
I20251211 20:52:39.441869 588 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_063f94689e6547e2b871d8465153a098" } new_table_name: "default.Table_66eb1b021eed464f82f9cd223c88297b" modify_external_catalogs: true
I20251211 20:52:39.442131 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet c74427cb9d6f4a679e43eae796ff8c3f (table default.Table_31d573e7af144e9997dfe480f8466902 [id=f689abc175c94366bfda8217545156aa]) successfully deleted
I20251211 20:52:39.442461 1163 tablet_replica.cc:333] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:39.442692 1163 raft_consensus.cc:2243] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:39.442827 1163 raft_consensus.cc:2272] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:39.443382 1163 ts_tablet_manager.cc:1916] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:39.444058 711 consensus_peers.cc:597] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:39.446306 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_8ead90e9bdef45bf8bf6c5d4c26c347a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:39.446942 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8ead90e9bdef45bf8bf6c5d4c26c347a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:39.447357 1164 ts_tablet_manager.cc:1950] T 6dfa7480e2ab49b9b8a938da0dc6cf0d P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:39.450587 1168 tablet_bootstrap.cc:492] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:39.450799 1168 ts_tablet_manager.cc:1403] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.052s user 0.003s sys 0.000s
I20251211 20:52:39.448331 1160 tablet_bootstrap.cc:492] T b072b759d1254859b0b8d401e3078792 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:39.451717 1168 raft_consensus.cc:359] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.451941 1168 raft_consensus.cc:385] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:39.452088 1168 raft_consensus.cc:740] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:39.452325 1168 consensus_queue.cc:260] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.454005 1160 ts_tablet_manager.cc:1403] T b072b759d1254859b0b8d401e3078792 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.069s user 0.000s sys 0.005s
I20251211 20:52:39.454607 1168 ts_tablet_manager.cc:1434] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251211 20:52:39.454910 1168 tablet_bootstrap.cc:492] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:39.454867 1160 raft_consensus.cc:359] T b072b759d1254859b0b8d401e3078792 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.455157 1160 raft_consensus.cc:385] T b072b759d1254859b0b8d401e3078792 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:39.455260 1160 raft_consensus.cc:740] T b072b759d1254859b0b8d401e3078792 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:39.455519 1160 consensus_queue.cc:260] T b072b759d1254859b0b8d401e3078792 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.456483 1168 tablet_bootstrap.cc:654] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:39.456926 1160 ts_tablet_manager.cc:1434] T b072b759d1254859b0b8d401e3078792 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:39.457248 1160 tablet_bootstrap.cc:492] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:39.458839 1160 tablet_bootstrap.cc:654] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:39.460613 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 777) took 1507 ms. Trace:
I20251211 20:52:39.463640 1164 rpcz_store.cc:276] 1211 20:52:37.953151 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.953305 (+ 154us) service_pool.cc:225] Handling call
1211 20:52:39.409760 (+1456455us) ts_tablet_manager.cc:1151] Deleting tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d
1211 20:52:39.409773 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.419801 (+ 10028us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.422735 (+ 2934us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.460590 (+ 37855us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1456271}
I20251211 20:52:39.464257 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 6dfa7480e2ab49b9b8a938da0dc6cf0d (table default.Table_c817dec9a7674f699b3366fc302ead57 [id=d555d1fc6aef4a21b4d3f2804264d030]) successfully deleted
I20251211 20:52:39.464636 1164 tablet_replica.cc:333] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
W20251211 20:52:39.464937 588 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5773c13034714a008c8d7dbe777b4828 consensus state has no leader
I20251211 20:52:39.465312 588 master_service.cc:954] ReplaceTablet: received request to replace tablet 940dfd8781de444a9caa78dd363c7451 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:39.466495 1164 raft_consensus.cc:2243] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:39.466666 1164 raft_consensus.cc:2272] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:39.469072 1166 tablet_bootstrap.cc:492] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:39.469275 1166 ts_tablet_manager.cc:1403] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.065s user 0.003s sys 0.001s
I20251211 20:52:39.470129 1166 raft_consensus.cc:359] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.472048 1166 raft_consensus.cc:385] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:39.472163 1166 raft_consensus.cc:740] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:39.472434 1166 consensus_queue.cc:260] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.480350 1160 tablet_bootstrap.cc:492] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:39.480664 1160 ts_tablet_manager.cc:1403] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.023s user 0.005s sys 0.000s
I20251211 20:52:39.481870 1160 raft_consensus.cc:359] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.482133 1160 raft_consensus.cc:385] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:39.482228 1160 raft_consensus.cc:740] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:39.482498 1160 consensus_queue.cc:260] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.483090 1160 ts_tablet_manager.cc:1434] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:39.486223 1160 tablet_bootstrap.cc:492] T a9b2aa21ab124b28aebc33adba0583d2 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:39.487870 1160 tablet_bootstrap.cc:654] T a9b2aa21ab124b28aebc33adba0583d2 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:39.488898 1165 ts_tablet_manager.cc:1929] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:39.489068 1165 log.cc:1199] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/fb030a62796248869d46b682656eaeb0
I20251211 20:52:39.489971 1165 ts_tablet_manager.cc:1950] T fb030a62796248869d46b682656eaeb0 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:39.490784 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 809) took 1547 ms. Trace:
I20251211 20:52:39.490909 1165 rpcz_store.cc:276] 1211 20:52:37.943289 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.943338 (+ 49us) service_pool.cc:225] Handling call
1211 20:52:39.338704 (+1395366us) ts_tablet_manager.cc:1151] Deleting tablet fb030a62796248869d46b682656eaeb0
1211 20:52:39.338714 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.410388 (+ 71674us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.488766 (+ 78378us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.490761 (+ 1995us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1395151}
I20251211 20:52:39.491307 1165 tablet_replica.cc:333] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:39.491456 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet fb030a62796248869d46b682656eaeb0 (table default.Table_069bb219e15e43be995ff011e2289a84 [id=bc0b945e86cf43fdb99d56658ca1576c]) successfully deleted
I20251211 20:52:39.491510 1165 raft_consensus.cc:2243] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:39.491851 1165 raft_consensus.cc:2272] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:39.495018 1165 ts_tablet_manager.cc:1916] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:39.504508 1166 ts_tablet_manager.cc:1434] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.035s user 0.001s sys 0.000s
I20251211 20:52:39.506372 1166 tablet_bootstrap.cc:492] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:39.510458 1166 tablet_bootstrap.cc:654] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:39.512307 1160 tablet_bootstrap.cc:492] T a9b2aa21ab124b28aebc33adba0583d2 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:39.512548 1160 ts_tablet_manager.cc:1403] T a9b2aa21ab124b28aebc33adba0583d2 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.000s
I20251211 20:52:39.513440 1160 raft_consensus.cc:359] T a9b2aa21ab124b28aebc33adba0583d2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.513689 1160 raft_consensus.cc:385] T a9b2aa21ab124b28aebc33adba0583d2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:39.513783 1160 raft_consensus.cc:740] T a9b2aa21ab124b28aebc33adba0583d2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:39.514055 1160 consensus_queue.cc:260] T a9b2aa21ab124b28aebc33adba0583d2 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.514583 1160 ts_tablet_manager.cc:1434] T a9b2aa21ab124b28aebc33adba0583d2 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:39.518052 1163 ts_tablet_manager.cc:1929] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:39.518294 1163 log.cc:1199] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/bb112b6431af42a6a7b328dfdefbc2a2
I20251211 20:52:39.518704 1163 ts_tablet_manager.cc:1950] T bb112b6431af42a6a7b328dfdefbc2a2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:39.504143 1168 tablet_bootstrap.cc:492] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:39.523182 1168 ts_tablet_manager.cc:1403] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.068s user 0.003s sys 0.000s
I20251211 20:52:39.525243 1168 raft_consensus.cc:359] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.525496 1168 raft_consensus.cc:385] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:39.525578 1168 raft_consensus.cc:740] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:39.525830 1168 consensus_queue.cc:260] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.526453 1168 ts_tablet_manager.cc:1434] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:39.526777 1168 tablet_bootstrap.cc:492] T bb0fc179a337469eae056c5bb3815e6b P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:39.528419 1168 tablet_bootstrap.cc:654] T bb0fc179a337469eae056c5bb3815e6b P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:39.528779 597 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet bfb52935296443c58190d4d06e2e6e9c (table default.Table_3721b5725e8c4297a392d3986a081128 [id=383392d835da4c50848068d83261cecc]). Expected version 7 got 0
I20251211 20:52:39.529078 597 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet c7a2610f20df4c1ca81eae2fc7dba6bb (table default.Table_59072235d93b41c0b5cd5e3debcb8ddf [id=9a15a8a075af4969ae2c8b3be1bfed78]). Expected version 1 got 0
I20251211 20:52:39.531564 590 catalog_manager.cc:5654] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:39.532027 590 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet bfb52935296443c58190d4d06e2e6e9c (table default.Table_3721b5725e8c4297a392d3986a081128 [id=383392d835da4c50848068d83261cecc]). Expected version 7 got 0
I20251211 20:52:39.532263 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4d0dab040d6c4dde86eadda5cf74061b" } new_table_name: "default.Table_d887c953026046dc883ede774a58fd8e" modify_external_catalogs: true
I20251211 20:52:39.532315 590 catalog_manager.cc:5654] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:39.534330 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_eeb5507eb6a74588905b9e236e414324"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:39.534902 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eeb5507eb6a74588905b9e236e414324 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:39.535176 590 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet c7a2610f20df4c1ca81eae2fc7dba6bb (table default.Table_59072235d93b41c0b5cd5e3debcb8ddf [id=9a15a8a075af4969ae2c8b3be1bfed78]). Expected version 1 got 0
I20251211 20:52:39.535463 590 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b072b759d1254859b0b8d401e3078792 (table default.Table_b56de1e85a1b42ea81308593884dadb8 [id=a137f0ddf52145a99c62444602ab3fdb]). Expected version 4 got 0
I20251211 20:52:39.540376 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 4585e05d06ef42cfb06c4ceb7524fd71
I20251211 20:52:39.541078 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_70adbfea12154508bd1988d085007a3e" } new_table_name: "default.Table_3418cf6b51394b03afec9b03cde8861c" modify_external_catalogs: true
I20251211 20:52:39.542737 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 4585e05d06ef42cfb06c4ceb7524fd71 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:39.544123 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 4585e05d06ef42cfb06c4ceb7524fd71 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:39.552452 595 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 7bc03ed8c35740f58fe48ddbaa120ea0
I20251211 20:52:39.553890 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 7bc03ed8c35740f58fe48ddbaa120ea0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:39.553941 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 7bc03ed8c35740f58fe48ddbaa120ea0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:39.554502 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet a295a24e81f14e5fa126a4a58ffd08cf
I20251211 20:52:39.554979 593 catalog_manager.cc:6724] ReplaceTablet: tablet a295a24e81f14e5fa126a4a58ffd08cf deleted and replaced by tablet 77bd176108f24e79a632c798566e9e31
I20251211 20:52:39.555810 747 tablet_service.cc:1552] Processing DeleteTablet for tablet a295a24e81f14e5fa126a4a58ffd08cf with delete_type TABLET_DATA_DELETED (replaced by tablet 77bd176108f24e79a632c798566e9e31) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:39.555852 888 tablet_service.cc:1552] Processing DeleteTablet for tablet a295a24e81f14e5fa126a4a58ffd08cf with delete_type TABLET_DATA_DELETED (replaced by tablet 77bd176108f24e79a632c798566e9e31) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:39.556370 596 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 6ad6ab53aad14cc2b6d201acfecf6723
I20251211 20:52:39.557459 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 6ad6ab53aad14cc2b6d201acfecf6723 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:39.557842 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 6ad6ab53aad14cc2b6d201acfecf6723 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:39.558082 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 48820386e41b47129a13bdd5768f021b
I20251211 20:52:39.558475 591 catalog_manager.cc:6724] ReplaceTablet: tablet 48820386e41b47129a13bdd5768f021b deleted and replaced by tablet 7a299ee03d0e48aba7c4f71e3e0f381a
I20251211 20:52:39.559101 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 48820386e41b47129a13bdd5768f021b with delete_type TABLET_DATA_DELETED (replaced by tablet 7a299ee03d0e48aba7c4f71e3e0f381a) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:39.559448 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 48820386e41b47129a13bdd5768f021b with delete_type TABLET_DATA_DELETED (replaced by tablet 7a299ee03d0e48aba7c4f71e3e0f381a) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:39.570816 1164 ts_tablet_manager.cc:1916] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:39.574144 665 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 4585e05d06ef42cfb06c4ceb7524fd71
I20251211 20:52:39.575132 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 4585e05d06ef42cfb06c4ceb7524fd71 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:39.576226 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 4585e05d06ef42cfb06c4ceb7524fd71 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:39.576538 1160 tablet_bootstrap.cc:492] T 773cf53745a64f62817113b33e3182e7 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:39.578329 1813 raft_consensus.cc:493] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:39.578473 1813 raft_consensus.cc:515] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.578895 888 tablet_service.cc:1505] Processing CreateTablet for tablet 9b60966ae2ee4928a5e1b0e9c912ff77 (DEFAULT_TABLE table=default.Table_862fff1ce3564d939f033f3f2fe049ac [id=d35a5d6eb11049239d1898d175b356b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:39.579066 1813 leader_election.cc:290] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:39.579838 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8e17645a26947ab8c01f9b093fa422c" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:39.581490 1168 tablet_bootstrap.cc:492] T bb0fc179a337469eae056c5bb3815e6b P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:39.581725 1168 ts_tablet_manager.cc:1403] T bb0fc179a337469eae056c5bb3815e6b P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.055s user 0.003s sys 0.001s
I20251211 20:52:39.582535 1168 raft_consensus.cc:359] T bb0fc179a337469eae056c5bb3815e6b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.582777 1168 raft_consensus.cc:385] T bb0fc179a337469eae056c5bb3815e6b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:39.582881 1168 raft_consensus.cc:740] T bb0fc179a337469eae056c5bb3815e6b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:39.583139 1168 consensus_queue.cc:260] T bb0fc179a337469eae056c5bb3815e6b P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.583707 1168 ts_tablet_manager.cc:1434] T bb0fc179a337469eae056c5bb3815e6b P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:39.583796 914 raft_consensus.cc:3060] T b072b759d1254859b0b8d401e3078792 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.584053 1168 tablet_bootstrap.cc:492] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:39.585515 747 tablet_service.cc:1505] Processing CreateTablet for tablet 9b60966ae2ee4928a5e1b0e9c912ff77 (DEFAULT_TABLE table=default.Table_862fff1ce3564d939f033f3f2fe049ac [id=d35a5d6eb11049239d1898d175b356b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:39.585613 1168 tablet_bootstrap.cc:654] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:39.585907 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b60966ae2ee4928a5e1b0e9c912ff77. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:39.595801 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b60966ae2ee4928a5e1b0e9c912ff77. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:39.596194 1160 tablet_bootstrap.cc:654] T 773cf53745a64f62817113b33e3182e7 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:39.596817 877 tablet_service.cc:1505] Processing CreateTablet for tablet 1a05ed40a4454e7e8721a556ac3de092 (DEFAULT_TABLE table=default.Table_9e7258165cbf458c83d7cd4874a8755f [id=6f1e5458bbef44fdb8ae111900bf5159]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:39.555889 1021 tablet_service.cc:1552] Processing DeleteTablet for tablet 7bc03ed8c35740f58fe48ddbaa120ea0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:39.556272 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet a295a24e81f14e5fa126a4a58ffd08cf with delete_type TABLET_DATA_DELETED (replaced by tablet 77bd176108f24e79a632c798566e9e31) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:39.559901 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 48820386e41b47129a13bdd5768f021b with delete_type TABLET_DATA_DELETED (replaced by tablet 7a299ee03d0e48aba7c4f71e3e0f381a) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:39.519572 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 755) took 2659 ms. Trace:
I20251211 20:52:39.598491 1163 rpcz_store.cc:276] 1211 20:52:36.860426 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:36.861005 (+ 579us) service_pool.cc:225] Handling call
1211 20:52:39.442406 (+2581401us) ts_tablet_manager.cc:1151] Deleting tablet bb112b6431af42a6a7b328dfdefbc2a2
1211 20:52:39.442417 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.446411 (+ 3994us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.517913 (+ 71502us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.519521 (+ 1608us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2581152}
I20251211 20:52:39.598997 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet bb112b6431af42a6a7b328dfdefbc2a2 (table default.Table_0c2f5186505f4a14b5c9bba07ca14178 [id=b4d34bd45a214d3b898045da7537ee10]) successfully deleted
I20251211 20:52:39.568629 2135 raft_consensus.cc:493] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:39.577466 1019 tablet_service.cc:1552] Processing DeleteTablet for tablet 4585e05d06ef42cfb06c4ceb7524fd71 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:39.599388 2135 raft_consensus.cc:515] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.595330 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 9b60966ae2ee4928a5e1b0e9c912ff77 (DEFAULT_TABLE table=default.Table_862fff1ce3564d939f033f3f2fe049ac [id=d35a5d6eb11049239d1898d175b356b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:39.543264 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 4585e05d06ef42cfb06c4ceb7524fd71 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:39.557859 1015 tablet_service.cc:1552] Processing DeleteTablet for tablet 6ad6ab53aad14cc2b6d201acfecf6723 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:39.571722 2216 raft_consensus.cc:493] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:39.599964 2216 raft_consensus.cc:515] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.600092 2135 leader_election.cc:290] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:39.600507 2216 leader_election.cc:290] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:39.579767 1043 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8e17645a26947ab8c01f9b093fa422c" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:39.601488 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7ec261837cf04bf88a48ff8add1b0d66" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:39.601488 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb0fc179a337469eae056c5bb3815e6b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:39.601687 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b60966ae2ee4928a5e1b0e9c912ff77. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:39.601711 776 raft_consensus.cc:2468] T bb0fc179a337469eae056c5bb3815e6b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 0.
I20251211 20:52:39.601739 847 leader_election.cc:304] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
W20251211 20:52:39.602144 982 leader_election.cc:343] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.602347 982 leader_election.cc:304] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:39.602795 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a05ed40a4454e7e8721a556ac3de092. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:39.602861 2216 raft_consensus.cc:2804] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:39.602949 2216 raft_consensus.cc:493] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:39.603034 2216 raft_consensus.cc:3060] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.603452 1163 tablet_replica.cc:333] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:39.604800 1163 raft_consensus.cc:2243] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:39.604984 1163 raft_consensus.cc:2272] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:39.605502 1163 ts_tablet_manager.cc:1916] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:39.605425 2216 raft_consensus.cc:515] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.605859 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7ec261837cf04bf88a48ff8add1b0d66" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:39.606153 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb0fc179a337469eae056c5bb3815e6b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:39.606441 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb0fc179a337469eae056c5bb3815e6b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:39.606484 1813 raft_consensus.cc:2749] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:39.606770 983 leader_election.cc:343] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.606914 983 leader_election.cc:304] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:39.607295 983 leader_election.cc:343] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:39.607561 983 leader_election.cc:343] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.607779 2219 raft_consensus.cc:2749] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:39.608855 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb0fc179a337469eae056c5bb3815e6b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:39.608889 747 tablet_service.cc:1505] Processing CreateTablet for tablet 1a05ed40a4454e7e8721a556ac3de092 (DEFAULT_TABLE table=default.Table_9e7258165cbf458c83d7cd4874a8755f [id=6f1e5458bbef44fdb8ae111900bf5159]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:39.609035 776 raft_consensus.cc:3060] T bb0fc179a337469eae056c5bb3815e6b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.609246 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a05ed40a4454e7e8721a556ac3de092. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:39.610217 1160 tablet_bootstrap.cc:492] T 773cf53745a64f62817113b33e3182e7 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:39.610338 1160 ts_tablet_manager.cc:1403] T 773cf53745a64f62817113b33e3182e7 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.034s user 0.001s sys 0.001s
I20251211 20:52:39.611071 1160 raft_consensus.cc:359] T 773cf53745a64f62817113b33e3182e7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.611218 1160 raft_consensus.cc:385] T 773cf53745a64f62817113b33e3182e7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:39.611261 1160 raft_consensus.cc:740] T 773cf53745a64f62817113b33e3182e7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:39.611500 776 raft_consensus.cc:2468] T bb0fc179a337469eae056c5bb3815e6b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 1.
I20251211 20:52:39.611451 1160 consensus_queue.cc:260] T 773cf53745a64f62817113b33e3182e7 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.612308 982 leader_election.cc:304] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:39.613206 2135 raft_consensus.cc:2804] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:39.613348 2135 raft_consensus.cc:697] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 LEADER]: Becoming Leader. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Running, Role: LEADER
I20251211 20:52:39.613586 2135 consensus_queue.cc:237] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.613607 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 1a05ed40a4454e7e8721a556ac3de092 (DEFAULT_TABLE table=default.Table_9e7258165cbf458c83d7cd4874a8755f [id=6f1e5458bbef44fdb8ae111900bf5159]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:39.614215 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a05ed40a4454e7e8721a556ac3de092. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:39.615386 2216 leader_election.cc:290] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:39.615638 1164 ts_tablet_manager.cc:1929] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:39.615751 1164 log.cc:1199] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/822abda3c85848728205b71847ef7029
I20251211 20:52:39.619179 1164 ts_tablet_manager.cc:1950] T 822abda3c85848728205b71847ef7029 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:39.619524 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 776) took 1666 ms. Trace:
I20251211 20:52:39.619642 1164 rpcz_store.cc:276] 1211 20:52:37.953045 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.953590 (+ 545us) service_pool.cc:225] Handling call
1211 20:52:39.464584 (+1510994us) ts_tablet_manager.cc:1151] Deleting tablet 822abda3c85848728205b71847ef7029
1211 20:52:39.464595 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.578050 (+113455us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.615508 (+ 37458us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.619506 (+ 3998us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1510815}
I20251211 20:52:39.620042 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 783) took 1600 ms. Trace:
I20251211 20:52:39.620126 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 822abda3c85848728205b71847ef7029 (table default.Table_4aad4cf01eff415f86b8446c29977007 [id=01282adb7b834261a7750c350ad0b250]) successfully deleted
I20251211 20:52:39.620126 1164 rpcz_store.cc:276] 1211 20:52:38.019642 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.019696 (+ 54us) service_pool.cc:225] Handling call
1211 20:52:39.619918 (+1600222us) ts_tablet_manager.cc:1151] Deleting tablet b61c4f0cb15940a988c5acb6879d265a
1211 20:52:39.619928 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.620026 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1600031}
I20251211 20:52:39.620416 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 797) took 1458 ms. Trace:
I20251211 20:52:39.620482 1164 rpcz_store.cc:276] 1211 20:52:38.162046 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.162107 (+ 61us) service_pool.cc:225] Handling call
1211 20:52:39.620326 (+1458219us) ts_tablet_manager.cc:1151] Deleting tablet ccc3ddd4b7f74f11a3835997cb7ae8d0
1211 20:52:39.620335 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.620403 (+ 68us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1457994}
I20251211 20:52:39.620769 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 800) took 1451 ms. Trace:
I20251211 20:52:39.620836 1164 rpcz_store.cc:276] 1211 20:52:38.169436 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.169488 (+ 52us) service_pool.cc:225] Handling call
1211 20:52:39.620680 (+1451192us) ts_tablet_manager.cc:1151] Deleting tablet a471a9accbfc4875bd1d7802f5620808
1211 20:52:39.620688 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.620754 (+ 66us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1451004}
W20251211 20:52:39.620944 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet b61c4f0cb15940a988c5acb6879d265a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b61c4f0cb15940a988c5acb6879d265a already in progress: creating tablet
I20251211 20:52:39.621104 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 806) took 1392 ms. Trace:
W20251211 20:52:39.621166 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet ccc3ddd4b7f74f11a3835997cb7ae8d0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ccc3ddd4b7f74f11a3835997cb7ae8d0 already in progress: creating tablet
I20251211 20:52:39.621161 1164 rpcz_store.cc:276] 1211 20:52:38.228589 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.228664 (+ 75us) service_pool.cc:225] Handling call
1211 20:52:39.621017 (+1392353us) ts_tablet_manager.cc:1151] Deleting tablet 745ce63314834c8abdecaa367133a068
1211 20:52:39.621024 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.621088 (+ 64us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1392148}
I20251211 20:52:39.621376 1164 tablet_replica.cc:333] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:39.621541 1164 raft_consensus.cc:2243] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:39.621716 1164 raft_consensus.cc:2272] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:39.621805 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet a471a9accbfc4875bd1d7802f5620808 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a471a9accbfc4875bd1d7802f5620808 already in progress: creating tablet
W20251211 20:52:39.622011 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 745ce63314834c8abdecaa367133a068 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 745ce63314834c8abdecaa367133a068 already in progress: creating tablet
I20251211 20:52:39.622282 1164 ts_tablet_manager.cc:1916] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:39.633576 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 13269a20421240d1a98215129443248b has no consensus state
I20251211 20:52:39.634374 597 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_b008891c4e894b02aa0db32d1740371a [id=8fe47ac33eb2461784b859e64fbd3e24] alter complete (version 5)
I20251211 20:52:39.637194 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet bb0fc179a337469eae056c5bb3815e6b with delete_type TABLET_DATA_DELETED (replaced by tablet d53b074865f4490693152a247b0fffdc) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:39.642196 592 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet c8e17645a26947ab8c01f9b093fa422c (table default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5]). Expected version 2 got 0
I20251211 20:52:39.645387 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f196fe07bc5d407d895f5d36dd956786" } modify_external_catalogs: true
I20251211 20:52:39.645579 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f196fe07bc5d407d895f5d36dd956786" } modify_external_catalogs: true
I20251211 20:52:39.646797 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_4ec0d87202fc419380c2542d827087de"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:39.647373 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4ec0d87202fc419380c2542d827087de in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:39.648768 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_e870d2f987ff44e6b91ebe0205f6fe9e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:39.649371 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e870d2f987ff44e6b91ebe0205f6fe9e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:39.650512 710 consensus_peers.cc:597] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb112b6431af42a6a7b328dfdefbc2a2. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:39.651175 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_d44ac0044b77421f88b8a7b00811c8d3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:39.651333 1165 ts_tablet_manager.cc:1929] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20251211 20:52:39.651458 1165 log.cc:1199] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/a5eeb56709f046d9be92ac600ebc330e
W20251211 20:52:39.651743 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d44ac0044b77421f88b8a7b00811c8d3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:39.651888 1165 ts_tablet_manager.cc:1950] T a5eeb56709f046d9be92ac600ebc330e P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:39.652669 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 810) took 1709 ms. Trace:
I20251211 20:52:39.652755 1165 rpcz_store.cc:276] 1211 20:52:37.943366 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.943500 (+ 134us) service_pool.cc:225] Handling call
1211 20:52:39.491252 (+1547752us) ts_tablet_manager.cc:1151] Deleting tablet a5eeb56709f046d9be92ac600ebc330e
1211 20:52:39.491264 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.604101 (+112837us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.651069 (+ 46968us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.652644 (+ 1575us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1547599}
I20251211 20:52:39.653309 1165 tablet_replica.cc:333] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:39.653316 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_e841712540dc46ffa1821599e1f10e86"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:39.653792 1160 ts_tablet_manager.cc:1434] T 773cf53745a64f62817113b33e3182e7 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.043s user 0.002s sys 0.000s
W20251211 20:52:39.653803 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e841712540dc46ffa1821599e1f10e86 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:39.655510 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_31a95f7c09a34c4585eef1f7914a97a7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:39.656020 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_31a95f7c09a34c4585eef1f7914a97a7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:39.657243 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_df9af0c629bc44728c89b7ae449b7ea1" } new_table_name: "default.Table_db4c0f706e0643b39d39ffa84352826d" modify_external_catalogs: true
I20251211 20:52:39.658694 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet a5eeb56709f046d9be92ac600ebc330e (table default.Table_279bb02bdbd743159babc9a4e6498437 [id=f78e1da479af4d23a552af1c4fc55f81]) successfully deleted
I20251211 20:52:39.663866 888 tablet_service.cc:1505] Processing CreateTablet for tablet a3b263c205eb479b8210ac79c766523f (DEFAULT_TABLE table=default.Table_948151057bf94b75a251b99e3906f4a4 [id=1ab6db436e504bfabeb44e435a5453b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:39.664321 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3b263c205eb479b8210ac79c766523f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:39.665619 1015 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ec261837cf04bf88a48ff8add1b0d66 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:39.673111 1160 tablet_bootstrap.cc:492] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:39.674490 1160 tablet_bootstrap.cc:654] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:39.683636 2135 raft_consensus.cc:493] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:39.684031 2135 raft_consensus.cc:515] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.685878 2135 leader_election.cc:290] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:39.687742 592 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_4e8d74eff9674d67b5b0f54370b06598 [id=c842792b01cb4c689fcfd3d796151508] alter complete (version 3)
I20251211 20:52:39.688499 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_37fd7b6c68b64deaa3fb0ac9a1965eff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:39.689121 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_37fd7b6c68b64deaa3fb0ac9a1965eff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:39.691100 593 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_58802d3fc36742ffb050db7bb7f0c064" } new_table_name: "default.Table_7506ffecc2b64648a9ba736e5298ddf4" modify_external_catalogs: true
I20251211 20:52:39.692617 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_eaad557efa374b0bbd5653d90a967f8b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:39.693172 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eaad557efa374b0bbd5653d90a967f8b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:39.694577 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_8e6bffee74fd4ea4a95ec5e51b0154c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:39.695065 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8e6bffee74fd4ea4a95ec5e51b0154c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:39.696805 591 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b6cc819332e1445c96ce3ffac4861454 consensus state has no leader
I20251211 20:52:39.697187 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_d1fc77dff3464153a4fff421df27eb33" } new_table_name: "default.Table_823505200e3142dbbe8cd56ac78daf55" modify_external_catalogs: true
I20251211 20:52:39.699139 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_84d90fc27f874a01bf3187d6d02b2053"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:39.699667 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_84d90fc27f874a01bf3187d6d02b2053 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:39.702050 1163 ts_tablet_manager.cc:1929] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:39.702625 1163 log.cc:1199] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/fb030a62796248869d46b682656eaeb0
I20251211 20:52:39.703136 1163 ts_tablet_manager.cc:1950] T fb030a62796248869d46b682656eaeb0 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:39.704133 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "52ef1707f6f2425aafd454f303af0333" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:39.704324 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 754) took 2857 ms. Trace:
I20251211 20:52:39.679085 2101 raft_consensus.cc:493] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:39.704893 2101 raft_consensus.cc:515] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.705652 2101 leader_election.cc:290] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:39.685722 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "52ef1707f6f2425aafd454f303af0333" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:39.706070 741 tablet_service.cc:1552] Processing DeleteTablet for tablet b529b38d754f46ba8cb60b19e9705256 with delete_type TABLET_DATA_DELETED (replaced by tablet 84bb3583857848459e0e865283de623d) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:39.706105 776 raft_consensus.cc:2468] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 0.
I20251211 20:52:39.707726 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 14e2650f8ec1482ea9511b989fdc963c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:39.704774 983 leader_election.cc:343] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.707967 1043 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b529b38d754f46ba8cb60b19e9705256" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:39.709945 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_514c179c4a144ce398e7c9b8fbba2bce" } modify_external_catalogs: true
I20251211 20:52:39.710081 1043 raft_consensus.cc:2468] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
W20251211 20:52:39.710139 711 consensus_peers.cc:597] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:39.710145 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_514c179c4a144ce398e7c9b8fbba2bce" } modify_external_catalogs: true
W20251211 20:52:39.710316 711 consensus_peers.cc:597] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:39.708845 1042 raft_consensus.cc:1275] T 72e23f3033e14c9db6de8539830bac3b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Refusing update from remote peer 6f8d784fd1cd492bbe3809403ec34919: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251211 20:52:39.710587 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b529b38d754f46ba8cb60b19e9705256" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:39.710688 982 leader_election.cc:304] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:39.711140 711 leader_election.cc:343] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.711266 711 leader_election.cc:304] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:39.711434 2135 raft_consensus.cc:2804] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:39.711521 2135 raft_consensus.cc:493] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:39.711625 2135 raft_consensus.cc:3060] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.711875 2101 raft_consensus.cc:2804] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:39.711941 2101 raft_consensus.cc:493] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:39.711993 2101 raft_consensus.cc:3060] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.714092 2019 consensus_queue.cc:1048] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:39.714349 2101 raft_consensus.cc:515] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.714365 2135 raft_consensus.cc:515] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.715222 2135 leader_election.cc:290] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:39.689993 1168 tablet_bootstrap.cc:492] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:39.715677 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "52ef1707f6f2425aafd454f303af0333" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:39.715764 1168 ts_tablet_manager.cc:1403] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.132s user 0.005s sys 0.002s
I20251211 20:52:39.715754 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 66cdc6f297b24cfba3a3a0ae9f45d12d
I20251211 20:52:39.715873 776 raft_consensus.cc:3060] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.715436 1163 rpcz_store.cc:276] 1211 20:52:36.846780 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:36.849361 (+ 2581us) service_pool.cc:225] Handling call
1211 20:52:39.603374 (+2754013us) ts_tablet_manager.cc:1151] Deleting tablet fb030a62796248869d46b682656eaeb0
1211 20:52:39.603386 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.667941 (+ 64555us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.701910 (+ 33969us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.704285 (+ 2375us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2736237}
I20251211 20:52:39.716575 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 757) took 2814 ms. Trace:
I20251211 20:52:39.716543 1168 raft_consensus.cc:359] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.716712 1168 raft_consensus.cc:385] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:39.716778 1168 raft_consensus.cc:740] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:39.716744 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet fb030a62796248869d46b682656eaeb0 (table default.Table_069bb219e15e43be995ff011e2289a84 [id=bc0b945e86cf43fdb99d56658ca1576c]) successfully deleted
I20251211 20:52:39.717005 1168 consensus_queue.cc:260] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.717226 1163 rpcz_store.cc:276] 1211 20:52:36.902478 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:36.902579 (+ 101us) service_pool.cc:225] Handling call
1211 20:52:39.716432 (+2813853us) ts_tablet_manager.cc:1151] Deleting tablet 7452dd736b0440fcb4626858e794676f
1211 20:52:39.716443 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.716538 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":663,"tablet-delete.queue_time_us":2813659}
I20251211 20:52:39.717387 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 66cdc6f297b24cfba3a3a0ae9f45d12d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:39.717718 1168 ts_tablet_manager.cc:1434] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20251211 20:52:39.717921 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 7452dd736b0440fcb4626858e794676f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7452dd736b0440fcb4626858e794676f already in progress: creating tablet
I20251211 20:52:39.718400 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 765) took 2342 ms. Trace:
I20251211 20:52:39.718429 776 raft_consensus.cc:2468] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 1.
I20251211 20:52:39.718855 1163 rpcz_store.cc:276] 1211 20:52:37.376128 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.376187 (+ 59us) service_pool.cc:225] Handling call
1211 20:52:39.718261 (+2342074us) ts_tablet_manager.cc:1151] Deleting tablet 80df719919a44aba9b4e3c62d6830cf3
1211 20:52:39.718272 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.718365 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2341879}
I20251211 20:52:39.719949 2093 raft_consensus.cc:493] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:39.720067 2093 raft_consensus.cc:515] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:39.720417 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 80df719919a44aba9b4e3c62d6830cf3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 80df719919a44aba9b4e3c62d6830cf3 already in progress: creating tablet
I20251211 20:52:39.720631 2093 leader_election.cc:290] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:39.720872 1163 tablet_replica.cc:333] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:39.721282 1163 raft_consensus.cc:2243] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:39.721736 1163 raft_consensus.cc:2272] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:39.721819 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 66cdc6f297b24cfba3a3a0ae9f45d12d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:39.722076 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "793f543d6b614ad7b3ab8e717db2ceaf" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:39.722188 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "52ef1707f6f2425aafd454f303af0333" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:39.722554 1163 ts_tablet_manager.cc:1916] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:39.722805 2101 leader_election.cc:290] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:39.723399 1043 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "793f543d6b614ad7b3ab8e717db2ceaf" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
W20251211 20:52:39.723898 711 leader_election.cc:343] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.724067 1043 raft_consensus.cc:2468] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:39.724251 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b529b38d754f46ba8cb60b19e9705256" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:39.723462 982 leader_election.cc:304] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
W20251211 20:52:39.724694 711 leader_election.cc:343] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.724785 2216 raft_consensus.cc:2804] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:39.724902 2216 raft_consensus.cc:697] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 LEADER]: Becoming Leader. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Running, Role: LEADER
I20251211 20:52:39.725150 2216 consensus_queue.cc:237] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:39.726466 983 leader_election.cc:343] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.726514 590 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet b072b759d1254859b0b8d401e3078792 (table default.Table_b56de1e85a1b42ea81308593884dadb8 [id=a137f0ddf52145a99c62444602ab3fdb]). Expected version 4 got 0
I20251211 20:52:39.723655 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b529b38d754f46ba8cb60b19e9705256" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:39.727136 1045 raft_consensus.cc:3060] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.728027 1164 ts_tablet_manager.cc:1929] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:39.728142 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 60d966c0397c4a4ea3718445e87e9703 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:39.728137 1164 log.cc:1199] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/28ef13be3b84410d9c786c43502a3478
I20251211 20:52:39.728571 1164 ts_tablet_manager.cc:1950] T 28ef13be3b84410d9c786c43502a3478 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:39.728876 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 820) took 1244 ms. Trace:
I20251211 20:52:39.728948 1164 rpcz_store.cc:276] 1211 20:52:38.484409 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.484467 (+ 58us) service_pool.cc:225] Handling call
1211 20:52:39.621336 (+1136869us) ts_tablet_manager.cc:1151] Deleting tablet 28ef13be3b84410d9c786c43502a3478
1211 20:52:39.621344 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.691169 (+ 69825us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.727907 (+ 36738us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.728860 (+ 953us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1136645}
I20251211 20:52:39.729434 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 28ef13be3b84410d9c786c43502a3478 (table default.Table_88dcd36b4447481487bc918a05511862 [id=d9453cea23a8430c98a3317856b3cec3]) successfully deleted
I20251211 20:52:39.729854 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 821) took 1235 ms. Trace:
I20251211 20:52:39.729929 1164 rpcz_store.cc:276] 1211 20:52:38.493968 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.494018 (+ 50us) service_pool.cc:225] Handling call
1211 20:52:39.729722 (+1235704us) ts_tablet_manager.cc:1151] Deleting tablet 5bcc94ec9630484cb7a3e8bf6d4f378a
1211 20:52:39.729733 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.729837 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1235531}
I20251211 20:52:39.730052 1045 raft_consensus.cc:2468] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
W20251211 20:52:39.730358 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 5bcc94ec9630484cb7a3e8bf6d4f378a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5bcc94ec9630484cb7a3e8bf6d4f378a already in progress: creating tablet
I20251211 20:52:39.730784 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 822) took 1170 ms. Trace:
I20251211 20:52:39.730880 710 leader_election.cc:304] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:39.731124 2093 raft_consensus.cc:2804] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:39.731202 2093 raft_consensus.cc:493] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:39.731258 2093 raft_consensus.cc:3060] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.731776 710 leader_election.cc:304] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:39.732115 2086 raft_consensus.cc:2804] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:39.732208 2086 raft_consensus.cc:697] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:39.732412 2086 consensus_queue.cc:237] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.733733 2093 raft_consensus.cc:515] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.734308 2093 leader_election.cc:290] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:39.735954 1043 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "793f543d6b614ad7b3ab8e717db2ceaf" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:39.736176 1043 raft_consensus.cc:3060] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.738786 1043 raft_consensus.cc:2468] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:39.739931 710 leader_election.cc:304] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:39.743270 2093 raft_consensus.cc:2804] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:39.743705 2093 raft_consensus.cc:697] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
W20251211 20:52:39.743986 593 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8b793241ee3b42b1a38e9f2f05e1173b consensus state has no leader
I20251211 20:52:39.744038 2093 consensus_queue.cc:237] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.736232 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "793f543d6b614ad7b3ab8e717db2ceaf" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:39.745306 1164 rpcz_store.cc:276] 1211 20:52:38.560113 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.560946 (+ 833us) service_pool.cc:225] Handling call
1211 20:52:39.730670 (+1169724us) ts_tablet_manager.cc:1151] Deleting tablet 75dd170c5828409fa69bd68fdf28a72b
1211 20:52:39.730680 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.730768 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1169356}
W20251211 20:52:39.745410 711 leader_election.cc:343] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.745947 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 825) took 1145 ms. Trace:
I20251211 20:52:39.746555 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 1944267def4e46949fd680ab0a369be9
I20251211 20:52:39.746708 1164 rpcz_store.cc:276] 1211 20:52:38.600958 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.603600 (+ 2642us) service_pool.cc:225] Handling call
1211 20:52:39.745827 (+1142227us) ts_tablet_manager.cc:1151] Deleting tablet 449921ea74f04a9c88eb9e094e8226c8
1211 20:52:39.745840 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.745930 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1141870}
I20251211 20:52:39.747061 2168 tablet.cc:1722] T 72e23f3033e14c9db6de8539830bac3b P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:39.747321 1164 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43888 (request call id 826) took 1128 ms. Trace:
I20251211 20:52:39.747669 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 1944267def4e46949fd680ab0a369be9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:39.747915 1015 tablet_service.cc:1552] Processing DeleteTablet for tablet 1944267def4e46949fd680ab0a369be9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:39.748098 591 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 51d12723f5194afb97d2c05fcbaf3776 consensus state has no leader
I20251211 20:52:39.748317 1164 rpcz_store.cc:276] 1211 20:52:38.619143 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.639065 (+ 19922us) service_pool.cc:225] Handling call
1211 20:52:39.747133 (+1108068us) ts_tablet_manager.cc:1151] Deleting tablet 6029c1fd563d4023a8cf7cfb8ffa0876
1211 20:52:39.747143 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.747304 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1107647}
I20251211 20:52:39.749552 1164 tablet_replica.cc:333] T 97ada0641165444a998db26796793507 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:39.749758 1164 raft_consensus.cc:2243] T 97ada0641165444a998db26796793507 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:39.749907 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 1944267def4e46949fd680ab0a369be9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:39.750478 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 75dd170c5828409fa69bd68fdf28a72b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75dd170c5828409fa69bd68fdf28a72b already in progress: creating tablet
W20251211 20:52:39.750877 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 449921ea74f04a9c88eb9e094e8226c8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 449921ea74f04a9c88eb9e094e8226c8 already in progress: creating tablet
W20251211 20:52:39.751374 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 6029c1fd563d4023a8cf7cfb8ffa0876 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6029c1fd563d4023a8cf7cfb8ffa0876 already in progress: creating tablet
W20251211 20:52:39.753142 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 80c566822fa94ada983aa7266df4bf9d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 80c566822fa94ada983aa7266df4bf9d already in progress: creating tablet
W20251211 20:52:39.753437 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet e95c63b4d99a4dd0b2c0fa384ca9d552 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e95c63b4d99a4dd0b2c0fa384ca9d552 already in progress: creating tablet
I20251211 20:52:39.754110 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_35338ef5e24d4a1c9972400654a56d83" } modify_external_catalogs: true
I20251211 20:52:39.754279 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_35338ef5e24d4a1c9972400654a56d83" } modify_external_catalogs: true
I20251211 20:52:39.755658 592 master_service.cc:954] ReplaceTablet: received request to replace tablet f3855ffb51a74101b26fd28429e65bc2 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:39.754973 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_964232e162ed4d26ba69b353b2f2065e" } modify_external_catalogs: true
I20251211 20:52:39.755836 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_964232e162ed4d26ba69b353b2f2065e" } modify_external_catalogs: true
I20251211 20:52:39.756407 2100 tablet.cc:1722] T 72e23f3033e14c9db6de8539830bac3b P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:39.749912 1164 raft_consensus.cc:2272] T 97ada0641165444a998db26796793507 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:39.766716 914 raft_consensus.cc:3060] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.767376 1160 tablet_bootstrap.cc:492] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:39.767525 1160 ts_tablet_manager.cc:1403] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.094s user 0.002s sys 0.000s
I20251211 20:52:39.768326 1160 raft_consensus.cc:359] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.768500 1160 raft_consensus.cc:385] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:39.768560 1160 raft_consensus.cc:740] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:39.770896 1813 raft_consensus.cc:493] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:39.771042 1813 raft_consensus.cc:515] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.757496 1164 ts_tablet_manager.cc:1916] T 97ada0641165444a998db26796793507 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:39.771807 1813 leader_election.cc:290] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:39.772967 847 leader_election.cc:343] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 60d966c0397c4a4ea3718445e87e9703
W20251211 20:52:39.774060 847 leader_election.cc:343] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 60d966c0397c4a4ea3718445e87e9703
I20251211 20:52:39.774169 847 leader_election.cc:304] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:39.774644 1813 raft_consensus.cc:2749] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:39.775283 2101 consensus_queue.cc:1048] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:39.775627 592 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 3b2c6064c5eb4c0f8979c485f5d85a75 (table default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f]). Expected version 1 got 0
I20251211 20:52:39.775931 592 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet e4647105ab57454d883ebfcde7bde891 (table default.Table_c817dec9a7674f699b3366fc302ead57 [id=d555d1fc6aef4a21b4d3f2804264d030]). Expected version 1 got 0
I20251211 20:52:39.776474 1165 raft_consensus.cc:2243] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:39.776562 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 7d4e63f54d8443ae8322d4659a165e76
I20251211 20:52:39.776834 1165 raft_consensus.cc:2272] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:39.777568 1015 tablet_service.cc:1552] Processing DeleteTablet for tablet 7d4e63f54d8443ae8322d4659a165e76 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:39.778182 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 7d4e63f54d8443ae8322d4659a165e76 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:39.780753 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 0f8808bfd68841e4ae30818018040cf9
I20251211 20:52:39.781471 1165 ts_tablet_manager.cc:1916] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:39.781788 1015 tablet_service.cc:1552] Processing DeleteTablet for tablet 0f8808bfd68841e4ae30818018040cf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:39.787693 594 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_d17d0534f0544cef99924179cb1376b0 [id=85ab3a18cda143e5a7f72e49e6671318] alter complete (version 2)
I20251211 20:52:39.788771 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 4df68d1b6a91458e9f212ae81aa25438 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:39.789057 1015 tablet_service.cc:1552] Processing DeleteTablet for tablet 7d4bba0aa7ea403e8dfcb74cec2e1f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:39.781986 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 0f8808bfd68841e4ae30818018040cf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:39.791826 1164 ts_tablet_manager.cc:1929] T 97ada0641165444a998db26796793507 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:39.791972 1164 log.cc:1199] T 97ada0641165444a998db26796793507 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/97ada0641165444a998db26796793507
I20251211 20:52:39.792317 1164 ts_tablet_manager.cc:1950] T 97ada0641165444a998db26796793507 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:39.795110 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 7d4e63f54d8443ae8322d4659a165e76 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:39.795504 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 0f8808bfd68841e4ae30818018040cf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:39.795560 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 97ada0641165444a998db26796793507 (table default.Table_a66a87006b684bbd87431e2aad969dcf [id=021234dec4df4bcb9ec0fee4cb1f887e]) successfully deleted
I20251211 20:52:39.795951 1160 consensus_queue.cc:260] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.796617 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ed6ce5986e224e49a69922db6e940db6" } modify_external_catalogs: true
I20251211 20:52:39.797384 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ed6ce5986e224e49a69922db6e940db6" } modify_external_catalogs: true
I20251211 20:52:39.800457 2093 raft_consensus.cc:493] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:39.801901 888 tablet_service.cc:1552] Processing DeleteTablet for tablet fb2d30147cd74d1893783be1e1694d1f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:39.802465 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 773cf53745a64f62817113b33e3182e7 with delete_type TABLET_DATA_DELETED (replaced by tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:39.803158 888 tablet_service.cc:1552] Processing DeleteTablet for tablet a9b2aa21ab124b28aebc33adba0583d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:43888
W20251211 20:52:39.805786 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet d15eff49af284a38bb1babdb231a35df because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d15eff49af284a38bb1babdb231a35df already in progress: creating tablet
W20251211 20:52:39.806468 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet c97d410272a947de8b5d7fb40fe4777d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c97d410272a947de8b5d7fb40fe4777d already in progress: creating tablet
I20251211 20:52:39.806771 1164 tablet_replica.cc:333] T 3cd7a465249246c28f07b8fb2e42675f P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:39.817571 590 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_3418cf6b51394b03afec9b03cde8861c [id=c7c19676e1484bbcb73b3bf14f147ae5] alter complete (version 1)
I20251211 20:52:39.818727 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet fbc2d4aea084492c8270ad1d87096e58
I20251211 20:52:39.822839 2100 tablet.cc:1722] T 1843dc4548134f29969d46fbc760f5d7 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:39.825222 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 66cdc6f297b24cfba3a3a0ae9f45d12d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:39.826170 888 tablet_service.cc:1552] Processing DeleteTablet for tablet fbc2d4aea084492c8270ad1d87096e58 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:39.808357 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 2197822e0f11484f9679c4858626ac2c with delete_type TABLET_DATA_DELETED (replaced by tablet 16d2ff8925e342729a71c93db378d031) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:39.809026 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 793f543d6b614ad7b3ab8e717db2ceaf with delete_type TABLET_DATA_DELETED (replaced by tablet 06b8f8e46e524545aeab6c931efaf116) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:39.820310 2168 tablet.cc:1722] T 1843dc4548134f29969d46fbc760f5d7 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:39.828514 2093 raft_consensus.cc:515] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.829627 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2197822e0f11484f9679c4858626ac2c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:39.830117 711 leader_election.cc:343] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.830315 1043 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2197822e0f11484f9679c4858626ac2c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:39.824234 739 tablet_service.cc:1552] Processing DeleteTablet for tablet bb0fc179a337469eae056c5bb3815e6b with delete_type TABLET_DATA_DELETED (replaced by tablet d53b074865f4490693152a247b0fffdc) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:39.830440 1163 ts_tablet_manager.cc:1929] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:39.825012 1168 tablet_bootstrap.cc:492] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:39.827394 754 tablet_service.cc:1552] Processing DeleteTablet for tablet fbc2d4aea084492c8270ad1d87096e58 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:39.830559 1163 log.cc:1199] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/2dde7f28e61d41f48015ba6e4f3757f2
I20251211 20:52:39.830518 1043 raft_consensus.cc:2468] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:39.830892 1163 ts_tablet_manager.cc:1950] T 2dde7f28e61d41f48015ba6e4f3757f2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:39.831149 710 leader_election.cc:304] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:39.831528 2273 raft_consensus.cc:2804] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:39.831684 2273 raft_consensus.cc:493] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:39.831797 2273 raft_consensus.cc:3060] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.832175 1168 tablet_bootstrap.cc:654] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:39.834303 2273 raft_consensus.cc:515] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.835387 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2197822e0f11484f9679c4858626ac2c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:39.836036 711 leader_election.cc:343] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.836555 1160 ts_tablet_manager.cc:1434] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.069s user 0.000s sys 0.002s
I20251211 20:52:39.836925 1160 tablet_bootstrap.cc:492] T 171d5b17dfee49159925b476b08787a4 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:39.838692 1160 tablet_bootstrap.cc:654] T 171d5b17dfee49159925b476b08787a4 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:39.838924 1164 raft_consensus.cc:2243] T 3cd7a465249246c28f07b8fb2e42675f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:39.839172 1164 raft_consensus.cc:2272] T 3cd7a465249246c28f07b8fb2e42675f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:39.839558 754 tablet_service.cc:1552] Processing DeleteTablet for tablet e6ba12f94f5f4d688f4222a4c2fb5436 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:39.840200 1164 ts_tablet_manager.cc:1916] T 3cd7a465249246c28f07b8fb2e42675f P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:39.842284 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_51015ac894894382b4c0d1931b24f4d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:39.842943 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_51015ac894894382b4c0d1931b24f4d4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:39.846884 1164 ts_tablet_manager.cc:1929] T 3cd7a465249246c28f07b8fb2e42675f P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:39.847008 1164 log.cc:1199] T 3cd7a465249246c28f07b8fb2e42675f P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/3cd7a465249246c28f07b8fb2e42675f
I20251211 20:52:39.847270 1164 ts_tablet_manager.cc:1950] T 3cd7a465249246c28f07b8fb2e42675f P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:39.848276 2278 tablet.cc:1722] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:39.850425 2281 tablet.cc:1722] T 1843dc4548134f29969d46fbc760f5d7 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:39.850975 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet fbc2d4aea084492c8270ad1d87096e58 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:39.855836 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_90271d921fcd4221a1ab532ae40b2f9f" } new_table_name: "default.Table_e00092673b0e4f09b8d7eec63e1ee034" modify_external_catalogs: true
I20251211 20:52:39.856297 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 769) took 2394 ms. Trace:
I20251211 20:52:39.856727 1163 rpcz_store.cc:276] 1211 20:52:37.461462 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.461521 (+ 59us) service_pool.cc:225] Handling call
1211 20:52:39.720804 (+2259283us) ts_tablet_manager.cc:1151] Deleting tablet 2dde7f28e61d41f48015ba6e4f3757f2
1211 20:52:39.720814 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.815761 (+ 94947us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.830312 (+ 14551us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.856253 (+ 25941us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2259064}
I20251211 20:52:39.858705 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 2dde7f28e61d41f48015ba6e4f3757f2 (table default.Table_8b66a5217c494c8ebfa10a2b986e5e78 [id=499af44163bd46e7a93b9edcdd695ae7]) successfully deleted
I20251211 20:52:39.859328 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 770) took 2382 ms. Trace:
I20251211 20:52:39.860214 1163 rpcz_store.cc:276] 1211 20:52:37.476597 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.477263 (+ 666us) service_pool.cc:225] Handling call
1211 20:52:39.859169 (+2381906us) ts_tablet_manager.cc:1151] Deleting tablet 32f8d104f6a948d1a97a83769ea82449
1211 20:52:39.859181 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.859287 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2381684}
W20251211 20:52:39.862571 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 32f8d104f6a948d1a97a83769ea82449 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 32f8d104f6a948d1a97a83769ea82449 already in progress: creating tablet
I20251211 20:52:39.863044 591 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet c8e17645a26947ab8c01f9b093fa422c (table default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5]). Expected version 2 got 0
I20251211 20:52:39.863102 1163 tablet_replica.cc:333] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:39.863298 1163 raft_consensus.cc:2243] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:39.863482 1163 raft_consensus.cc:2272] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:39.866918 1163 ts_tablet_manager.cc:1916] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:39.871848 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 0f046ea2637e4a5583b7969e19a605d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:39.878564 1163 ts_tablet_manager.cc:1929] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:39.882776 594 master_service.cc:954] ReplaceTablet: received request to replace tablet 0add14ccd2af42aeb02fad7a28ed4520 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:39.884423 2101 consensus_queue.cc:1048] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:39.887985 1043 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2197822e0f11484f9679c4858626ac2c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:39.895622 1043 raft_consensus.cc:3060] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.897941 1813 raft_consensus.cc:493] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:39.898088 1813 raft_consensus.cc:515] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.898715 1813 leader_election.cc:290] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:39.899240 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 3cd7a465249246c28f07b8fb2e42675f (table default.Table_227fbf051d5643528315ca00194583bc [id=1515561ab78d46f6b5aa37a20d90d3f5]) successfully deleted
W20251211 20:52:39.900274 847 leader_election.cc:343] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 0f046ea2637e4a5583b7969e19a605d5
I20251211 20:52:39.891928 2216 raft_consensus.cc:493] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251211 20:52:39.900563 847 leader_election.cc:343] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 0f046ea2637e4a5583b7969e19a605d5
I20251211 20:52:39.900682 1164 tablet_replica.cc:333] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:39.900681 847 leader_election.cc:304] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:39.900722 2216 raft_consensus.cc:515] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.900898 1164 raft_consensus.cc:2243] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:39.901041 1164 raft_consensus.cc:2272] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:39.901078 1813 raft_consensus.cc:2749] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:39.901347 2216 leader_election.cc:290] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:39.901643 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7d4bba0aa7ea403e8dfcb74cec2e1f9b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:39.902161 983 leader_election.cc:343] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:39.906910 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet e9ead34065e240eeb1f946319537d32a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e9ead34065e240eeb1f946319537d32a already in progress: creating tablet
I20251211 20:52:39.907512 1164 ts_tablet_manager.cc:1916] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:39.907742 1163 log.cc:1199] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/cce44b80c97948e1af7ac68e77cccfd6
I20251211 20:52:39.908156 1163 ts_tablet_manager.cc:1950] T cce44b80c97948e1af7ac68e77cccfd6 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:39.908927 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 776) took 1923 ms. Trace:
I20251211 20:52:39.909075 1163 rpcz_store.cc:276] 1211 20:52:37.985260 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.985304 (+ 44us) service_pool.cc:225] Handling call
1211 20:52:39.863031 (+1877727us) ts_tablet_manager.cc:1151] Deleting tablet cce44b80c97948e1af7ac68e77cccfd6
1211 20:52:39.863043 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.875503 (+ 12460us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.878444 (+ 2941us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.908905 (+ 30461us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1807069}
I20251211 20:52:39.909658 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet cce44b80c97948e1af7ac68e77cccfd6 (table default.Table_32ebd46bd3cc460986098c79c9b67ca7 [id=1bd6b7930f2447659da365b7c06b1640]) successfully deleted
I20251211 20:52:39.910094 1163 tablet_replica.cc:333] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:39.910122 1160 tablet_bootstrap.cc:492] T 171d5b17dfee49159925b476b08787a4 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:39.910265 1160 ts_tablet_manager.cc:1403] T 171d5b17dfee49159925b476b08787a4 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.073s user 0.001s sys 0.003s
I20251211 20:52:39.910418 1163 raft_consensus.cc:2243] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
W20251211 20:52:39.910506 582 catalog_manager.cc:4952] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 4585e05d06ef42cfb06c4ceb7524fd71 because the tablet was not found. No further retry: Not found: Tablet not found: 4585e05d06ef42cfb06c4ceb7524fd71
I20251211 20:52:39.910648 1163 raft_consensus.cc:2272] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:39.911051 1160 raft_consensus.cc:359] T 171d5b17dfee49159925b476b08787a4 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.911185 1163 ts_tablet_manager.cc:1916] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:39.911224 1160 raft_consensus.cc:385] T 171d5b17dfee49159925b476b08787a4 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:39.911267 1160 raft_consensus.cc:740] T 171d5b17dfee49159925b476b08787a4 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:39.911444 1160 consensus_queue.cc:260] T 171d5b17dfee49159925b476b08787a4 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:39.912036 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 7bc03ed8c35740f58fe48ddbaa120ea0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7bc03ed8c35740f58fe48ddbaa120ea0 already in progress: creating tablet
I20251211 20:52:39.912068 1160 ts_tablet_manager.cc:1434] T 171d5b17dfee49159925b476b08787a4 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:39.912328 1160 tablet_bootstrap.cc:492] T 33addf3d158941098f240cfcdec45d28 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:39.912698 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet a295a24e81f14e5fa126a4a58ffd08cf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a295a24e81f14e5fa126a4a58ffd08cf already in progress: creating tablet
W20251211 20:52:39.913743 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 6ad6ab53aad14cc2b6d201acfecf6723 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6ad6ab53aad14cc2b6d201acfecf6723 already in progress: creating tablet
I20251211 20:52:39.913882 1160 tablet_bootstrap.cc:654] T 33addf3d158941098f240cfcdec45d28 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:39.914649 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 48820386e41b47129a13bdd5768f021b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 48820386e41b47129a13bdd5768f021b already in progress: creating tablet
W20251211 20:52:39.915426 582 catalog_manager.cc:4952] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 4585e05d06ef42cfb06c4ceb7524fd71 because the tablet was not found. No further retry: Not found: Tablet not found: 4585e05d06ef42cfb06c4ceb7524fd71
W20251211 20:52:39.916118 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 66cdc6f297b24cfba3a3a0ae9f45d12d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66cdc6f297b24cfba3a3a0ae9f45d12d already in progress: creating tablet
I20251211 20:52:39.930732 2273 leader_election.cc:290] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:39.931001 2101 consensus_queue.cc:1048] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
W20251211 20:52:39.931007 711 consensus_peers.cc:597] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:39.931676 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7d4bba0aa7ea403e8dfcb74cec2e1f9b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:39.932782 982 leader_election.cc:343] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.932927 982 leader_election.cc:304] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:39.933257 2135 raft_consensus.cc:2749] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:39.933916 2093 leader_election.cc:290] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:39.934281 2277 raft_consensus.cc:493] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:39.934547 2277 raft_consensus.cc:515] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.935729 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e4647105ab57454d883ebfcde7bde891" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:39.936305 711 leader_election.cc:343] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.936853 2277 leader_election.cc:290] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:39.937327 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e4647105ab57454d883ebfcde7bde891" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:39.937605 1042 raft_consensus.cc:2468] T e4647105ab57454d883ebfcde7bde891 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:39.937793 588 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4abd8d4223574788846f1682e0e522ff" } new_table_name: "default.Table_8c30c4151d484db0bb9e2b9ca262248d" modify_external_catalogs: true
I20251211 20:52:39.938066 710 leader_election.cc:304] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:39.938498 2277 raft_consensus.cc:2804] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:39.938611 2277 raft_consensus.cc:493] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:39.938692 2277 raft_consensus.cc:3060] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:39.946149 2277 raft_consensus.cc:515] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.948578 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e4647105ab57454d883ebfcde7bde891" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:39.948834 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e4647105ab57454d883ebfcde7bde891" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:39.949045 1045 raft_consensus.cc:3060] T e4647105ab57454d883ebfcde7bde891 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:39.949396 711 leader_election.cc:343] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:39.952977 1045 raft_consensus.cc:2468] T e4647105ab57454d883ebfcde7bde891 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:39.953549 710 leader_election.cc:304] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:39.953982 2086 raft_consensus.cc:2804] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:39.954355 2086 raft_consensus.cc:697] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:39.954979 2086 consensus_queue.cc:237] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.965171 754 tablet_service.cc:1505] Processing CreateTablet for tablet a3b263c205eb479b8210ac79c766523f (DEFAULT_TABLE table=default.Table_948151057bf94b75a251b99e3906f4a4 [id=1ab6db436e504bfabeb44e435a5453b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:39.965657 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3b263c205eb479b8210ac79c766523f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:39.972712 2277 leader_election.cc:290] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:39.973948 1163 ts_tablet_manager.cc:1929] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:39.976329 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_198aadd702614de39863c677cd5d885b" } modify_external_catalogs: true
I20251211 20:52:39.976554 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_198aadd702614de39863c677cd5d885b" } modify_external_catalogs: true
I20251211 20:52:39.978514 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b56de1e85a1b42ea81308593884dadb8" } new_table_name: "default.Table_3de7aa51e2814433b4d98f3226b96c97" modify_external_catalogs: true
I20251211 20:52:39.983597 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_ee37a9ef24e548ebb2b6b55b54fa6f91"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:39.984256 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ee37a9ef24e548ebb2b6b55b54fa6f91 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:39.986879 591 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_339c5a45d0294fe7adf34fdbfa8e3d98"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:39.988734 2093 raft_consensus.cc:493] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251211 20:52:39.988756 591 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_339c5a45d0294fe7adf34fdbfa8e3d98 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:39.989051 2093 raft_consensus.cc:515] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.990181 2093 leader_election.cc:290] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:39.990273 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e6ba12f94f5f4d688f4222a4c2fb5436" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:39.991086 711 leader_election.cc:343] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:39.994611 711 consensus_peers.cc:597] T f737a7bfb0f44196abdc831bb22d5285 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:39.997475 754 tablet_service.cc:1505] Processing CreateTablet for tablet 1c35fd2abd054ad789fecdb29edbe55e (DEFAULT_TABLE table=default.Table_96bab30f95d5434ab659bb63869eb23d [id=86063f1c437a443c8d81b2f0cc4095de]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:39.997982 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c35fd2abd054ad789fecdb29edbe55e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:39.998629 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e6ba12f94f5f4d688f4222a4c2fb5436" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:39.999246 1045 raft_consensus.cc:2468] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:39.999740 1163 log.cc:1199] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/a2058b67ddf1413db338bfa2f64e7a77
I20251211 20:52:39.999755 710 leader_election.cc:304] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:40.000236 1163 ts_tablet_manager.cc:1950] T a2058b67ddf1413db338bfa2f64e7a77 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:40.000921 2093 raft_consensus.cc:2804] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:40.000996 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 775) took 2015 ms. Trace:
I20251211 20:52:40.001129 2093 raft_consensus.cc:493] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:40.001078 1163 rpcz_store.cc:276] 1211 20:52:37.985172 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.985403 (+ 231us) service_pool.cc:225] Handling call
1211 20:52:39.910038 (+1924635us) ts_tablet_manager.cc:1151] Deleting tablet a2058b67ddf1413db338bfa2f64e7a77
1211 20:52:39.910049 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.947850 (+ 37801us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:39.973829 (+ 25979us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.000977 (+ 27148us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1854003}
I20251211 20:52:40.001379 2093 raft_consensus.cc:3060] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:40.001539 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet a2058b67ddf1413db338bfa2f64e7a77 (table default.Table_91453272b5a341baab304eedbc5b2b34 [id=e1b92959cd2f46469c90526286f30b27]) successfully deleted
I20251211 20:52:40.003327 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 778) took 2017 ms. Trace:
I20251211 20:52:40.003419 1163 rpcz_store.cc:276] 1211 20:52:37.985998 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.986044 (+ 46us) service_pool.cc:225] Handling call
1211 20:52:40.003190 (+2017146us) ts_tablet_manager.cc:1151] Deleting tablet c91ff29847d54e55b19321652a857e9f
1211 20:52:40.003202 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.003309 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1943557}
I20251211 20:52:39.971498 1043 raft_consensus.cc:2468] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:40.003937 1017 tablet_service.cc:1505] Processing CreateTablet for tablet a3b263c205eb479b8210ac79c766523f (DEFAULT_TABLE table=default.Table_948151057bf94b75a251b99e3906f4a4 [id=1ab6db436e504bfabeb44e435a5453b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.004317 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3b263c205eb479b8210ac79c766523f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.004382 710 leader_election.cc:304] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:40.004971 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet c91ff29847d54e55b19321652a857e9f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c91ff29847d54e55b19321652a857e9f already in progress: creating tablet
I20251211 20:52:40.005016 2093 raft_consensus.cc:515] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.005414 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 779) took 2019 ms. Trace:
I20251211 20:52:40.005813 1163 rpcz_store.cc:276] 1211 20:52:37.986232 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.986271 (+ 39us) service_pool.cc:225] Handling call
1211 20:52:40.005288 (+2019017us) ts_tablet_manager.cc:1151] Deleting tablet 098054406a1d459b824a094f50e71e7e
1211 20:52:40.005299 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.005395 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1945002}
I20251211 20:52:40.006206 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e6ba12f94f5f4d688f4222a4c2fb5436" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:40.006448 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 098054406a1d459b824a094f50e71e7e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 098054406a1d459b824a094f50e71e7e already in progress: creating tablet
W20251211 20:52:40.006749 711 leader_election.cc:343] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:40.006922 2273 raft_consensus.cc:2804] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:40.007074 2273 raft_consensus.cc:697] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:40.007314 2273 consensus_queue.cc:237] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.007517 2093 leader_election.cc:290] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:40.009263 1164 ts_tablet_manager.cc:1929] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:40.009405 1164 log.cc:1199] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/60d966c0397c4a4ea3718445e87e9703
I20251211 20:52:40.009711 1164 ts_tablet_manager.cc:1950] T 60d966c0397c4a4ea3718445e87e9703 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:40.010982 1164 tablet_replica.cc:333] T fb2d30147cd74d1893783be1e1694d1f P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:40.011165 1164 raft_consensus.cc:2243] T fb2d30147cd74d1893783be1e1694d1f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:40.011302 1164 raft_consensus.cc:2272] T fb2d30147cd74d1893783be1e1694d1f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.006814 1163 tablet_replica.cc:333] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:40.011790 1163 raft_consensus.cc:2243] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:40.012106 1163 raft_consensus.cc:2272] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.012647 1163 ts_tablet_manager.cc:1916] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.011736 1043 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e6ba12f94f5f4d688f4222a4c2fb5436" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:40.017663 1043 raft_consensus.cc:3060] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:40.018545 710 consensus_peers.cc:597] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c74427cb9d6f4a679e43eae796ff8c3f. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:40.020346 1043 raft_consensus.cc:2468] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:40.021217 1165 ts_tablet_manager.cc:1929] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:40.021453 1165 log.cc:1199] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/25454e67437541d9ac2bfab29bad55f9
I20251211 20:52:40.021865 1165 ts_tablet_manager.cc:1950] T 25454e67437541d9ac2bfab29bad55f9 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:40.022708 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 808) took 2079 ms. Trace:
I20251211 20:52:40.026523 1164 ts_tablet_manager.cc:1916] T fb2d30147cd74d1893783be1e1694d1f P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.031510 592 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_99917cb4c70840e19004ad381746d77e [id=6abb187bfd214e91b2e11267065a2841] alter complete (version 1)
I20251211 20:52:40.032044 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_dde47886ad4d4ea8a332c68e13085fe0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:40.032631 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dde47886ad4d4ea8a332c68e13085fe0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:40.025297 1015 tablet_service.cc:1505] Processing CreateTablet for tablet 1c35fd2abd054ad789fecdb29edbe55e (DEFAULT_TABLE table=default.Table_96bab30f95d5434ab659bb63869eb23d [id=86063f1c437a443c8d81b2f0cc4095de]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.026170 1020 tablet_service.cc:1505] Processing CreateTablet for tablet b6cc819332e1445c96ce3ffac4861454 (DEFAULT_TABLE table=default.Table_db4c0f706e0643b39d39ffa84352826d [id=4d459c42bf584068ac3851227d80331a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.027040 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 0b38de97bc4a42af91b2f2cd70ece669 (DEFAULT_TABLE table=default.Table_b6ff995fa24746b8a8d1cfe5a656893f [id=83f8ca2368b9482ca769fd0da3d3606f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.039942 596 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet bfb52935296443c58190d4d06e2e6e9c (table default.Table_3721b5725e8c4297a392d3986a081128 [id=383392d835da4c50848068d83261cecc]). Expected version 7 got 0
I20251211 20:52:40.042690 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_a32f150ce5604ec999ac5f3edc6324b4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:40.043624 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a32f150ce5604ec999ac5f3edc6324b4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:40.043990 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c35fd2abd054ad789fecdb29edbe55e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.044559 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6cc819332e1445c96ce3ffac4861454. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.045029 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b38de97bc4a42af91b2f2cd70ece669. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.045257 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_25d3985a897841b8b55ad7c44bbff154"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:40.045748 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_25d3985a897841b8b55ad7c44bbff154 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:40.047915 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4c0455a5b4f44412803475d1c4852b48 has no consensus state
I20251211 20:52:40.048266 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_491ce822e76742a1882861a98eff45b5" } new_table_name: "default.Table_fa6592c2e2624d81ad21abb13ed36045" modify_external_catalogs: true
I20251211 20:52:40.053061 888 tablet_service.cc:1505] Processing CreateTablet for tablet 1c35fd2abd054ad789fecdb29edbe55e (DEFAULT_TABLE table=default.Table_96bab30f95d5434ab659bb63869eb23d [id=86063f1c437a443c8d81b2f0cc4095de]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.053503 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c35fd2abd054ad789fecdb29edbe55e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.064441 1164 ts_tablet_manager.cc:1929] T fb2d30147cd74d1893783be1e1694d1f P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:40.064594 1164 log.cc:1199] T fb2d30147cd74d1893783be1e1694d1f P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/fb2d30147cd74d1893783be1e1694d1f
I20251211 20:52:40.064868 1164 ts_tablet_manager.cc:1950] T fb2d30147cd74d1893783be1e1694d1f P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
W20251211 20:52:40.066874 588 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6212b3b4425543168837946a54d60834 consensus state has no leader
I20251211 20:52:40.069849 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_fb5ab0d4096b4494b7ca5a55deb992c2" } modify_external_catalogs: true
I20251211 20:52:40.070200 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_fb5ab0d4096b4494b7ca5a55deb992c2" } modify_external_catalogs: true
I20251211 20:52:40.070353 1163 ts_tablet_manager.cc:1929] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20251211 20:52:40.072606 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_4b87bd2fa22f4d59b01baeca62ce98f8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:40.073140 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4b87bd2fa22f4d59b01baeca62ce98f8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:40.075268 1164 tablet_replica.cc:333] T 773cf53745a64f62817113b33e3182e7 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:40.070605 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 4810f8d8b53a45eb939bf3c8bf1b4e77
I20251211 20:52:40.075495 1164 raft_consensus.cc:2243] T 773cf53745a64f62817113b33e3182e7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:40.075687 1164 raft_consensus.cc:2272] T 773cf53745a64f62817113b33e3182e7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.076306 1163 log.cc:1199] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/3eac21a07ba34e1fa275e25ee9dcabb1
I20251211 20:52:40.076256 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_4e6562227887405aae36515729c90a28"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:40.076694 1163 ts_tablet_manager.cc:1950] T 3eac21a07ba34e1fa275e25ee9dcabb1 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
W20251211 20:52:40.076803 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4e6562227887405aae36515729c90a28 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:40.077488 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 780) took 2091 ms. Trace:
I20251211 20:52:40.077572 1163 rpcz_store.cc:276] 1211 20:52:37.986460 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.986500 (+ 40us) service_pool.cc:225] Handling call
1211 20:52:40.006769 (+2020269us) ts_tablet_manager.cc:1151] Deleting tablet 3eac21a07ba34e1fa275e25ee9dcabb1
1211 20:52:40.006780 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.029266 (+ 22486us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.070245 (+ 40979us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.077465 (+ 7220us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1946425}
I20251211 20:52:40.079162 1019 tablet_service.cc:1552] Processing DeleteTablet for tablet 4810f8d8b53a45eb939bf3c8bf1b4e77 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:40.079210 1164 ts_tablet_manager.cc:1916] T 773cf53745a64f62817113b33e3182e7 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.080119 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_23402fab961249179b57df4d79b53a18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:40.081187 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_23402fab961249179b57df4d79b53a18 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:40.090407 594 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 0add14ccd2af42aeb02fad7a28ed4520
I20251211 20:52:40.092836 1019 tablet_service.cc:1552] Processing DeleteTablet for tablet 0add14ccd2af42aeb02fad7a28ed4520 with delete_type TABLET_DATA_DELETED (replaced by tablet 84cca3ee8fc446fcaf9971171235cf55) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:40.093556 594 catalog_manager.cc:6724] ReplaceTablet: tablet 0add14ccd2af42aeb02fad7a28ed4520 deleted and replaced by tablet 84cca3ee8fc446fcaf9971171235cf55
I20251211 20:52:40.094327 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a3a395764457481b9b9082279213a9ef" } new_table_name: "default.Table_c9d86e82aebb4ef0bc83789720352e38" modify_external_catalogs: true
I20251211 20:52:40.096645 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 781) took 2109 ms. Trace:
W20251211 20:52:40.099656 1166 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.588s user 0.028s sys 0.076s
W20251211 20:52:40.099871 1166 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.589s user 0.028s sys 0.076s
I20251211 20:52:40.100008 1163 rpcz_store.cc:276] 1211 20:52:37.986745 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.986787 (+ 42us) service_pool.cc:225] Handling call
1211 20:52:40.096485 (+2109698us) ts_tablet_manager.cc:1151] Deleting tablet 042f8bf13a7a4712b5bdc04671cd9938
1211 20:52:40.096499 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.096619 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2036101}
I20251211 20:52:40.100214 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 60d966c0397c4a4ea3718445e87e9703 (table default.Table_f51ac924827c4dbea700109489db0963 [id=cae3f6884694487e82fc81328e9527d1]) successfully deleted
I20251211 20:52:40.100306 877 tablet_service.cc:1505] Processing CreateTablet for tablet 0b38de97bc4a42af91b2f2cd70ece669 (DEFAULT_TABLE table=default.Table_b6ff995fa24746b8a8d1cfe5a656893f [id=83f8ca2368b9482ca769fd0da3d3606f]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:40.100526 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 1944267def4e46949fd680ab0a369be9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1944267def4e46949fd680ab0a369be9 already in progress: creating tablet
W20251211 20:52:40.100690 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 042f8bf13a7a4712b5bdc04671cd9938 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 042f8bf13a7a4712b5bdc04671cd9938 already in progress: creating tablet
I20251211 20:52:40.100742 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b38de97bc4a42af91b2f2cd70ece669. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:40.100746 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 7d4e63f54d8443ae8322d4659a165e76 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d4e63f54d8443ae8322d4659a165e76 already in progress: creating tablet
W20251211 20:52:40.100952 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 0f8808bfd68841e4ae30818018040cf9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0f8808bfd68841e4ae30818018040cf9 already in progress: creating tablet
I20251211 20:52:40.101153 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 782) took 2114 ms. Trace:
I20251211 20:52:40.101214 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet fb2d30147cd74d1893783be1e1694d1f (table default.Table_59fd4cd22d6b4065bc70dc11849fa628 [id=c5b5110e2f1f4eb29d233b6a9eef4737]) successfully deleted
I20251211 20:52:40.101231 1163 rpcz_store.cc:276] 1211 20:52:37.986956 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.987015 (+ 59us) service_pool.cc:225] Handling call
1211 20:52:40.101018 (+2114003us) ts_tablet_manager.cc:1151] Deleting tablet 14f367adb60744c8a593d3b9d4711823
1211 20:52:40.101029 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.101131 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2040603}
W20251211 20:52:40.101687 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 14f367adb60744c8a593d3b9d4711823 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14f367adb60744c8a593d3b9d4711823 already in progress: creating tablet
I20251211 20:52:40.103864 885 tablet_service.cc:1505] Processing CreateTablet for tablet 1bcf36efcd204fea868fd04ee7a9cdc6 (DEFAULT_TABLE table=default.Table_b7a123d140e64879b3fb2606998af217 [id=eee1b058701c499fac48dfab3dc873d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.104210 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bcf36efcd204fea868fd04ee7a9cdc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.104955 1163 tablet_replica.cc:333] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:40.105144 1163 raft_consensus.cc:2243] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:40.105304 1163 raft_consensus.cc:2272] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.105841 1163 ts_tablet_manager.cc:1916] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.107527 1166 tablet_bootstrap.cc:492] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:40.107720 1166 ts_tablet_manager.cc:1403] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.601s user 0.028s sys 0.078s
I20251211 20:52:40.107734 1164 ts_tablet_manager.cc:1929] T 773cf53745a64f62817113b33e3182e7 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:40.107831 1164 log.cc:1199] T 773cf53745a64f62817113b33e3182e7 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/773cf53745a64f62817113b33e3182e7
I20251211 20:52:40.108129 1164 ts_tablet_manager.cc:1950] T 773cf53745a64f62817113b33e3182e7 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:40.108698 2284 raft_consensus.cc:493] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9230f94a6a1a49cead51aeea9f069087)
I20251211 20:52:40.108960 1166 raft_consensus.cc:359] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.109052 2284 raft_consensus.cc:515] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:40.109225 1166 raft_consensus.cc:385] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.109340 1166 raft_consensus.cc:740] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.109613 1166 consensus_queue.cc:260] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.110354 2284 leader_election.cc:290] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1830b0c58cfe495989f3d911dda0409a (127.0.104.62:41123), 9230f94a6a1a49cead51aeea9f069087 (127.0.104.60:42087)
I20251211 20:52:40.110617 1350 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11f5f94732f049d0be4c97cebfffcb05" candidate_term: 2 candidate_status { last_received { term: 1 index: 848 } } ignore_live_leader: false dest_uuid: "1830b0c58cfe495989f3d911dda0409a" is_pre_election: true
I20251211 20:52:40.100315 888 tablet_service.cc:1505] Processing CreateTablet for tablet b6cc819332e1445c96ce3ffac4861454 (DEFAULT_TABLE table=default.Table_db4c0f706e0643b39d39ffa84352826d [id=4d459c42bf584068ac3851227d80331a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.113113 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6cc819332e1445c96ce3ffac4861454. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.117163 607 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11f5f94732f049d0be4c97cebfffcb05" candidate_term: 2 candidate_status { last_received { term: 1 index: 848 } } ignore_live_leader: false dest_uuid: "9230f94a6a1a49cead51aeea9f069087" is_pre_election: true
I20251211 20:52:40.118326 1166 ts_tablet_manager.cc:1434] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.010s user 0.001s sys 0.000s
I20251211 20:52:40.123039 1166 tablet_bootstrap.cc:492] T a3b1aadf17ea4bf0a896d23cd543fca7 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:40.124564 1166 tablet_bootstrap.cc:654] T a3b1aadf17ea4bf0a896d23cd543fca7 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.027802 739 tablet_service.cc:1505] Processing CreateTablet for tablet 0b38de97bc4a42af91b2f2cd70ece669 (DEFAULT_TABLE table=default.Table_b6ff995fa24746b8a8d1cfe5a656893f [id=83f8ca2368b9482ca769fd0da3d3606f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.128015 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b38de97bc4a42af91b2f2cd70ece669. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.128068 1160 tablet_bootstrap.cc:492] T 33addf3d158941098f240cfcdec45d28 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:40.128193 1160 ts_tablet_manager.cc:1403] T 33addf3d158941098f240cfcdec45d28 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.216s user 0.028s sys 0.039s
I20251211 20:52:40.128932 1160 raft_consensus.cc:359] T 33addf3d158941098f240cfcdec45d28 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.129094 1160 raft_consensus.cc:385] T 33addf3d158941098f240cfcdec45d28 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.129137 1160 raft_consensus.cc:740] T 33addf3d158941098f240cfcdec45d28 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.129359 1160 consensus_queue.cc:260] T 33addf3d158941098f240cfcdec45d28 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.131750 1164 tablet_replica.cc:333] T a9b2aa21ab124b28aebc33adba0583d2 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:40.131945 1164 raft_consensus.cc:2243] T a9b2aa21ab124b28aebc33adba0583d2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:40.132097 1164 raft_consensus.cc:2272] T a9b2aa21ab124b28aebc33adba0583d2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.132614 1164 ts_tablet_manager.cc:1916] T a9b2aa21ab124b28aebc33adba0583d2 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.134761 1160 ts_tablet_manager.cc:1434] T 33addf3d158941098f240cfcdec45d28 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20251211 20:52:40.135154 1160 tablet_bootstrap.cc:492] T 648f43e95e034f2188a20e1b837f2818 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:40.136930 1160 tablet_bootstrap.cc:654] T 648f43e95e034f2188a20e1b837f2818 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.028679 747 tablet_service.cc:1505] Processing CreateTablet for tablet b6cc819332e1445c96ce3ffac4861454 (DEFAULT_TABLE table=default.Table_db4c0f706e0643b39d39ffa84352826d [id=4d459c42bf584068ac3851227d80331a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.037534 710 leader_election.cc:304] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:40.140276 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 102792us
I20251211 20:52:40.142400 2101 raft_consensus.cc:2804] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:40.142599 2101 raft_consensus.cc:697] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:40.044925 2019 raft_consensus.cc:493] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:40.078996 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 4810f8d8b53a45eb939bf3c8bf1b4e77 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:40.092016 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 0add14ccd2af42aeb02fad7a28ed4520 with delete_type TABLET_DATA_DELETED (replaced by tablet 84cca3ee8fc446fcaf9971171235cf55) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:40.142851 2101 consensus_queue.cc:237] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.143080 2019 raft_consensus.cc:515] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.144058 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14e2650f8ec1482ea9511b989fdc963c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:40.144672 711 leader_election.cc:343] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:40.145110 2019 leader_election.cc:290] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:40.120023 1168 tablet_bootstrap.cc:492] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:40.145443 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14e2650f8ec1482ea9511b989fdc963c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:40.145540 1168 ts_tablet_manager.cc:1403] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.321s user 0.010s sys 0.001s
I20251211 20:52:40.145629 1047 raft_consensus.cc:2468] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:40.146023 710 leader_election.cc:304] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:40.148209 2286 raft_consensus.cc:2804] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:40.148393 2286 raft_consensus.cc:493] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:40.148561 2286 raft_consensus.cc:3060] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:40.150056 711 consensus_peers.cc:597] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:40.151322 2286 raft_consensus.cc:515] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:40.152328 711 consensus_peers.cc:597] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:40.152429 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14e2650f8ec1482ea9511b989fdc963c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:40.152506 1165 rpcz_store.cc:276] 1211 20:52:37.943195 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.943676 (+ 481us) service_pool.cc:225] Handling call
1211 20:52:39.653239 (+1709563us) ts_tablet_manager.cc:1151] Deleting tablet 25454e67437541d9ac2bfab29bad55f9
1211 20:52:39.653251 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:39.874049 (+220798us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.021087 (+147038us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.022640 (+ 1553us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":9340,"tablet-delete.queue_time_us":1709398}
I20251211 20:52:40.122906 1784 leader_election.cc:304] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f5f94732f049d0be4c97cebfffcb05; no voters: 1830b0c58cfe495989f3d911dda0409a, 9230f94a6a1a49cead51aeea9f069087
W20251211 20:52:40.152952 711 leader_election.cc:343] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:40.154280 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6cc819332e1445c96ce3ffac4861454. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.161182 1166 tablet_bootstrap.cc:492] T a3b1aadf17ea4bf0a896d23cd543fca7 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:40.161419 1166 ts_tablet_manager.cc:1403] T a3b1aadf17ea4bf0a896d23cd543fca7 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.038s user 0.003s sys 0.000s
I20251211 20:52:40.162304 1166 raft_consensus.cc:359] T a3b1aadf17ea4bf0a896d23cd543fca7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.162559 1166 raft_consensus.cc:385] T a3b1aadf17ea4bf0a896d23cd543fca7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.162652 1166 raft_consensus.cc:740] T a3b1aadf17ea4bf0a896d23cd543fca7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.162921 1166 consensus_queue.cc:260] T a3b1aadf17ea4bf0a896d23cd543fca7 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:39.161899 1829 rpcz_store.cc:276] 1211 20:52:34.476533 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:34.476594 (+ 61us) service_pool.cc:225] Handling call
1211 20:52:37.381596 (+2905002us) raft_consensus.cc:1471] Updating replica for 409 ops
1211 20:52:37.455548 (+ 73952us) raft_consensus.cc:1534] Early marking committed up to index 0
1211 20:52:37.455558 (+ 10us) raft_consensus.cc:1539] Triggering prepare for 409 ops
1211 20:52:37.508554 (+ 52996us) write_op.cc:270] Start()
1211 20:52:37.508622 (+ 68us) write_op.cc:276] Timestamp: P: 1765486348143214 usec, L: 0
1211 20:52:37.632178 (+123556us) write_op.cc:270] Start()
1211 20:52:37.632219 (+ 41us) write_op.cc:276] Timestamp: P: 1765486348146138 usec, L: 0
1211 20:52:37.632488 (+ 269us) write_op.cc:270] Start()
1211 20:52:37.632527 (+ 39us) write_op.cc:276] Timestamp: P: 1765486348146677 usec, L: 0
1211 20:52:37.632684 (+ 157us) write_op.cc:270] Start()
1211 20:52:37.632703 (+ 19us) write_op.cc:276] Timestamp: P: 1765486348147160 usec, L: 0
1211 20:52:37.632865 (+ 162us) write_op.cc:270] Start()
1211 20:52:37.632883 (+ 18us) write_op.cc:276] Timestamp: P: 1765486348147912 usec, L: 0
1211 20:52:37.633001 (+ 118us) write_op.cc:270] Start()
1211 20:52:37.633016 (+ 15us) write_op.cc:276] Timestamp: P: 1765486348148445 usec, L: 0
1211 20:52:37.633150 (+ 134us) write_op.cc:270] Start()
1211 20:52:37.633761 (+ 611us) write_op.cc:276] Timestamp: P: 1765486348149956 usec, L: 0
1211 20:52:37.633952 (+ 191us) write_op.cc:270] Start()
1211 20:52:37.633975 (+ 23us) write_op.cc:276] Timestamp: P: 1765486348150543 usec, L: 0
1211 20:52:37.634124 (+ 149us) write_op.cc:270] Start()
1211 20:52:37.634145 (+ 21us) write_op.cc:276] Timestamp: P: 1765486348150983 usec, L: 0
1211 20:52:37.634280 (+ 135us) write_op.cc:270] Start()
1211 20:52:37.634300 (+ 20us) write_op.cc:276] Timestamp: P: 1765486348152855 usec, L: 0
1211 20:52:37.634450 (+ 150us) write_op.cc:270] Start()
1211 20:52:37.634468 (+ 18us) write_op.cc:276] Timestamp: P: 1765486348185844 usec, L: 0
1211 20:52:37.634614 (+ 146us) write_op.cc:270] Start()
1211 20:52:37.634633 (+ 19us) write_op.cc:276] Timestamp: P: 1765486348234116 usec, L: 0
1211 20:52:37.634773 (+ 140us) write_op.cc:270] Start()
1211 20:52:37.634795 (+ 22us) write_op.cc:276] Timestamp: P: 1765486348234765 usec, L: 0
1211 20:52:37.634911 (+ 116us) write_op.cc:270] Start()
1211 20:52:37.634928 (+ 17us) write_op.cc:276] Timestamp: P: 1765486348286192 usec, L: 0
1211 20:52:37.635053 (+ 125us) write_op.cc:270] Start()
1211 20:52:37.635069 (+ 16us) write_op.cc:276] Timestamp: P: 1765486348308854 usec, L: 0
1211 20:52:37.635180 (+ 111us) write_op.cc:270] Start()
1211 20:52:37.635197 (+ 17us) write_op.cc:276] Timestamp: P: 1765486348310398 usec, L: 0
1211 20:52:37.635321 (+ 124us) write_op.cc:270] Start()
1211 20:52:37.635344 (+ 23us) write_op.cc:276] Timestamp: P: 1765486348311817 usec, L: 0
1211 20:52:37.635462 (+ 118us) write_op.cc:270] Start()
1211 20:52:37.635479 (+ 17us) write_op.cc:276] Timestamp: P: 1765486348328241 usec, L: 0
1211 20:52:37.668525 (+ 33046us) write_op.cc:270] Start()
1211 20:52:37.668575 (+ 50us) write_op.cc:276] Timestamp: P: 1765486348357067 usec, L: 0
1211 20:52:37.668829 (+ 254us) write_op.cc:270] Start()
1211 20:52:37.668857 (+ 28us) write_op.cc:276] Timestamp: P: 1765486348366156 usec, L: 0
1211 20:52:37.669016 (+ 159us) write_op.cc:270] Start()
1211 20:52:37.669036 (+ 20us) write_op.cc:276] Timestamp: P: 1765486348384134 usec, L: 0
1211 20:52:37.669163 (+ 127us) write_op.cc:270] Start()
1211 20:52:37.669181 (+ 18us) write_op.cc:276] Timestamp: P: 1765486348388132 usec, L: 0
1211 20:52:37.669322 (+ 141us) write_op.cc:270] Start()
1211 20:52:37.669341 (+ 19us) write_op.cc:276] Timestamp: P: 1765486348388696 usec, L: 0
1211 20:52:37.669464 (+ 123us) write_op.cc:270] Start()
1211 20:52:37.669480 (+ 16us) write_op.cc:276] Timestamp: P: 1765486348397877 usec, L: 0
1211 20:52:37.669667 (+ 187us) write_op.cc:270] Start()
1211 20:52:37.669692 (+ 25us) write_op.cc:276] Timestamp: P: 1765486348507974 usec, L: 0
1211 20:52:37.669857 (+ 165us) write_op.cc:270] Start()
1211 20:52:37.669878 (+ 21us) write_op.cc:276] Timestamp: P: 1765486348508578 usec, L: 0
1211 20:52:37.670035 (+ 157us) write_op.cc:270] Start()
1211 20:52:37.670054 (+ 19us) write_op.cc:276] Timestamp: P: 1765486348509145 usec, L: 0
1211 20:52:37.681270 (+ 11216us) write_op.cc:270] Start()
1211 20:52:37.681307 (+ 37us) write_op.cc:276] Timestamp: P: 1765486348509609 usec, L: 0
1211 20:52:37.681506 (+ 199us) write_op.cc:270] Start()
1211 20:52:37.681529 (+ 23us) write_op.cc:276] Timestamp: P: 1765486348534016 usec, L: 0
1211 20:52:37.681662 (+ 133us) write_op.cc:270] Start()
1211 20:52:37.681678 (+ 16us) write_op.cc:276] Timestamp: P: 1765486348535522 usec, L: 0
1211 20:52:37.681802 (+ 124us) write_op.cc:270] Start()
1211 20:52:37.681818 (+ 16us) write_op.cc:276] Timestamp: P: 1765486348536110 usec, L: 0
1211 20:52:37.682507 (+ 689us) write_op.cc:270] Start()
1211 20:52:37.682532 (+ 25us) write_op.cc:276] Timestamp: P: 1765486348536640 usec, L: 0
1211 20:52:37.682939 (+ 407us) write_op.cc:270] Start()
1211 20:52:37.682965 (+ 26us) write_op.cc:276] Timestamp: P: 1765486348569232 usec, L: 0
1211 20:52:37.683105 (+ 140us) write_op.cc:270] Start()
1211 20:52:37.683567 (+ 462us) write_op.cc:276] Timestamp: P: 1765486348570146 usec, L: 0
1211 20:52:37.683767 (+ 200us) write_op.cc:270] Start()
1211 20:52:37.683790 (+ 23us) write_op.cc:276] Timestamp: P: 1765486348573786 usec, L: 0
1211 20:52:37.685455 (+ 1665us) write_op.cc:270] Start()
1211 20:52:37.685483 (+ 28us) write_op.cc:276] Timestamp: P: 1765486348588637 usec, L: 0
1211 20:52:37.685638 (+ 155us) write_op.cc:270] Start()
1211 20:52:37.685670 (+ 32us) write_op.cc:276] Timestamp: P: 1765486348589713 usec, L: 0
1211 20:52:37.685838 (+ 168us) write_op.cc:270] Start()
1211 20:52:37.685855 (+ 17us) write_op.cc:276] Timestamp: P: 1765486348602609 usec, L: 0
1211 20:52:37.685984 (+ 129us) write_op.cc:270] Start()
1211 20:52:37.686001 (+ 17us) write_op.cc:276] Timestamp: P: 1765486348603206 usec, L: 0
1211 20:52:37.686124 (+ 123us) write_op.cc:270] Start()
1211 20:52:37.686142 (+ 18us) write_op.cc:276] Timestamp: P: 1765486348653899 usec, L: 0
1211 20:52:37.686276 (+ 134us) write_op.cc:270] Start()
1211 20:52:37.686294 (+ 18us) write_op.cc:276] Timestamp: P: 1765486348685371 usec, L: 0
1211 20:52:37.686416 (+ 122us) write_op.cc:270] Start()
1211 20:52:37.686434 (+ 18us) write_op.cc:276] Timestamp: P: 1765486348705045 usec, L: 0
1211 20:52:37.686542 (+ 108us) write_op.cc:270] Start()
1211 20:52:37.686559 (+ 17us) write_op.cc:276] Timestamp: P: 1765486348715365 usec, L: 0
1211 20:52:37.686683 (+ 124us) write_op.cc:270] Start()
1211 20:52:37.686701 (+ 18us) write_op.cc:276] Timestamp: P: 1765486348715991 usec, L: 0
1211 20:52:37.686831 (+ 130us) write_op.cc:270] Start()
1211 20:52:37.686848 (+ 17us) write_op.cc:276] Timestamp: P: 1765486348716557 usec, L: 0
1211 20:52:37.686960 (+ 112us) write_op.cc:270] Start()
1211 20:52:37.686977 (+ 17us) write_op.cc:276] Timestamp: P: 1765486348717064 usec, L: 0
1211 20:52:37.687087 (+ 110us) write_op.cc:270] Start()
1211 20:52:37.687104 (+ 17us) write_op.cc:276] Timestamp: P: 1765486348717536 usec, L: 0
1211 20:52:37.687222 (+ 118us) write_op.cc:270] Start()
1211 20:52:37.687239 (+ 17us) write_op.cc:276] Timestamp: P: 1765486348720990 usec, L: 0
1211 20:52:37.687365 (+ 126us) write_op.cc:270] Start()
1211 20:52:37.687382 (+ 17us) write_op.cc:276] Timestamp: P: 1765486348724027 usec, L: 0
1211 20:52:37.687523 (+ 141us) write_op.cc:270] Start()
1211 20:52:37.687542 (+ 19us) write_op.cc:276] Timestamp: P: 1765486348734543 usec, L: 0
1211 20:52:37.713999 (+ 26457us) write_op.cc:270] Start()
1211 20:52:37.714033 (+ 34us) write_op.cc:276] Timestamp: P: 1765486348737912 usec, L: 0
1211 20:52:37.714209 (+ 176us) write_op.cc:270] Start()
1211 20:52:37.714227 (+ 18us) write_op.cc:276] Timestamp: P: 1765486348738422 usec, L: 0
1211 20:52:37.714358 (+ 131us) write_op.cc:270] Start()
1211 20:52:37.714375 (+ 17us) write_op.cc:276] Timestamp: P: 1765486348739056 usec, L: 0
1211 20:52:37.714491 (+ 116us) write_op.cc:270] Start()
1211 20:52:37.714510 (+ 19us) write_op.cc:276] Timestamp: P: 1765486348739527 usec, L: 0
1211 20:52:37.714640 (+ 130us) write_op.cc:270] Start()
1211 20:52:37.714659 (+ 19us) write_op.cc:276] Timestamp: P: 1765486348757439 usec, L: 0
1211 20:52:37.714783 (+ 124us) write_op.cc:270] Start()
1211 20:52:37.714802 (+ 19us) write_op.cc:276] Timestamp: P: 1765486348792623 usec, L: 0
1211 20:52:37.714934 (+ 132us) write_op.cc:270] Start()
1211 20:52:37.714952 (+ 18us) write_op.cc:276] Timestamp: P: 1765486348795590 usec, L: 0
1211 20:52:37.715065 (+ 113us) write_op.cc:270] Start()
1211 20:52:37.715081 (+ 16us) write_op.cc:276] Timestamp: P: 1765486348816861 usec, L: 0
1211 20:52:37.715183 (+ 102us) write_op.cc:270] Start()
1211 20:52:37.715201 (+ 18us) write_op.cc:276] Timestamp: P: 1765486348826324 usec, L: 0
1211 20:52:37.715324 (+ 123us) write_op.cc:270] Start()
1211 20:52:37.715364 (+ 40us) write_op.cc:276] Timestamp: P: 1765486348834410 usec, L: 0
1211 20:52:37.715479 (+ 115us) write_op.cc:270] Start()
1211 20:52:37.715495 (+ 16us) write_op.cc:276] Timestamp: P: 1765486348841730 usec, L: 0
1211 20:52:37.723293 (+ 7798us) write_op.cc:270] Start()
1211 20:52:37.723330 (+ 37us) write_op.cc:276] Timestamp: P: 1765486348845090 usec, L: 0
1211 20:52:37.723551 (+ 221us) write_op.cc:270] Start()
1211 20:52:37.723600 (+ 49us) write_op.cc:276] Timestamp: P: 1765486349122280 usec, L: 0
1211 20:52:37.723765 (+ 165us) write_op.cc:270] Start()
1211 20:52:37.723785 (+ 20us) write_op.cc:276] Timestamp: P: 1765486349135832 usec, L: 0
1211 20:52:37.723949 (+ 164us) write_op.cc:270] Start()
1211 20:52:37.723972 (+ 23us) write_op.cc:276] Timestamp: P: 1765486349138363 usec, L: 0
1211 20:52:37.724100 (+ 128us) write_op.cc:270] Start()
1211 20:52:37.724120 (+ 20us) write_op.cc:276] Timestamp: P: 1765486349139077 usec, L: 0
1211 20:52:37.724241 (+ 121us) write_op.cc:270] Start()
1211 20:52:37.724260 (+ 19us) write_op.cc:276] Timestamp: P: 1765486349199317 usec, L: 0
1211 20:52:37.724375 (+ 115us) write_op.cc:270] Start()
1211 20:52:37.724391 (+ 16us) write_op.cc:276] Timestamp: P: 1765486349200332 usec, L: 0
1211 20:52:37.724501 (+ 110us) write_op.cc:270] Start()
1211 20:52:37.724518 (+ 17us) write_op.cc:276] Timestamp: P: 1765486349204798 usec, L: 0
1211 20:52:37.724615 (+ 97us) write_op.cc:270] Start()
1211 20:52:37.724631 (+ 16us) write_op.cc:276] Timestamp: P: 1765486349205711 usec, L: 0
1211 20:52:37.724783 (+ 152us) write_op.cc:270] Start()
1211 20:52:37.724803 (+ 20us) write_op.cc:276] Timestamp: P: 1765486349206477 usec, L: 0
1211 20:52:37.724929 (+ 126us) write_op.cc:270] Start()
1211 20:52:37.724946 (+ 17us) write_op.cc:276] Timestamp: P: 1765486349223602 usec, L: 0
1211 20:52:37.725057 (+ 111us) write_op.cc:270] Start()
1211 20:52:37.725071 (+ 14us) write_op.cc:276] Timestamp: P: 1765486349225053 usec, L: 0
1211 20:52:37.725171 (+ 100us) write_op.cc:270] Start()
1211 20:52:37.725186 (+ 15us) write_op.cc:276] Timestamp: P: 1765486349228017 usec, L: 0
1211 20:52:37.725280 (+ 94us) write_op.cc:270] Start()
1211 20:52:37.725304 (+ 24us) write_op.cc:276] Timestamp: P: 1765486349228878 usec, L: 0
1211 20:52:37.725430 (+ 126us) write_op.cc:270] Start()
1211 20:52:37.725448 (+ 18us) write_op.cc:276] Timestamp: P: 1765486349229922 usec, L: 0
1211 20:52:37.725574 (+ 126us) write_op.cc:270] Start()
1211 20:52:37.725591 (+ 17us) write_op.cc:276] Timestamp: P: 1765486349230790 usec, L: 0
1211 20:52:37.725726 (+ 135us) write_op.cc:270] Start()
1211 20:52:37.725756 (+ 30us) write_op.cc:276] Timestamp: P: 1765486349231612 usec, L: 0
1211 20:52:37.725885 (+ 129us) write_op.cc:270] Start()
1211 20:52:37.725931 (+ 46us) write_op.cc:276] Timestamp: P: 1765486349232681 usec, L: 0
1211 20:52:37.726071 (+ 140us) write_op.cc:270] Start()
1211 20:52:37.726088 (+ 17us) write_op.cc:276] Timestamp: P: 1765486349235191 usec, L: 0
1211 20:52:37.726214 (+ 126us) write_op.cc:270] Start()
1211 20:52:37.726236 (+ 22us) write_op.cc:276] Timestamp: P: 1765486349251934 usec, L: 0
1211 20:52:37.726384 (+ 148us) write_op.cc:270] Start()
1211 20:52:37.726401 (+ 17us) write_op.cc:276] Timestamp: P: 1765486349259602 usec, L: 0
1211 20:52:37.726525 (+ 124us) write_op.cc:270] Start()
1211 20:52:37.726540 (+ 15us) write_op.cc:276] Timestamp: P: 1765486349262519 usec, L: 0
1211 20:52:37.726656 (+ 116us) write_op.cc:270] Start()
1211 20:52:37.726673 (+ 17us) write_op.cc:276] Timestamp: P: 1765486349312035 usec, L: 0
1211 20:52:37.726804 (+ 131us) write_op.cc:270] Start()
1211 20:52:37.726823 (+ 19us) write_op.cc:276] Timestamp: P: 1765486349319161 usec, L: 0
1211 20:52:37.726931 (+ 108us) write_op.cc:270] Start()
1211 20:52:37.726947 (+ 16us) write_op.cc:276] Timestamp: P: 1765486349321475 usec, L: 0
1211 20:52:37.727065 (+ 118us) write_op.cc:270] Start()
1211 20:52:37.727082 (+ 17us) write_op.cc:276] Timestamp: P: 1765486349345140 usec, L: 0
1211 20:52:37.727210 (+ 128us) write_op.cc:270] Start()
1211 20:52:37.727225 (+ 15us) write_op.cc:276] Timestamp: P: 1765486349381749 usec, L: 0
1211 20:52:37.727349 (+ 124us) write_op.cc:270] Start()
1211 20:52:37.727365 (+ 16us) write_op.cc:276] Timestamp: P: 1765486349391879 usec, L: 0
1211 20:52:37.727485 (+ 120us) write_op.cc:270] Start()
1211 20:52:37.727504 (+ 19us) write_op.cc:276] Timestamp: P: 1765486349393429 usec, L: 0
1211 20:52:37.754768 (+ 27264us) write_op.cc:270] Start()
1211 20:52:37.754812 (+ 44us) write_op.cc:276] Timestamp: P: 1765486349416180 usec, L: 0
1211 20:52:37.755043 (+ 231us) write_op.cc:270] Start()
1211 20:52:37.755069 (+ 26us) write_op.cc:276] Timestamp: P: 1765486349419824 usec, L: 0
1211 20:52:37.755209 (+ 140us) write_op.cc:270] Start()
1211 20:52:37.755235 (+ 26us) write_op.cc:276] Timestamp: P: 1765486349433452 usec, L: 0
1211 20:52:37.755351 (+ 116us) write_op.cc:270] Start()
1211 20:52:37.755368 (+ 17us) write_op.cc:276] Timestamp: P: 1765486349439407 usec, L: 0
1211 20:52:37.755475 (+ 107us) write_op.cc:270] Start()
1211 20:52:37.755490 (+ 15us) write_op.cc:276] Timestamp: P: 1765486349454582 usec, L: 0
1211 20:52:37.755664 (+ 174us) write_op.cc:270] Start()
1211 20:52:37.755684 (+ 20us) write_op.cc:276] Timestamp: P: 1765486349455215 usec, L: 0
1211 20:52:37.755815 (+ 131us) write_op.cc:270] Start()
1211 20:52:37.755831 (+ 16us) write_op.cc:276] Timestamp: P: 1765486349456266 usec, L: 0
1211 20:52:37.755940 (+ 109us) write_op.cc:270] Start()
1211 20:52:37.755955 (+ 15us) write_op.cc:276] Timestamp: P: 1765486349550731 usec, L: 0
1211 20:52:37.756068 (+ 113us) write_op.cc:270] Start()
1211 20:52:37.756086 (+ 18us) write_op.cc:276] Timestamp: P: 1765486349556674 usec, L: 0
1211 20:52:37.756214 (+ 128us) write_op.cc:270] Start()
1211 20:52:37.756234 (+ 20us) write_op.cc:276] Timestamp: P: 1765486349557327 usec, L: 0
1211 20:52:37.759695 (+ 3461us) write_op.cc:270] Start()
1211 20:52:37.759740 (+ 45us) write_op.cc:276] Timestamp: P: 1765486349557902 usec, L: 0
1211 20:52:37.759940 (+ 200us) write_op.cc:270] Start()
1211 20:52:37.759960 (+ 20us) write_op.cc:276] Timestamp: P: 1765486349573467 usec, L: 0
1211 20:52:37.760076 (+ 116us) write_op.cc:270] Start()
1211 20:52:37.760093 (+ 17us) write_op.cc:276] Timestamp: P: 1765486349575423 usec, L: 0
1211 20:52:37.760201 (+ 108us) write_op.cc:270] Start()
1211 20:52:37.760217 (+ 16us) write_op.cc:276] Timestamp: P: 1765486349576060 usec, L: 0
1211 20:52:37.760328 (+ 111us) write_op.cc:270] Start()
1211 20:52:37.760345 (+ 17us) write_op.cc:276] Timestamp: P: 1765486349577544 usec, L: 0
1211 20:52:37.760473 (+ 128us) write_op.cc:270] Start()
1211 20:52:37.760493 (+ 20us) write_op.cc:276] Timestamp: P: 1765486349578909 usec, L: 0
1211 20:52:37.760610 (+ 117us) write_op.cc:270] Start()
1211 20:52:37.760627 (+ 17us) write_op.cc:276] Timestamp: P: 1765486349579812 usec, L: 0
1211 20:52:37.760731 (+ 104us) write_op.cc:270] Start()
1211 20:52:37.760756 (+ 25us) write_op.cc:276] Timestamp: P: 1765486349582129 usec, L: 0
1211 20:52:37.760869 (+ 113us) write_op.cc:270] Start()
1211 20:52:37.760885 (+ 16us) write_op.cc:276] Timestamp: P: 1765486349608104 usec, L: 0
1211 20:52:37.761003 (+ 118us) write_op.cc:270] Start()
1211 20:52:37.761021 (+ 18us) write_op.cc:276] Timestamp: P: 1765486349610826 usec, L: 0
1211 20:52:37.761149 (+ 128us) write_op.cc:270] Start()
1211 20:52:37.761166 (+ 17us) write_op.cc:276] Timestamp: P: 1765486349623911 usec, L: 0
1211 20:52:37.761326 (+ 160us) write_op.cc:270] Start()
1211 20:52:37.761346 (+ 20us) write_op.cc:276] Timestamp: P: 1765486349753054 usec, L: 0
1211 20:52:37.761476 (+ 130us) write_op.cc:270] Start()
1211 20:52:37.761491 (+ 15us) write_op.cc:276] Timestamp: P: 1765486349784542 usec, L: 0
1211 20:52:37.761641 (+ 150us) write_op.cc:270] Start()
1211 20:52:37.761659 (+ 18us) write_op.cc:276] Timestamp: P: 1765486349792760 usec, L: 0
1211 20:52:37.761782 (+ 123us) write_op.cc:270] Start()
1211 20:52:37.761798 (+ 16us) write_op.cc:276] Timestamp: P: 1765486349793392 usec, L: 0
1211 20:52:37.761904 (+ 106us) write_op.cc:270] Start()
1211 20:52:37.761920 (+ 16us) write_op.cc:276] Timestamp: P: 1765486349793917 usec, L: 0
1211 20:52:37.762022 (+ 102us) write_op.cc:270] Start()
1211 20:52:37.762038 (+ 16us) write_op.cc:276] Timestamp: P: 1765486349794551 usec, L: 0
1211 20:52:37.762150 (+ 112us) write_op.cc:270] Start()
1211 20:52:37.762166 (+ 16us) write_op.cc:276] Timestamp: P: 1765486349795948 usec, L: 0
1211 20:52:37.762292 (+ 126us) write_op.cc:270] Start()
1211 20:52:37.762308 (+ 16us) write_op.cc:276] Timestamp: P: 1765486349797790 usec, L: 0
1211 20:52:37.762421 (+ 113us) write_op.cc:270] Start()
1211 20:52:37.762438 (+ 17us) write_op.cc:276] Timestamp: P: 1765486349799877 usec, L: 0
1211 20:52:37.762558 (+ 120us) write_op.cc:270] Start()
1211 20:52:37.762576 (+ 18us) write_op.cc:276] Timestamp: P: 1765486349807080 usec, L: 0
1211 20:52:37.762711 (+ 135us) write_op.cc:270] Start()
1211 20:52:37.762729 (+ 18us) write_op.cc:276] Timestamp: P: 1765486349980166 usec, L: 0
1211 20:52:37.762857 (+ 128us) write_op.cc:270] Start()
1211 20:52:37.762873 (+ 16us) write_op.cc:276] Timestamp: P: 1765486349980861 usec, L: 0
1211 20:52:37.762995 (+ 122us) write_op.cc:270] Start()
1211 20:52:37.763012 (+ 17us) write_op.cc:276] Timestamp: P: 1765486349983066 usec, L: 0
1211 20:52:37.763118 (+ 106us) write_op.cc:270] Start()
1211 20:52:37.763133 (+ 15us) write_op.cc:276] Timestamp: P: 1765486349983675 usec, L: 0
1211 20:52:37.763251 (+ 118us) write_op.cc:270] Start()
1211 20:52:37.763268 (+ 17us) write_op.cc:276] Timestamp: P: 1765486349986527 usec, L: 0
1211 20:52:37.763401 (+ 133us) write_op.cc:270] Start()
1211 20:52:37.763417 (+ 16us) write_op.cc:276] Timestamp: P: 1765486350004045 usec, L: 0
1211 20:52:37.763529 (+ 112us) write_op.cc:270] Start()
1211 20:52:37.763545 (+ 16us) write_op.cc:276] Timestamp: P: 1765486350043730 usec, L: 0
1211 20:52:37.787704 (+ 24159us) write_op.cc:270] Start()
1211 20:52:37.787750 (+ 46us) write_op.cc:276] Timestamp: P: 1765486350088708 usec, L: 0
1211 20:52:37.787954 (+ 204us) write_op.cc:270] Start()
1211 20:52:37.787974 (+ 20us) write_op.cc:276] Timestamp: P: 1765486350200045 usec, L: 0
1211 20:52:37.788101 (+ 127us) write_op.cc:270] Start()
1211 20:52:37.788141 (+ 40us) write_op.cc:276] Timestamp: P: 1765486350210152 usec, L: 0
1211 20:52:37.788273 (+ 132us) write_op.cc:270] Start()
1211 20:52:37.788293 (+ 20us) write_op.cc:276] Timestamp: P: 1765486350212717 usec, L: 0
1211 20:52:37.788414 (+ 121us) write_op.cc:270] Start()
1211 20:52:37.788432 (+ 18us) write_op.cc:276] Timestamp: P: 1765486350214824 usec, L: 0
1211 20:52:37.788562 (+ 130us) write_op.cc:270] Start()
1211 20:52:37.788581 (+ 19us) write_op.cc:276] Timestamp: P: 1765486350217839 usec, L: 0
1211 20:52:37.788719 (+ 138us) write_op.cc:270] Start()
1211 20:52:37.788749 (+ 30us) write_op.cc:276] Timestamp: P: 1765486350219480 usec, L: 0
1211 20:52:37.790285 (+ 1536us) write_op.cc:270] Start()
1211 20:52:37.790312 (+ 27us) write_op.cc:276] Timestamp: P: 1765486350221237 usec, L: 0
1211 20:52:37.790473 (+ 161us) write_op.cc:270] Start()
1211 20:52:37.790494 (+ 21us) write_op.cc:276] Timestamp: P: 1765486350222817 usec, L: 0
1211 20:52:37.790630 (+ 136us) write_op.cc:270] Start()
1211 20:52:37.790650 (+ 20us) write_op.cc:276] Timestamp: P: 1765486350297287 usec, L: 0
1211 20:52:37.790780 (+ 130us) write_op.cc:270] Start()
1211 20:52:37.790798 (+ 18us) write_op.cc:276] Timestamp: P: 1765486350297907 usec, L: 0
1211 20:52:37.790921 (+ 123us) write_op.cc:270] Start()
1211 20:52:37.790937 (+ 16us) write_op.cc:276] Timestamp: P: 1765486350298447 usec, L: 0
1211 20:52:37.791050 (+ 113us) write_op.cc:270] Start()
1211 20:52:37.791066 (+ 16us) write_op.cc:276] Timestamp: P: 1765486350298930 usec, L: 0
1211 20:52:37.791172 (+ 106us) write_op.cc:270] Start()
1211 20:52:37.791187 (+ 15us) write_op.cc:276] Timestamp: P: 1765486350321476 usec, L: 0
1211 20:52:37.791291 (+ 104us) write_op.cc:270] Start()
1211 20:52:37.791306 (+ 15us) write_op.cc:276] Timestamp: P: 1765486350322578 usec, L: 0
1211 20:52:37.791404 (+ 98us) write_op.cc:270] Start()
1211 20:52:37.791419 (+ 15us) write_op.cc:276] Timestamp: P: 1765486350323981 usec, L: 0
1211 20:52:37.791525 (+ 106us) write_op.cc:270] Start()
1211 20:52:37.791540 (+ 15us) write_op.cc:276] Timestamp: P: 1765486350325987 usec, L: 0
1211 20:52:37.791696 (+ 156us) write_op.cc:270] Start()
1211 20:52:37.791713 (+ 17us) write_op.cc:276] Timestamp: P: 1765486350327464 usec, L: 0
1211 20:52:37.791837 (+ 124us) write_op.cc:270] Start()
1211 20:52:37.791853 (+ 16us) write_op.cc:276] Timestamp: P: 1765486350328914 usec, L: 0
1211 20:52:37.791961 (+ 108us) write_op.cc:270] Start()
1211 20:52:37.791978 (+ 17us) write_op.cc:276] Timestamp: P: 1765486350382157 usec, L: 0
1211 20:52:37.792087 (+ 109us) write_op.cc:270] Start()
1211 20:52:37.792106 (+ 19us) write_op.cc:276] Timestamp: P: 1765486350382806 usec, L: 0
1211 20:52:37.792238 (+ 132us) write_op.cc:270] Start()
1211 20:52:37.792255 (+ 17us) write_op.cc:276] Timestamp: P: 1765486350387004 usec, L: 0
1211 20:52:37.792364 (+ 109us) write_op.cc:270] Start()
1211 20:52:37.792378 (+ 14us) write_op.cc:276] Timestamp: P: 1765486350473318 usec, L: 0
1211 20:52:37.792512 (+ 134us) write_op.cc:270] Start()
1211 20:52:37.792529 (+ 17us) write_op.cc:276] Timestamp: P: 1765486350474043 usec, L: 0
1211 20:52:37.792630 (+ 101us) write_op.cc:270] Start()
1211 20:52:37.792645 (+ 15us) write_op.cc:276] Timestamp: P: 1765486350474635 usec, L: 0
1211 20:52:37.792767 (+ 122us) write_op.cc:270] Start()
1211 20:52:37.792782 (+ 15us) write_op.cc:276] Timestamp: P: 1765486350495755 usec, L: 0
1211 20:52:37.792910 (+ 128us) write_op.cc:270] Start()
1211 20:52:37.792928 (+ 18us) write_op.cc:276] Timestamp: P: 1765486350532012 usec, L: 0
1211 20:52:37.793045 (+ 117us) write_op.cc:270] Start()
1211 20:52:37.793062 (+ 17us) write_op.cc:276] Timestamp: P: 1765486350532631 usec, L: 0
1211 20:52:37.793189 (+ 127us) write_op.cc:270] Start()
1211 20:52:37.793207 (+ 18us) write_op.cc:276] Timestamp: P: 1765486350533213 usec, L: 0
1211 20:52:37.793325 (+ 118us) write_op.cc:270] Start()
1211 20:52:37.793342 (+ 17us) write_op.cc:276] Timestamp: P: 1765486350537769 usec, L: 0
1211 20:52:37.793484 (+ 142us) write_op.cc:270] Start()
1211 20:52:37.793504 (+ 20us) write_op.cc:276] Timestamp: P: 1765486350538329 usec, L: 0
1211 20:52:37.793655 (+ 151us) write_op.cc:270] Start()
1211 20:52:37.793673 (+ 18us) write_op.cc:276] Timestamp: P: 1765486350538848 usec, L: 0
1211 20:52:37.793805 (+ 132us) write_op.cc:270] Start()
1211 20:52:37.793821 (+ 16us) write_op.cc:276] Timestamp: P: 1765486350552046 usec, L: 0
1211 20:52:37.793948 (+ 127us) write_op.cc:270] Start()
1211 20:52:37.793967 (+ 19us) write_op.cc:276] Timestamp: P: 1765486350559868 usec, L: 0
1211 20:52:37.794092 (+ 125us) write_op.cc:270] Start()
1211 20:52:37.794109 (+ 17us) write_op.cc:276] Timestamp: P: 1765486350562536 usec, L: 0
1211 20:52:37.794226 (+ 117us) write_op.cc:270] Start()
1211 20:52:37.794244 (+ 18us) write_op.cc:276] Timestamp: P: 1765486350564152 usec, L: 0
1211 20:52:37.794370 (+ 126us) write_op.cc:270] Start()
1211 20:52:37.794387 (+ 17us) write_op.cc:276] Timestamp: P: 1765486350565451 usec, L: 0
1211 20:52:37.794538 (+ 151us) write_op.cc:270] Start()
1211 20:52:37.794567 (+ 29us) write_op.cc:276] Timestamp: P: 1765486350584925 usec, L: 0
1211 20:52:37.794704 (+ 137us) write_op.cc:270] Start()
1211 20:52:37.794779 (+ 75us) write_op.cc:276] Timestamp: P: 1765486350585549 usec, L: 0
1211 20:52:37.794951 (+ 172us) write_op.cc:270] Start()
1211 20:52:37.794969 (+ 18us) write_op.cc:276] Timestamp: P: 1765486350588744 usec, L: 0
1211 20:52:37.795095 (+ 126us) write_op.cc:270] Start()
1211 20:52:37.795111 (+ 16us) write_op.cc:276] Timestamp: P: 1765486350589689 usec, L: 0
1211 20:52:37.795222 (+ 111us) write_op.cc:270] Start()
1211 20:52:37.795238 (+ 16us) write_op.cc:276] Timestamp: P: 1765486350590723 usec, L: 0
1211 20:52:37.795343 (+ 105us) write_op.cc:270] Start()
1211 20:52:37.795358 (+ 15us) write_op.cc:276] Timestamp: P: 1765486350591610 usec, L: 0
1211 20:52:37.795457 (+ 99us) write_op.cc:270] Start()
1211 20:52:37.795472 (+ 15us) write_op.cc:276] Timestamp: P: 1765486350592865 usec, L: 0
1211 20:52:37.795613 (+ 141us) write_op.cc:270] Start()
1211 20:52:37.795632 (+ 19us) write_op.cc:276] Timestamp: P: 1765486350680592 usec, L: 0
1211 20:52:37.795758 (+ 126us) write_op.cc:270] Start()
1211 20:52:37.795774 (+ 16us) write_op.cc:276] Timestamp: P: 1765486350692274 usec, L: 0
1211 20:52:37.795908 (+ 134us) write_op.cc:270] Start()
1211 20:52:37.795925 (+ 17us) write_op.cc:276] Timestamp: P: 1765486350695324 usec, L: 0
1211 20:52:37.796034 (+ 109us) write_op.cc:270] Start()
1211 20:52:37.796048 (+ 14us) write_op.cc:276] Timestamp: P: 1765486350695925 usec, L: 0
1211 20:52:37.796160 (+ 112us) write_op.cc:270] Start()
1211 20:52:37.796177 (+ 17us) write_op.cc:276] Timestamp: P: 1765486350756131 usec, L: 0
1211 20:52:37.796333 (+ 156us) write_op.cc:270] Start()
1211 20:52:37.796356 (+ 23us) write_op.cc:276] Timestamp: P: 1765486350785460 usec, L: 0
1211 20:52:37.796483 (+ 127us) write_op.cc:270] Start()
1211 20:52:37.796503 (+ 20us) write_op.cc:276] Timestamp: P: 1765486350809128 usec, L: 0
1211 20:52:37.796627 (+ 124us) write_op.cc:270] Start()
1211 20:52:37.796645 (+ 18us) write_op.cc:276] Timestamp: P: 1765486350830973 usec, L: 0
1211 20:52:37.831714 (+ 35069us) write_op.cc:270] Start()
1211 20:52:37.831766 (+ 52us) write_op.cc:276] Timestamp: P: 1765486350929981 usec, L: 0
1211 20:52:37.832617 (+ 851us) write_op.cc:270] Start()
1211 20:52:37.832643 (+ 26us) write_op.cc:276] Timestamp: P: 1765486350930739 usec, L: 0
1211 20:52:37.832792 (+ 149us) write_op.cc:270] Start()
1211 20:52:37.832810 (+ 18us) write_op.cc:276] Timestamp: P: 1765486350939955 usec, L: 0
1211 20:52:37.832924 (+ 114us) write_op.cc:270] Start()
1211 20:52:37.832942 (+ 18us) write_op.cc:276] Timestamp: P: 1765486350943068 usec, L: 0
1211 20:52:37.833042 (+ 100us) write_op.cc:270] Start()
1211 20:52:37.833058 (+ 16us) write_op.cc:276] Timestamp: P: 1765486350946162 usec, L: 0
1211 20:52:37.833182 (+ 124us) write_op.cc:270] Start()
1211 20:52:37.833201 (+ 19us) write_op.cc:276] Timestamp: P: 1765486350946786 usec, L: 0
1211 20:52:37.833332 (+ 131us) write_op.cc:270] Start()
1211 20:52:37.833349 (+ 17us) write_op.cc:276] Timestamp: P: 1765486350947387 usec, L: 0
1211 20:52:37.833450 (+ 101us) write_op.cc:270] Start()
1211 20:52:37.833466 (+ 16us) write_op.cc:276] Timestamp: P: 1765486350948893 usec, L: 0
1211 20:52:37.833573 (+ 107us) write_op.cc:270] Start()
1211 20:52:37.833589 (+ 16us) write_op.cc:276] Timestamp: P: 1765486350949572 usec, L: 0
1211 20:52:37.833708 (+ 119us) write_op.cc:270] Start()
1211 20:52:37.833730 (+ 22us) write_op.cc:276] Timestamp: P: 1765486350950093 usec, L: 0
1211 20:52:37.833857 (+ 127us) write_op.cc:270] Start()
1211 20:52:37.833874 (+ 17us) write_op.cc:276] Timestamp: P: 1765486350953875 usec, L: 0
1211 20:52:37.833991 (+ 117us) write_op.cc:270] Start()
1211 20:52:37.834007 (+ 16us) write_op.cc:276] Timestamp: P: 1765486350963237 usec, L: 0
1211 20:52:37.834110 (+ 103us) write_op.cc:270] Start()
1211 20:52:37.834125 (+ 15us) write_op.cc:276] Timestamp: P: 1765486350992689 usec, L: 0
1211 20:52:37.834232 (+ 107us) write_op.cc:270] Start()
1211 20:52:37.834248 (+ 16us) write_op.cc:276] Timestamp: P: 1765486350993934 usec, L: 0
1211 20:52:37.834357 (+ 109us) write_op.cc:270] Start()
1211 20:52:37.834373 (+ 16us) write_op.cc:276] Timestamp: P: 1765486350994962 usec, L: 0
1211 20:52:37.834474 (+ 101us) write_op.cc:270] Start()
1211 20:52:37.834488 (+ 14us) write_op.cc:276] Timestamp: P: 1765486350996257 usec, L: 0
1211 20:52:37.834600 (+ 112us) write_op.cc:270] Start()
1211 20:52:37.834616 (+ 16us) write_op.cc:276] Timestamp: P: 1765486
I20251211 20:52:40.164063 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 25454e67437541d9ac2bfab29bad55f9 (table default.Table_b0e00b07650849958a1a4486eeb3c660 [id=73346680c9a043cea75e13865bdc7bed]) successfully deleted
I20251211 20:52:40.164817 2286 leader_election.cc:290] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:40.165673 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14e2650f8ec1482ea9511b989fdc963c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:40.165833 1047 raft_consensus.cc:3060] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:40.168248 1047 raft_consensus.cc:2468] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:40.168797 1166 ts_tablet_manager.cc:1434] T a3b1aadf17ea4bf0a896d23cd543fca7 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20251211 20:52:40.169119 1166 tablet_bootstrap.cc:492] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:40.170691 1166 tablet_bootstrap.cc:654] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.170730 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 1bcf36efcd204fea868fd04ee7a9cdc6 (DEFAULT_TABLE table=default.Table_b7a123d140e64879b3fb2606998af217 [id=eee1b058701c499fac48dfab3dc873d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.172987 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bcf36efcd204fea868fd04ee7a9cdc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.173302 2281 tablet.cc:1722] T b072b759d1254859b0b8d401e3078792 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20251211 20:52:40.176653 1163 ts_tablet_manager.cc:1929] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:40.179697 1163 log.cc:1199] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/2197822e0f11484f9679c4858626ac2c
I20251211 20:52:40.180125 1163 ts_tablet_manager.cc:1950] T 2197822e0f11484f9679c4858626ac2c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:40.184403 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 783) took 2197 ms. Trace:
I20251211 20:52:40.165741 1168 raft_consensus.cc:359] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.194296 1168 raft_consensus.cc:385] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.194473 1168 raft_consensus.cc:740] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.194761 1168 consensus_queue.cc:260] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.195638 1168 ts_tablet_manager.cc:1434] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.050s user 0.002s sys 0.000s
I20251211 20:52:40.196058 1168 tablet_bootstrap.cc:492] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:40.166237 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 817) took 2150 ms. Trace:
I20251211 20:52:40.168708 710 leader_election.cc:304] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:40.197440 2273 raft_consensus.cc:2804] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:40.169441 746 tablet_service.cc:1505] Processing CreateTablet for tablet 1bcf36efcd204fea868fd04ee7a9cdc6 (DEFAULT_TABLE table=default.Table_b7a123d140e64879b3fb2606998af217 [id=eee1b058701c499fac48dfab3dc873d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.197588 2273 raft_consensus.cc:697] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:40.172179 741 tablet_service.cc:1505] Processing CreateTablet for tablet 63f368b53f374bbe8e1ce9a4757d1639 (DEFAULT_TABLE table=default.Table_08f40dd56ad24729abbd776dabf07b8f [id=81c5998c4b8d4b7cb68ad8adb957cc0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.172434 2168 tablet.cc:1722] T b072b759d1254859b0b8d401e3078792 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20251211 20:52:40.197870 2273 consensus_queue.cc:237] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.198297 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bcf36efcd204fea868fd04ee7a9cdc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.204365 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63f368b53f374bbe8e1ce9a4757d1639. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.165050 2284 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:52:40.207670 1163 rpcz_store.cc:276] 1211 20:52:37.987058 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.987103 (+ 45us) service_pool.cc:225] Handling call
1211 20:52:40.104902 (+2117799us) ts_tablet_manager.cc:1151] Deleting tablet 2197822e0f11484f9679c4858626ac2c
1211 20:52:40.104913 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.153649 (+ 48736us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.176572 (+ 22923us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.184370 (+ 7798us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2044452}
I20251211 20:52:40.213740 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 2197822e0f11484f9679c4858626ac2c (table default.Table_f6f977edac0f45bea7e94de5bfa51ee4 [id=4a74b9a60cbb4198b55cee630b2f4bef]) successfully deleted
I20251211 20:52:40.218597 1165 rpcz_store.cc:276] 1211 20:52:38.015444 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.019706 (+ 4262us) service_pool.cc:225] Handling call
1211 20:52:40.166029 (+2146323us) ts_tablet_manager.cc:1151] Deleting tablet b61c4f0cb15940a988c5acb6879d265a
1211 20:52:40.166043 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.166172 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2146139}
W20251211 20:52:40.219340 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet b61c4f0cb15940a988c5acb6879d265a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b61c4f0cb15940a988c5acb6879d265a already in progress: creating tablet
I20251211 20:52:40.219936 1165 tablet_replica.cc:333] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:40.220228 1165 raft_consensus.cc:2243] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:40.220695 1165 raft_consensus.cc:2272] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.221508 1165 ts_tablet_manager.cc:1916] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.225821 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 171d5b17dfee49159925b476b08787a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:25 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:40.226693 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 4810f8d8b53a45eb939bf3c8bf1b4e77 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:40.208177 1166 tablet_bootstrap.cc:492] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:40.222577 2216 consensus_queue.cc:1048] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20251211 20:52:40.227962 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 0add14ccd2af42aeb02fad7a28ed4520 with delete_type TABLET_DATA_DELETED (replaced by tablet 84cca3ee8fc446fcaf9971171235cf55) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:40.228071 1166 ts_tablet_manager.cc:1403] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.059s user 0.003s sys 0.000s
I20251211 20:52:40.229244 888 tablet_service.cc:1505] Processing CreateTablet for tablet 63f368b53f374bbe8e1ce9a4757d1639 (DEFAULT_TABLE table=default.Table_08f40dd56ad24729abbd776dabf07b8f [id=81c5998c4b8d4b7cb68ad8adb957cc0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.229712 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63f368b53f374bbe8e1ce9a4757d1639. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.209682 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 784) took 2222 ms. Trace:
I20251211 20:52:40.233651 1163 rpcz_store.cc:276] 1211 20:52:37.987294 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:37.987342 (+ 48us) service_pool.cc:225] Handling call
1211 20:52:40.209554 (+2222212us) ts_tablet_manager.cc:1151] Deleting tablet 2ef524747034491c99f4f9767acd3c9d
1211 20:52:40.209564 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.209663 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2149070}
I20251211 20:52:40.233987 1163 tablet_replica.cc:333] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:40.234159 1163 raft_consensus.cc:2243] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
W20251211 20:52:40.234134 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 2ef524747034491c99f4f9767acd3c9d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2ef524747034491c99f4f9767acd3c9d already in progress: creating tablet
I20251211 20:52:40.234232 2100 tablet.cc:1722] T b072b759d1254859b0b8d401e3078792 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20251211 20:52:40.234310 1163 raft_consensus.cc:2272] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.235534 1163 ts_tablet_manager.cc:1916] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.245545 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 63f368b53f374bbe8e1ce9a4757d1639 (DEFAULT_TABLE table=default.Table_08f40dd56ad24729abbd776dabf07b8f [id=81c5998c4b8d4b7cb68ad8adb957cc0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.228875 1166 raft_consensus.cc:359] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.246536 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63f368b53f374bbe8e1ce9a4757d1639. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.246632 1166 raft_consensus.cc:385] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.247637 1166 raft_consensus.cc:740] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.249074 1166 consensus_queue.cc:260] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.249675 1166 ts_tablet_manager.cc:1434] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.022s user 0.002s sys 0.000s
I20251211 20:52:40.251914 1164 ts_tablet_manager.cc:1929] T a9b2aa21ab124b28aebc33adba0583d2 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:40.255911 1166 tablet_bootstrap.cc:492] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:40.266403 983 consensus_peers.cc:597] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:40.270745 1163 ts_tablet_manager.cc:1929] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:40.271011 1164 log.cc:1199] T a9b2aa21ab124b28aebc33adba0583d2 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/a9b2aa21ab124b28aebc33adba0583d2
I20251211 20:52:40.271411 1164 ts_tablet_manager.cc:1950] T a9b2aa21ab124b28aebc33adba0583d2 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:40.273734 746 tablet_service.cc:1505] Processing CreateTablet for tablet 9838930aafd64da2aafaaf416ae95344 (DEFAULT_TABLE table=default.Table_7684c7e006504ea0a1ef81dc2eecaa8d [id=f943bf305b574c81b836e91e267d5d81]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.275324 1164 tablet_replica.cc:333] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:40.275859 741 tablet_service.cc:1505] Processing CreateTablet for tablet 199f0f51ba0d4dd887e65032b4ab9b7f (DEFAULT_TABLE table=default.Table_109eac21e7ee45e9a1ad808c49b10998 [id=8600a00e19a647188539792bdb9d6a86]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.276511 1164 raft_consensus.cc:2243] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:40.277175 1164 raft_consensus.cc:2272] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.277320 754 tablet_service.cc:1505] Processing CreateTablet for tablet 4c7b74f370b042ee8575f821c07dc085 (DEFAULT_TABLE table=default.Table_731826f26f9940b8afe425d694b642d9 [id=17dd9cea9ac844128ff9f82adb3f919f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.278597 739 tablet_service.cc:1505] Processing CreateTablet for tablet e61d2400b5534a119106ad54d5859cbf (DEFAULT_TABLE table=default.Table_142734af8412498fa19dd25ad1ef2c2e [id=edf08c4121454149890dcef509afd417]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.279917 1164 ts_tablet_manager.cc:1916] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.295701 1163 log.cc:1199] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/2d849c62f7434099a16f62c2a2fc5e85
I20251211 20:52:40.296373 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9838930aafd64da2aafaaf416ae95344. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.274978 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 9838930aafd64da2aafaaf416ae95344 (DEFAULT_TABLE table=default.Table_7684c7e006504ea0a1ef81dc2eecaa8d [id=f943bf305b574c81b836e91e267d5d81]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.276938 1015 tablet_service.cc:1505] Processing CreateTablet for tablet 199f0f51ba0d4dd887e65032b4ab9b7f (DEFAULT_TABLE table=default.Table_109eac21e7ee45e9a1ad808c49b10998 [id=8600a00e19a647188539792bdb9d6a86]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.297699 1163 ts_tablet_manager.cc:1950] T 2d849c62f7434099a16f62c2a2fc5e85 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:40.297823 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9838930aafd64da2aafaaf416ae95344. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.298013 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 199f0f51ba0d4dd887e65032b4ab9b7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.273497 1166 tablet_bootstrap.cc:654] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.302750 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 785) took 2315 ms. Trace:
I20251211 20:52:40.302851 1163 rpcz_store.cc:276] 1211 20:52:38.091489 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.091545 (+ 56us) service_pool.cc:225] Handling call
1211 20:52:40.233937 (+2142392us) ts_tablet_manager.cc:1151] Deleting tablet 2d849c62f7434099a16f62c2a2fc5e85
1211 20:52:40.233947 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.265168 (+ 31221us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.270633 (+ 5465us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.302720 (+ 32087us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2142183}
I20251211 20:52:40.303372 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 2d849c62f7434099a16f62c2a2fc5e85 (table default.Table_c504f8f3796f43b697488bd604268543 [id=2af76b9d32094a4ca656060a4478e08a]) successfully deleted
I20251211 20:52:40.303747 1163 tablet_replica.cc:333] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:40.304092 1163 raft_consensus.cc:2243] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:40.304278 1163 raft_consensus.cc:2272] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.304775 1163 ts_tablet_manager.cc:1916] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.304899 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 199f0f51ba0d4dd887e65032b4ab9b7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.307626 2273 raft_consensus.cc:493] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:40.307850 2273 raft_consensus.cc:515] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.308499 2273 leader_election.cc:290] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:40.309346 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b2c6064c5eb4c0f8979c485f5d85a75" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:40.310475 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b2c6064c5eb4c0f8979c485f5d85a75" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:40.310644 1047 raft_consensus.cc:2468] T 3b2c6064c5eb4c0f8979c485f5d85a75 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:40.311164 710 leader_election.cc:304] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:40.313364 1166 tablet_bootstrap.cc:492] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:40.313514 1166 ts_tablet_manager.cc:1403] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.058s user 0.004s sys 0.001s
I20251211 20:52:40.314350 1166 raft_consensus.cc:359] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.314620 1166 raft_consensus.cc:385] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.314742 1166 raft_consensus.cc:740] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.315032 1166 consensus_queue.cc:260] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.315604 1166 ts_tablet_manager.cc:1434] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:40.315920 1166 tablet_bootstrap.cc:492] T 4ce495233a5e4f3792faf44680df54ba P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:40.317480 1166 tablet_bootstrap.cc:654] T 4ce495233a5e4f3792faf44680df54ba P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.318001 2232 catalog_manager.cc:1269] Loaded cluster ID: 521ed6ceec724e0ab1d4cde545c10f31
I20251211 20:52:40.318115 2232 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: loading cluster ID for follower catalog manager: success
I20251211 20:52:40.318153 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e61d2400b5534a119106ad54d5859cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.320662 1164 ts_tablet_manager.cc:1929] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:40.320881 1164 log.cc:1199] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/0f046ea2637e4a5583b7969e19a605d5
I20251211 20:52:40.321214 1164 ts_tablet_manager.cc:1950] T 0f046ea2637e4a5583b7969e19a605d5 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:40.323984 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c7b74f370b042ee8575f821c07dc085. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:40.327405 582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.61:40749, user_credentials={real_user=slave}} blocked reactor thread for 48164.5us
I20251211 20:52:40.328320 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 773cf53745a64f62817113b33e3182e7 (table default.Table_021d4fb1667a475d87424f23aa0d5e28 [id=6bbfcc2561f349519a69ae1f56016cc7]) successfully deleted
I20251211 20:52:40.329063 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet a9b2aa21ab124b28aebc33adba0583d2 (table default.Table_b70392ad5fb6451482100395256494bd [id=eafa1350ab384c8cba766017840d59a6]) successfully deleted
W20251211 20:52:40.329387 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet fbc2d4aea084492c8270ad1d87096e58 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fbc2d4aea084492c8270ad1d87096e58 already in progress: creating tablet
I20251211 20:52:40.329645 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 0f046ea2637e4a5583b7969e19a605d5 (table default.Table_dffb2ba445c0475e96e343895b21cb51 [id=e96fd7db496847a5b1e0b79546a336d0]) successfully deleted
I20251211 20:52:40.330719 888 tablet_service.cc:1505] Processing CreateTablet for tablet 4c7b74f370b042ee8575f821c07dc085 (DEFAULT_TABLE table=default.Table_731826f26f9940b8afe425d694b642d9 [id=17dd9cea9ac844128ff9f82adb3f919f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.331305 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c7b74f370b042ee8575f821c07dc085. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:40.335214 711 leader_election.cc:343] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:40.335321 2273 raft_consensus.cc:2804] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:40.335409 2273 raft_consensus.cc:493] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:40.335487 2273 raft_consensus.cc:3060] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:40.335726 1163 ts_tablet_manager.cc:1929] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20251211 20:52:40.335839 1163 log.cc:1199] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/a5eeb56709f046d9be92ac600ebc330e
I20251211 20:52:40.336134 1163 ts_tablet_manager.cc:1950] T a5eeb56709f046d9be92ac600ebc330e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:40.336870 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 786) took 2244 ms. Trace:
I20251211 20:52:40.336956 1163 rpcz_store.cc:276] 1211 20:52:38.091931 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.091973 (+ 42us) service_pool.cc:225] Handling call
1211 20:52:40.303696 (+2211723us) ts_tablet_manager.cc:1151] Deleting tablet a5eeb56709f046d9be92ac600ebc330e
1211 20:52:40.303706 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.306609 (+ 2903us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.335569 (+ 28960us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.336843 (+ 1274us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2211585}
I20251211 20:52:40.337281 1163 tablet_replica.cc:333] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:40.337448 1163 raft_consensus.cc:2243] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:40.338116 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 4c7b74f370b042ee8575f821c07dc085 (DEFAULT_TABLE table=default.Table_731826f26f9940b8afe425d694b642d9 [id=17dd9cea9ac844128ff9f82adb3f919f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.338587 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c7b74f370b042ee8575f821c07dc085. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.338685 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet a5eeb56709f046d9be92ac600ebc330e (table default.Table_279bb02bdbd743159babc9a4e6498437 [id=f78e1da479af4d23a552af1c4fc55f81]) successfully deleted
I20251211 20:52:40.337941 2273 raft_consensus.cc:515] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.339529 2273 leader_election.cc:290] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:40.339967 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b2c6064c5eb4c0f8979c485f5d85a75" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:40.340509 711 leader_election.cc:343] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:40.340507 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b2c6064c5eb4c0f8979c485f5d85a75" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:40.340678 2273 raft_consensus.cc:493] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:40.340685 1042 raft_consensus.cc:3060] T 3b2c6064c5eb4c0f8979c485f5d85a75 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:40.340799 2273 raft_consensus.cc:515] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.341679 2273 leader_election.cc:290] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:40.341655 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cce44b80c97948e1af7ac68e77cccfd6" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:40.342083 711 leader_election.cc:343] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:40.342677 710 leader_election.cc:343] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: cce44b80c97948e1af7ac68e77cccfd6
I20251211 20:52:40.342770 710 leader_election.cc:304] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:40.342971 2273 raft_consensus.cc:2749] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:40.343246 1042 raft_consensus.cc:2468] T 3b2c6064c5eb4c0f8979c485f5d85a75 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:40.344162 1164 tablet_replica.cc:333] T 171d5b17dfee49159925b476b08787a4 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:40.344370 1164 raft_consensus.cc:2243] T 171d5b17dfee49159925b476b08787a4 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:40.344521 1164 raft_consensus.cc:2272] T 171d5b17dfee49159925b476b08787a4 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.347677 1168 tablet_bootstrap.cc:654] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.347828 1164 ts_tablet_manager.cc:1916] T 171d5b17dfee49159925b476b08787a4 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.348591 710 leader_election.cc:304] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:40.348887 2273 raft_consensus.cc:2804] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
W20251211 20:52:40.349078 710 consensus_peers.cc:597] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a2058b67ddf1413db338bfa2f64e7a77. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:40.352620 1166 tablet_bootstrap.cc:492] T 4ce495233a5e4f3792faf44680df54ba P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:40.352861 1166 ts_tablet_manager.cc:1403] T 4ce495233a5e4f3792faf44680df54ba P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.037s user 0.003s sys 0.000s
I20251211 20:52:40.353693 1166 raft_consensus.cc:359] T 4ce495233a5e4f3792faf44680df54ba P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.354709 2273 raft_consensus.cc:697] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
W20251211 20:52:40.354295 983 consensus_peers.cc:597] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:40.354986 1166 raft_consensus.cc:385] T 4ce495233a5e4f3792faf44680df54ba P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.355089 1166 raft_consensus.cc:740] T 4ce495233a5e4f3792faf44680df54ba P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.355091 2273 consensus_queue.cc:237] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.355388 1166 consensus_queue.cc:260] T 4ce495233a5e4f3792faf44680df54ba P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.355540 1990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9230f94a6a1a49cead51aeea9f069087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } } }
I20251211 20:52:40.355779 1990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087 [sys.catalog]: This master's current role is: LEADER
I20251211 20:52:40.360817 1166 ts_tablet_manager.cc:1434] T 4ce495233a5e4f3792faf44680df54ba P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20251211 20:52:40.364560 1166 tablet_bootstrap.cc:492] T 645b08b33676420c96e880de3d460829 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:40.365924 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_c1fc29564f5e4ee5b8f8cf8b19c744f0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:40.366544 595 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c1fc29564f5e4ee5b8f8cf8b19c744f0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:40.367889 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 18f125dd9efc401181f0644dad7aeffe consensus state has no leader
I20251211 20:52:40.368275 590 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a05c8b486c1940c29db8cbb1ea1dfb17" } modify_external_catalogs: true
I20251211 20:52:40.368490 590 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a05c8b486c1940c29db8cbb1ea1dfb17" } modify_external_catalogs: true
I20251211 20:52:40.368533 1166 tablet_bootstrap.cc:654] T 645b08b33676420c96e880de3d460829 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.370361 1163 raft_consensus.cc:2272] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.370934 1163 ts_tablet_manager.cc:1916] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.373906 2232 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: acquiring CA information for follower catalog manager: success
I20251211 20:52:40.374195 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 877aa237fe43452cbdbb4dac46f613ae
I20251211 20:52:40.377542 588 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet a3b1aadf17ea4bf0a896d23cd543fca7 (table default.Table_d18401707c88493abdf90d2c713432ae [id=a07662f8dc3f4178be5dfcc1cdc85837]). Expected version 1 got 0
I20251211 20:52:40.378438 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 877aa237fe43452cbdbb4dac46f613ae with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:40.384047 2216 raft_consensus.cc:493] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:40.384207 2216 raft_consensus.cc:515] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.384858 2216 leader_election.cc:290] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:40.385296 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4df68d1b6a91458e9f212ae81aa25438" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:40.385432 1164 ts_tablet_manager.cc:1929] T 171d5b17dfee49159925b476b08787a4 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:40.385517 1164 log.cc:1199] T 171d5b17dfee49159925b476b08787a4 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/171d5b17dfee49159925b476b08787a4
I20251211 20:52:40.385540 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4df68d1b6a91458e9f212ae81aa25438" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:40.385810 1164 ts_tablet_manager.cc:1950] T 171d5b17dfee49159925b476b08787a4 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
W20251211 20:52:40.385972 983 leader_election.cc:343] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:40.386184 982 leader_election.cc:343] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:40.386296 982 leader_election.cc:304] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:40.386667 2216 raft_consensus.cc:2749] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:40.381507 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f9c6b985657d4e44a74358541fb67452" } modify_external_catalogs: true
I20251211 20:52:40.387266 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f9c6b985657d4e44a74358541fb67452" } modify_external_catalogs: true
I20251211 20:52:40.387557 1015 tablet_service.cc:1552] Processing DeleteTablet for tablet 2207af105bc3471ca930d35a91d38e70 with delete_type TABLET_DATA_DELETED (replaced by tablet 8675e2a18f0f4a0e95a87005cf82d8b4) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:40.389753 592 catalog_manager.cc:5654] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:40.390832 592 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet e4647105ab57454d883ebfcde7bde891 (table default.Table_c817dec9a7674f699b3366fc302ead57 [id=d555d1fc6aef4a21b4d3f2804264d030]). Expected version 1 got 0
I20251211 20:52:40.392107 592 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet bfb52935296443c58190d4d06e2e6e9c (table default.Table_3721b5725e8c4297a392d3986a081128 [id=383392d835da4c50848068d83261cecc]). Expected version 7 got 0
I20251211 20:52:40.395519 1163 ts_tablet_manager.cc:1929] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:40.395789 1163 log.cc:1199] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/b529b38d754f46ba8cb60b19e9705256
I20251211 20:52:40.396176 1163 ts_tablet_manager.cc:1950] T b529b38d754f46ba8cb60b19e9705256 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:40.397050 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 787) took 2304 ms. Trace:
I20251211 20:52:40.397223 1163 rpcz_store.cc:276] 1211 20:52:38.092229 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.092267 (+ 38us) service_pool.cc:225] Handling call
1211 20:52:40.337229 (+2244962us) ts_tablet_manager.cc:1151] Deleting tablet b529b38d754f46ba8cb60b19e9705256
1211 20:52:40.337240 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.390026 (+ 52786us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.395379 (+ 5353us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.397020 (+ 1641us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2244836}
I20251211 20:52:40.397902 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet b529b38d754f46ba8cb60b19e9705256 (table default.Table_742d698db63f432aa7b88ceccf6dc0fc [id=6e5fe0a78e4b48f98dac028802f1ddf4]) successfully deleted
I20251211 20:52:40.398321 1163 tablet_replica.cc:333] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:40.398552 1163 raft_consensus.cc:2243] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:40.398777 1163 raft_consensus.cc:2272] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.399363 1163 ts_tablet_manager.cc:1916] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.406383 1163 ts_tablet_manager.cc:1929] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:40.406606 1163 log.cc:1199] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/e6ba12f94f5f4d688f4222a4c2fb5436
I20251211 20:52:40.418311 1163 ts_tablet_manager.cc:1950] T e6ba12f94f5f4d688f4222a4c2fb5436 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:40.419243 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 788) took 2326 ms. Trace:
I20251211 20:52:40.419445 1163 rpcz_store.cc:276] 1211 20:52:38.092507 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.092543 (+ 36us) service_pool.cc:225] Handling call
1211 20:52:40.398262 (+2305719us) ts_tablet_manager.cc:1151] Deleting tablet e6ba12f94f5f4d688f4222a4c2fb5436
1211 20:52:40.398275 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.402835 (+ 4560us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.406275 (+ 3440us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.419217 (+ 12942us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2305601}
I20251211 20:52:40.420094 1163 tablet_replica.cc:333] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:40.420035 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet e6ba12f94f5f4d688f4222a4c2fb5436 (table default.Table_70a00e0b1675435d9553c29cf9b0d3ea [id=3d49177647c544c2af198e8811dbbb42]) successfully deleted
I20251211 20:52:40.420358 1163 raft_consensus.cc:2243] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:40.420573 1163 raft_consensus.cc:2272] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.421178 1163 ts_tablet_manager.cc:1916] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.427239 1163 ts_tablet_manager.cc:1929] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:40.427459 1163 log.cc:1199] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/14e2650f8ec1482ea9511b989fdc963c
I20251211 20:52:40.427891 1163 ts_tablet_manager.cc:1950] T 14e2650f8ec1482ea9511b989fdc963c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:40.428701 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 789) took 2335 ms. Trace:
I20251211 20:52:40.429162 1163 rpcz_store.cc:276] 1211 20:52:38.092752 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.092797 (+ 45us) service_pool.cc:225] Handling call
1211 20:52:40.420037 (+2327240us) ts_tablet_manager.cc:1151] Deleting tablet 14e2650f8ec1482ea9511b989fdc963c
1211 20:52:40.420048 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.424282 (+ 4234us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.427134 (+ 2852us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.428678 (+ 1544us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2327123}
I20251211 20:52:40.429723 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 14e2650f8ec1482ea9511b989fdc963c (table default.Table_60175b4e1c9744e0b6aefa1dece53aa0 [id=2594e7a272e14856879b5d458c1a392b]) successfully deleted
I20251211 20:52:40.429821 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 794) took 2278 ms. Trace:
I20251211 20:52:40.429895 1163 rpcz_store.cc:276] 1211 20:52:38.151183 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.151224 (+ 41us) service_pool.cc:225] Handling call
1211 20:52:40.429687 (+2278463us) ts_tablet_manager.cc:1151] Deleting tablet b61c4f0cb15940a988c5acb6879d265a
1211 20:52:40.429699 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.429806 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2181870}
I20251211 20:52:40.430373 1163 tablet_replica.cc:333] T 4f807596fa6a4e66b89566b5d5dd1080 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
W20251211 20:52:40.430378 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet b61c4f0cb15940a988c5acb6879d265a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b61c4f0cb15940a988c5acb6879d265a already in progress: creating tablet
I20251211 20:52:40.430615 1163 raft_consensus.cc:2243] T 4f807596fa6a4e66b89566b5d5dd1080 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:40.430868 1163 raft_consensus.cc:2272] T 4f807596fa6a4e66b89566b5d5dd1080 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.431859 1163 ts_tablet_manager.cc:1916] T 4f807596fa6a4e66b89566b5d5dd1080 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.435520 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 877aa237fe43452cbdbb4dac46f613ae with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:40.437072 582 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet bb9fccb6981e446a939a28d7fdd38e25 (table default.Table_760e3e4714ea4128bd1de3da5a7ccccc [id=fe9b692163f943559ba64021faa648f1], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet bb9fccb6981e446a939a28d7fdd38e25 consensus state has no leader
I20251211 20:52:40.437408 877 tablet_service.cc:1505] Processing CreateTablet for tablet 9838930aafd64da2aafaaf416ae95344 (DEFAULT_TABLE table=default.Table_7684c7e006504ea0a1ef81dc2eecaa8d [id=f943bf305b574c81b836e91e267d5d81]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.437510 891 tablet_service.cc:1505] Processing CreateTablet for tablet e61d2400b5534a119106ad54d5859cbf (DEFAULT_TABLE table=default.Table_142734af8412498fa19dd25ad1ef2c2e [id=edf08c4121454149890dcef509afd417]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.437816 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9838930aafd64da2aafaaf416ae95344. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.438550 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 171d5b17dfee49159925b476b08787a4 (table default.Table_75c7933bd82449a1a24a7481815ee23b [id=4f5722552e534a00874e4113c0dfbaf3]) successfully deleted
W20251211 20:52:40.438771 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 4810f8d8b53a45eb939bf3c8bf1b4e77 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4810f8d8b53a45eb939bf3c8bf1b4e77 already in progress: creating tablet
W20251211 20:52:40.438968 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 0add14ccd2af42aeb02fad7a28ed4520 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0add14ccd2af42aeb02fad7a28ed4520 already in progress: creating tablet
I20251211 20:52:40.440470 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e61d2400b5534a119106ad54d5859cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.443543 885 tablet_service.cc:1505] Processing CreateTablet for tablet 199f0f51ba0d4dd887e65032b4ab9b7f (DEFAULT_TABLE table=default.Table_109eac21e7ee45e9a1ad808c49b10998 [id=8600a00e19a647188539792bdb9d6a86]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.443986 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 199f0f51ba0d4dd887e65032b4ab9b7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.448444 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 877aa237fe43452cbdbb4dac46f613ae with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:40.451231 1168 tablet_bootstrap.cc:492] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:40.451687 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 33addf3d158941098f240cfcdec45d28 with delete_type TABLET_DATA_DELETED (replaced by tablet 7be44df39a8f4d6da793479d425fa684) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:40.452232 1163 ts_tablet_manager.cc:1929] T 4f807596fa6a4e66b89566b5d5dd1080 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:40.452670 1168 ts_tablet_manager.cc:1403] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.257s user 0.000s sys 0.004s
I20251211 20:52:40.452790 1163 log.cc:1199] T 4f807596fa6a4e66b89566b5d5dd1080 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/4f807596fa6a4e66b89566b5d5dd1080
I20251211 20:52:40.453213 1163 ts_tablet_manager.cc:1950] T 4f807596fa6a4e66b89566b5d5dd1080 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:40.454013 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 797) took 2301 ms. Trace:
I20251211 20:52:40.454147 1163 rpcz_store.cc:276] 1211 20:52:38.152463 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.152506 (+ 43us) service_pool.cc:225] Handling call
1211 20:52:40.430324 (+2277818us) ts_tablet_manager.cc:1151] Deleting tablet 4f807596fa6a4e66b89566b5d5dd1080
1211 20:52:40.430333 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.434823 (+ 4490us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.452117 (+ 17294us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.453992 (+ 1875us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2178480}
I20251211 20:52:40.454658 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 4f807596fa6a4e66b89566b5d5dd1080 (table default.Table_ae0290b97d234621bbd1ff141b5c2400 [id=5d17fc32609b43c6b4fe461d3277dd2e]) successfully deleted
I20251211 20:52:40.454767 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 812) took 1974 ms. Trace:
I20251211 20:52:40.454830 1163 rpcz_store.cc:276] 1211 20:52:38.480567 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.480602 (+ 35us) service_pool.cc:225] Handling call
1211 20:52:40.454638 (+1974036us) ts_tablet_manager.cc:1151] Deleting tablet a471a9accbfc4875bd1d7802f5620808
1211 20:52:40.454649 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.454751 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1946263}
W20251211 20:52:40.455322 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet a471a9accbfc4875bd1d7802f5620808 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a471a9accbfc4875bd1d7802f5620808 already in progress: creating tablet
I20251211 20:52:40.455843 1160 tablet_bootstrap.cc:492] T 648f43e95e034f2188a20e1b837f2818 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:40.456033 1160 ts_tablet_manager.cc:1403] T 648f43e95e034f2188a20e1b837f2818 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.321s user 0.037s sys 0.028s
I20251211 20:52:40.456947 1020 tablet_service.cc:1505] Processing CreateTablet for tablet ced8a953d7744b3c806f0184a1921327 (DEFAULT_TABLE table=default.Table_25ddd02b2a6640c4a6898d1796553a8b [id=14985ae63d8740f4b692dea1578be3ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.456951 1160 raft_consensus.cc:359] T 648f43e95e034f2188a20e1b837f2818 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.457114 1160 raft_consensus.cc:385] T 648f43e95e034f2188a20e1b837f2818 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.457168 1160 raft_consensus.cc:740] T 648f43e95e034f2188a20e1b837f2818 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.457326 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ced8a953d7744b3c806f0184a1921327. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.457382 1160 consensus_queue.cc:260] T 648f43e95e034f2188a20e1b837f2818 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.462793 739 tablet_service.cc:1505] Processing CreateTablet for tablet a87c69ae71d24de09294c7c8c34d79da (DEFAULT_TABLE table=default.Table_ee696fa128ac464da5a6f99bd3d7ed7a [id=8cb270b7f6e04c8c9307869d9dd6df4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.462903 757 tablet_service.cc:1505] Processing CreateTablet for tablet 9898716a5fe04953b92a66dcbd10b2dc (DEFAULT_TABLE table=default.Table_cbecf49ce5e044049547838a5d6ce009 [id=dd75e76866b84667ae39862a9330a45d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.462986 1160 ts_tablet_manager.cc:1434] T 648f43e95e034f2188a20e1b837f2818 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20251211 20:52:40.462903 756 tablet_service.cc:1505] Processing CreateTablet for tablet f00128ba84c245d9a0e35f26bc842b3e (DEFAULT_TABLE table=default.Table_6f8c133c7a364b93af2c83d8911a0ac3 [id=6a21c38ee7224a02b937df088494f5d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.463315 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a87c69ae71d24de09294c7c8c34d79da. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.463655 1021 tablet_service.cc:1505] Processing CreateTablet for tablet f00128ba84c245d9a0e35f26bc842b3e (DEFAULT_TABLE table=default.Table_6f8c133c7a364b93af2c83d8911a0ac3 [id=6a21c38ee7224a02b937df088494f5d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.463871 1029 tablet_service.cc:1505] Processing CreateTablet for tablet a87c69ae71d24de09294c7c8c34d79da (DEFAULT_TABLE table=default.Table_ee696fa128ac464da5a6f99bd3d7ed7a [id=8cb270b7f6e04c8c9307869d9dd6df4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.464020 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f00128ba84c245d9a0e35f26bc842b3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.464296 1013 tablet_service.cc:1505] Processing CreateTablet for tablet 9898716a5fe04953b92a66dcbd10b2dc (DEFAULT_TABLE table=default.Table_cbecf49ce5e044049547838a5d6ce009 [id=dd75e76866b84667ae39862a9330a45d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.464562 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9898716a5fe04953b92a66dcbd10b2dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.466436 1160 tablet_bootstrap.cc:492] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:40.463655 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 34083c49cc514fbabfb8457980629185 (DEFAULT_TABLE table=default.Table_c15e1d9cc43c4d0b951d51344fb6ad09 [id=09a33d8a7c7c46c2b8b737fc9a8543b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.467837 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34083c49cc514fbabfb8457980629185. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.468025 1160 tablet_bootstrap.cc:654] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.468039 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f00128ba84c245d9a0e35f26bc842b3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.470750 1164 tablet_replica.cc:333] T 33addf3d158941098f240cfcdec45d28 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:40.470956 1164 raft_consensus.cc:2243] T 33addf3d158941098f240cfcdec45d28 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:40.471119 1164 raft_consensus.cc:2272] T 33addf3d158941098f240cfcdec45d28 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.471714 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a87c69ae71d24de09294c7c8c34d79da. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.475841 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9898716a5fe04953b92a66dcbd10b2dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.478868 1165 ts_tablet_manager.cc:1929] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:40.478997 1165 log.cc:1199] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/ccc3ddd4b7f74f11a3835997cb7ae8d0
I20251211 20:52:40.479312 1165 ts_tablet_manager.cc:1950] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:40.479766 1015 tablet_service.cc:1505] Processing CreateTablet for tablet e61d2400b5534a119106ad54d5859cbf (DEFAULT_TABLE table=default.Table_142734af8412498fa19dd25ad1ef2c2e [id=edf08c4121454149890dcef509afd417]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.480398 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 7ca98b43863a47c091baae0a6ce65630 (DEFAULT_TABLE table=default.Table_b158d0ce6ac74ce9a3a5f207a42ec4f3 [id=56c070ad80624c698ebe2e01b4929b6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.480865 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e61d2400b5534a119106ad54d5859cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.481178 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ca98b43863a47c091baae0a6ce65630. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.488032 877 tablet_service.cc:1505] Processing CreateTablet for tablet 18f125dd9efc401181f0644dad7aeffe (DEFAULT_TABLE table=default.Table_fa6592c2e2624d81ad21abb13ed36045 [id=9b48fc2b39db4fd5a4e5da19e34427dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.488145 891 tablet_service.cc:1505] Processing CreateTablet for tablet ced8a953d7744b3c806f0184a1921327 (DEFAULT_TABLE table=default.Table_25ddd02b2a6640c4a6898d1796553a8b [id=14985ae63d8740f4b692dea1578be3ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.488487 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ced8a953d7744b3c806f0184a1921327. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.488919 589 master_service.cc:954] ReplaceTablet: received request to replace tablet 4df68d1b6a91458e9f212ae81aa25438 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:40.489233 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18f125dd9efc401181f0644dad7aeffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.495918 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 18f125dd9efc401181f0644dad7aeffe (DEFAULT_TABLE table=default.Table_fa6592c2e2624d81ad21abb13ed36045 [id=9b48fc2b39db4fd5a4e5da19e34427dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.496397 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18f125dd9efc401181f0644dad7aeffe. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:40.498099 711 consensus_peers.cc:597] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:40.498334 1163 tablet_replica.cc:333] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:40.498585 1163 raft_consensus.cc:2243] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:40.498859 1163 raft_consensus.cc:2272] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.499142 885 tablet_service.cc:1505] Processing CreateTablet for tablet 9898716a5fe04953b92a66dcbd10b2dc (DEFAULT_TABLE table=default.Table_cbecf49ce5e044049547838a5d6ce009 [id=dd75e76866b84667ae39862a9330a45d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.499186 889 tablet_service.cc:1505] Processing CreateTablet for tablet 7ca98b43863a47c091baae0a6ce65630 (DEFAULT_TABLE table=default.Table_b158d0ce6ac74ce9a3a5f207a42ec4f3 [id=56c070ad80624c698ebe2e01b4929b6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.499224 746 tablet_service.cc:1505] Processing CreateTablet for tablet ced8a953d7744b3c806f0184a1921327 (DEFAULT_TABLE table=default.Table_25ddd02b2a6640c4a6898d1796553a8b [id=14985ae63d8740f4b692dea1578be3ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.499598 888 tablet_service.cc:1505] Processing CreateTablet for tablet a87c69ae71d24de09294c7c8c34d79da (DEFAULT_TABLE table=default.Table_ee696fa128ac464da5a6f99bd3d7ed7a [id=8cb270b7f6e04c8c9307869d9dd6df4e]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:40.499699 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 877aa237fe43452cbdbb4dac46f613ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 877aa237fe43452cbdbb4dac46f613ae already in progress: creating tablet
I20251211 20:52:40.499676 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9898716a5fe04953b92a66dcbd10b2dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.499939 1163 ts_tablet_manager.cc:1916] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.500710 1164 ts_tablet_manager.cc:1916] T 33addf3d158941098f240cfcdec45d28 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.502211 877 tablet_service.cc:1505] Processing CreateTablet for tablet 34083c49cc514fbabfb8457980629185 (DEFAULT_TABLE table=default.Table_c15e1d9cc43c4d0b951d51344fb6ad09 [id=09a33d8a7c7c46c2b8b737fc9a8543b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.502525 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34083c49cc514fbabfb8457980629185. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.503736 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ca98b43863a47c091baae0a6ce65630. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.503857 891 tablet_service.cc:1505] Processing CreateTablet for tablet f00128ba84c245d9a0e35f26bc842b3e (DEFAULT_TABLE table=default.Table_6f8c133c7a364b93af2c83d8911a0ac3 [id=6a21c38ee7224a02b937df088494f5d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.504164 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f00128ba84c245d9a0e35f26bc842b3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.505379 1163 ts_tablet_manager.cc:1929] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:40.505471 1163 log.cc:1199] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/ccc3ddd4b7f74f11a3835997cb7ae8d0
I20251211 20:52:40.505779 1163 ts_tablet_manager.cc:1950] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:40.506470 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 810) took 2026 ms. Trace:
I20251211 20:52:40.506546 1163 rpcz_store.cc:276] 1211 20:52:38.480237 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.558612 (+ 78375us) service_pool.cc:225] Handling call
1211 20:52:40.498286 (+1939674us) ts_tablet_manager.cc:1151] Deleting tablet ccc3ddd4b7f74f11a3835997cb7ae8d0
1211 20:52:40.498296 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.502677 (+ 4381us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.505288 (+ 2611us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.506444 (+ 1156us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1938974}
I20251211 20:52:40.506811 2232 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251211 20:52:40.507073 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a87c69ae71d24de09294c7c8c34d79da. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.507066 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet ccc3ddd4b7f74f11a3835997cb7ae8d0 (table default.Table_63e0e08eb63c40b9aece6be45b6d4944 [id=8e1b3bb8f14e49d19eecdb540ff2d85d]) successfully deleted
I20251211 20:52:40.508713 739 tablet_service.cc:1505] Processing CreateTablet for tablet 7ca98b43863a47c091baae0a6ce65630 (DEFAULT_TABLE table=default.Table_b158d0ce6ac74ce9a3a5f207a42ec4f3 [id=56c070ad80624c698ebe2e01b4929b6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.513067 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 818) took 1947 ms. Trace:
I20251211 20:52:40.513130 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f520281d794144b5b124b2a218ef3fea" } modify_external_catalogs: true
I20251211 20:52:40.513238 1163 rpcz_store.cc:276] 1211 20:52:38.566059 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.575678 (+ 9619us) service_pool.cc:225] Handling call
1211 20:52:40.512928 (+1937250us) ts_tablet_manager.cc:1151] Deleting tablet 745ce63314834c8abdecaa367133a068
1211 20:52:40.512940 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.513044 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1937038}
I20251211 20:52:40.513362 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f520281d794144b5b124b2a218ef3fea" } modify_external_catalogs: true
W20251211 20:52:40.513918 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 745ce63314834c8abdecaa367133a068 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 745ce63314834c8abdecaa367133a068 already in progress: creating tablet
I20251211 20:52:40.514298 1163 tablet_replica.cc:333] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:40.514513 1163 raft_consensus.cc:2243] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:40.514707 1163 raft_consensus.cc:2272] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.514920 593 master_service.cc:954] ReplaceTablet: received request to replace tablet a56f1a7bd5464d289bfd018c826ee574 from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:40.515419 1163 ts_tablet_manager.cc:1916] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:40.516278 594 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 94658fcf512a48d0a992a7e2e0f7c176 consensus state has no leader
I20251211 20:52:40.516522 594 master_service.cc:954] ReplaceTablet: received request to replace tablet 29463a8f71f943a0927942235af3c2da from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:40.517685 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_ca483456010c49c6a291fbaeb6f864ef" } new_table_name: "default.Table_b44b03dcd95a41c19061d5e52f5687c9" modify_external_catalogs: true
I20251211 20:52:40.518343 590 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 757ec913dff840f992a41a38b05f8d41
I20251211 20:52:40.518417 2216 raft_consensus.cc:493] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:40.518541 2216 raft_consensus.cc:515] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.519091 2216 leader_election.cc:290] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:40.519287 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_0910ec0ec3e64c139587947350a39ae5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:40.519810 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68792ccef54944a09af9b5456e9ff1fe" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:40.519912 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0910ec0ec3e64c139587947350a39ae5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:40.520243 751 tablet_service.cc:1552] Processing DeleteTablet for tablet 757ec913dff840f992a41a38b05f8d41 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:40.520560 983 leader_election.cc:343] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:40.520689 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 757ec913dff840f992a41a38b05f8d41 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:40.521080 747 tablet_service.cc:1505] Processing CreateTablet for tablet 34083c49cc514fbabfb8457980629185 (DEFAULT_TABLE table=default.Table_c15e1d9cc43c4d0b951d51344fb6ad09 [id=09a33d8a7c7c46c2b8b737fc9a8543b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.454494 1168 raft_consensus.cc:359] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.521502 1168 raft_consensus.cc:385] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.521582 1168 raft_consensus.cc:740] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.521539 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_742d698db63f432aa7b88ceccf6dc0fc" } new_table_name: "default.Table_92281c5fcfee4355a79215ffedeee588" modify_external_catalogs: true
I20251211 20:52:40.521833 1168 consensus_queue.cc:260] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.522266 1163 ts_tablet_manager.cc:1929] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:40.522461 1168 ts_tablet_manager.cc:1434] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.069s user 0.000s sys 0.001s
I20251211 20:52:40.522799 741 tablet_service.cc:1505] Processing CreateTablet for tablet 18f125dd9efc401181f0644dad7aeffe (DEFAULT_TABLE table=default.Table_fa6592c2e2624d81ad21abb13ed36045 [id=9b48fc2b39db4fd5a4e5da19e34427dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.527707 1163 log.cc:1199] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/793f543d6b614ad7b3ab8e717db2ceaf
I20251211 20:52:40.528110 1163 ts_tablet_manager.cc:1950] T 793f543d6b614ad7b3ab8e717db2ceaf P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:40.528928 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 827) took 1913 ms. Trace:
I20251211 20:52:40.529080 1163 rpcz_store.cc:276] 1211 20:52:38.615792 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.615846 (+ 54us) service_pool.cc:225] Handling call
1211 20:52:40.514245 (+1898399us) ts_tablet_manager.cc:1151] Deleting tablet 793f543d6b614ad7b3ab8e717db2ceaf
1211 20:52:40.514257 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.519477 (+ 5220us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.522176 (+ 2699us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.528906 (+ 6730us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1898187}
I20251211 20:52:40.529891 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 834) took 1869 ms. Trace:
I20251211 20:52:40.530025 1163 rpcz_store.cc:276] 1211 20:52:38.660869 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.678703 (+ 17834us) service_pool.cc:225] Handling call
1211 20:52:40.529757 (+1851054us) ts_tablet_manager.cc:1151] Deleting tablet 5bcc94ec9630484cb7a3e8bf6d4f378a
1211 20:52:40.529768 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.529870 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1850829}
I20251211 20:52:40.530462 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 5ddf7c1972564ef88e3c0a0bd1b430e0 from {username='slave'} at 127.0.0.1:44772
W20251211 20:52:40.530531 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 5bcc94ec9630484cb7a3e8bf6d4f378a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5bcc94ec9630484cb7a3e8bf6d4f378a already in progress: creating tablet
I20251211 20:52:40.530897 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 835) took 1851 ms. Trace:
I20251211 20:52:40.530967 1163 rpcz_store.cc:276] 1211 20:52:38.679116 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.679162 (+ 46us) service_pool.cc:225] Handling call
1211 20:52:40.530785 (+1851623us) ts_tablet_manager.cc:1151] Deleting tablet 75dd170c5828409fa69bd68fdf28a72b
1211 20:52:40.530795 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.530878 (+ 83us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1851459}
I20251211 20:52:40.531052 595 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e523eadf39e648adba47da28461e85e3" } new_table_name: "default.Table_ab826a3eec8e4c4aaa8d4021d87dd382" modify_external_catalogs: true
W20251211 20:52:40.531333 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 75dd170c5828409fa69bd68fdf28a72b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75dd170c5828409fa69bd68fdf28a72b already in progress: creating tablet
I20251211 20:52:40.531705 1163 tablet_replica.cc:333] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:40.531638 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68792ccef54944a09af9b5456e9ff1fe" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:40.531523 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 757ec913dff840f992a41a38b05f8d41 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:40.531873 1163 raft_consensus.cc:2243] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:40.532120 1163 pending_rounds.cc:70] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7: Trying to abort 1 pending ops.
I20251211 20:52:40.532163 1163 pending_rounds.cc:77] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432127932645376 op_type: NO_OP noop_request { }
I20251211 20:52:40.532248 1163 raft_consensus.cc:2889] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:40.532330 1163 raft_consensus.cc:2272] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:40.532714 982 leader_election.cc:343] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:40.532814 982 leader_election.cc:304] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:40.532856 1163 ts_tablet_manager.cc:1916] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.533149 2216 raft_consensus.cc:2749] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:40.530054 1166 tablet_bootstrap.cc:492] T 645b08b33676420c96e880de3d460829 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:40.535176 1166 ts_tablet_manager.cc:1403] T 645b08b33676420c96e880de3d460829 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.171s user 0.001s sys 0.006s
I20251211 20:52:40.537556 1166 raft_consensus.cc:359] T 645b08b33676420c96e880de3d460829 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.538754 1166 raft_consensus.cc:385] T 645b08b33676420c96e880de3d460829 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.538856 1166 raft_consensus.cc:740] T 645b08b33676420c96e880de3d460829 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.539110 1166 consensus_queue.cc:260] T 645b08b33676420c96e880de3d460829 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.543493 588 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: default.Table_3de7aa51e2814433b4d98f3226b96c97 [id=a137f0ddf52145a99c62444602ab3fdb] alter complete (version 5)
I20251211 20:52:40.544786 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 68792ccef54944a09af9b5456e9ff1fe with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:40.544833 1019 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ce495233a5e4f3792faf44680df54ba with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:40.544786 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 260bdeb73439437cbf234d20f7264ab3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:40.547035 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ced8a953d7744b3c806f0184a1921327. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.547111 1166 ts_tablet_manager.cc:1434] T 645b08b33676420c96e880de3d460829 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20251211 20:52:40.551723 1166 tablet_bootstrap.cc:492] T fed300d31c414f4a9b27c66e78097688 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:40.551757 591 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 79d92736bb1c46d2bfa5678b2d396ec2
I20251211 20:52:40.552660 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 79d92736bb1c46d2bfa5678b2d396ec2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:40.552911 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 79d92736bb1c46d2bfa5678b2d396ec2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:40.553068 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_be91bd80b0374d5eae01b605de36b203"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:40.553648 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_be91bd80b0374d5eae01b605de36b203 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:40.555048 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4d70f35e05524680b3d72b2e488adbd5" } modify_external_catalogs: true
I20251211 20:52:40.555214 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_4d70f35e05524680b3d72b2e488adbd5" } modify_external_catalogs: true
I20251211 20:52:40.555970 1168 tablet_bootstrap.cc:492] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:40.546729 1163 ts_tablet_manager.cc:1929] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:40.557363 1168 tablet_bootstrap.cc:654] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.557606 1163 log.cc:1199] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/e4b5a7553110424d901788c7a2ecdce6
I20251211 20:52:40.557639 754 tablet_service.cc:1505] Processing CreateTablet for tablet 3dfea28c44d84c68850db65a30d52ee6 (DEFAULT_TABLE table=default.Table_c504f8f3796f43b697488bd604268543 [id=2af76b9d32094a4ca656060a4478e08a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.558214 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dfea28c44d84c68850db65a30d52ee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.558669 1163 ts_tablet_manager.cc:1950] T e4b5a7553110424d901788c7a2ecdce6 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:40.560281 591 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 645b08b33676420c96e880de3d460829 (table default.Table_e02933dd91ce4fe7a40d1c3b2f9b2d4b [id=6d42f260246e420d8818b3841926302b]). Expected version 2 got 0
I20251211 20:52:40.560837 885 tablet_service.cc:1505] Processing CreateTablet for tablet 3dfea28c44d84c68850db65a30d52ee6 (DEFAULT_TABLE table=default.Table_c504f8f3796f43b697488bd604268543 [id=2af76b9d32094a4ca656060a4478e08a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.561262 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dfea28c44d84c68850db65a30d52ee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.567714 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 648f43e95e034f2188a20e1b837f2818 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:40.569363 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34083c49cc514fbabfb8457980629185. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.572474 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 829) took 1948 ms. Trace:
I20251211 20:52:40.572681 1163 rpcz_store.cc:276] 1211 20:52:38.625912 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.687062 (+ 61150us) service_pool.cc:225] Handling call
1211 20:52:40.531659 (+1844597us) ts_tablet_manager.cc:1151] Deleting tablet e4b5a7553110424d901788c7a2ecdce6
1211 20:52:40.531669 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.539052 (+ 7383us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.546617 (+ 7565us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.572445 (+ 25828us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1844167}
I20251211 20:52:40.573144 885 tablet_service.cc:1505] Processing CreateTablet for tablet 3416459feffc4f7f98e2d0e2b8109201 (DEFAULT_TABLE table=default.Table_d0cd234de1cb4bb485da3d10b527eb65 [id=4ea51139b7824638a5ad1eaf2c0f11f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.573352 746 tablet_service.cc:1505] Processing CreateTablet for tablet 3416459feffc4f7f98e2d0e2b8109201 (DEFAULT_TABLE table=default.Table_d0cd234de1cb4bb485da3d10b527eb65 [id=4ea51139b7824638a5ad1eaf2c0f11f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.573495 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18f125dd9efc401181f0644dad7aeffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.573675 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3416459feffc4f7f98e2d0e2b8109201. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.573764 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3416459feffc4f7f98e2d0e2b8109201. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.574311 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ca98b43863a47c091baae0a6ce65630. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.579278 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 79d92736bb1c46d2bfa5678b2d396ec2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:40.579866 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 3416459feffc4f7f98e2d0e2b8109201 (DEFAULT_TABLE table=default.Table_d0cd234de1cb4bb485da3d10b527eb65 [id=4ea51139b7824638a5ad1eaf2c0f11f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.580310 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3416459feffc4f7f98e2d0e2b8109201. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.580521 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet e4b5a7553110424d901788c7a2ecdce6 (table default.Table_9ee6894ba6364a54be16bb427ac7197b [id=04d7acd50017488eaa11f783233b6d4b]) successfully deleted
I20251211 20:52:40.575515 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 832) took 2413 ms. Trace:
I20251211 20:52:40.582173 1165 rpcz_store.cc:276] 1211 20:52:38.162511 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.163037 (+ 526us) service_pool.cc:225] Handling call
1211 20:52:40.219832 (+2056795us) ts_tablet_manager.cc:1151] Deleting tablet ccc3ddd4b7f74f11a3835997cb7ae8d0
1211 20:52:40.219845 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.475993 (+256148us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.478755 (+ 2762us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.575496 (+ 96741us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2056536}
I20251211 20:52:40.582281 2216 consensus_queue.cc:1048] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:40.582667 1166 tablet_bootstrap.cc:654] T fed300d31c414f4a9b27c66e78097688 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.582656 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet ccc3ddd4b7f74f11a3835997cb7ae8d0 (table default.Table_63e0e08eb63c40b9aece6be45b6d4944 [id=8e1b3bb8f14e49d19eecdb540ff2d85d]) successfully deleted
I20251211 20:52:40.583034 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 834) took 2413 ms. Trace:
I20251211 20:52:40.583112 1165 rpcz_store.cc:276] 1211 20:52:38.169465 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.169746 (+ 281us) service_pool.cc:225] Handling call
1211 20:52:40.582902 (+2413156us) ts_tablet_manager.cc:1151] Deleting tablet a471a9accbfc4875bd1d7802f5620808
1211 20:52:40.582913 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.583017 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2412532}
W20251211 20:52:40.583525 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet a471a9accbfc4875bd1d7802f5620808 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a471a9accbfc4875bd1d7802f5620808 already in progress: creating tablet
I20251211 20:52:40.583905 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 840) took 2355 ms. Trace:
I20251211 20:52:40.583981 1165 rpcz_store.cc:276] 1211 20:52:38.228825 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.229305 (+ 480us) service_pool.cc:225] Handling call
1211 20:52:40.583787 (+2354482us) ts_tablet_manager.cc:1151] Deleting tablet 745ce63314834c8abdecaa367133a068
1211 20:52:40.583797 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.583887 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2353795}
W20251211 20:52:40.584061 710 consensus_peers.cc:597] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b529b38d754f46ba8cb60b19e9705256. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:40.584308 581 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 745ce63314834c8abdecaa367133a068 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 745ce63314834c8abdecaa367133a068 already in progress: creating tablet
I20251211 20:52:40.584589 1165 tablet_replica.cc:333] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:40.584796 1165 raft_consensus.cc:2243] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:40.584972 1165 raft_consensus.cc:2272] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.579866 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 3dfea28c44d84c68850db65a30d52ee6 (DEFAULT_TABLE table=default.Table_c504f8f3796f43b697488bd604268543 [id=2af76b9d32094a4ca656060a4478e08a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.587239 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dfea28c44d84c68850db65a30d52ee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.590215 877 tablet_service.cc:1505] Processing CreateTablet for tablet 3809a89dfb31442d99455e06733b132e (DEFAULT_TABLE table=default.Table_f2301327ddd04d278201a1cb0da3afe8 [id=ace2fe24e1504556a7b7bd455d7c6c8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.593647 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 3809a89dfb31442d99455e06733b132e (DEFAULT_TABLE table=default.Table_f2301327ddd04d278201a1cb0da3afe8 [id=ace2fe24e1504556a7b7bd455d7c6c8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.594134 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3809a89dfb31442d99455e06733b132e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.595206 757 tablet_service.cc:1505] Processing CreateTablet for tablet 16d2ff8925e342729a71c93db378d031 (DEFAULT_TABLE table=default.Table_f6f977edac0f45bea7e94de5bfa51ee4 [id=4a74b9a60cbb4198b55cee630b2f4bef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.595533 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16d2ff8925e342729a71c93db378d031. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.596572 2216 raft_consensus.cc:493] T a3b1aadf17ea4bf0a896d23cd543fca7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:40.596711 2216 raft_consensus.cc:515] T a3b1aadf17ea4bf0a896d23cd543fca7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.597002 891 tablet_service.cc:1505] Processing CreateTablet for tablet 16d2ff8925e342729a71c93db378d031 (DEFAULT_TABLE table=default.Table_f6f977edac0f45bea7e94de5bfa51ee4 [id=4a74b9a60cbb4198b55cee630b2f4bef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.597290 2216 leader_election.cc:290] T a3b1aadf17ea4bf0a896d23cd543fca7 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:40.598001 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a3b1aadf17ea4bf0a896d23cd543fca7" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:40.598100 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a3b1aadf17ea4bf0a896d23cd543fca7" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:40.598738 983 leader_election.cc:343] T a3b1aadf17ea4bf0a896d23cd543fca7 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:40.598984 982 leader_election.cc:343] T a3b1aadf17ea4bf0a896d23cd543fca7 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:40.599071 982 leader_election.cc:304] T a3b1aadf17ea4bf0a896d23cd543fca7 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:40.599454 2216 raft_consensus.cc:2749] T a3b1aadf17ea4bf0a896d23cd543fca7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:40.599841 1166 tablet_bootstrap.cc:492] T fed300d31c414f4a9b27c66e78097688 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:40.600201 746 tablet_service.cc:1505] Processing CreateTablet for tablet 3809a89dfb31442d99455e06733b132e (DEFAULT_TABLE table=default.Table_f2301327ddd04d278201a1cb0da3afe8 [id=ace2fe24e1504556a7b7bd455d7c6c8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.600345 1165 ts_tablet_manager.cc:1916] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.601131 889 tablet_service.cc:1505] Processing CreateTablet for tablet ecc347e0b71a48e4ab803280232d139e (DEFAULT_TABLE table=default.Table_86d6443f6a564414a7f024e4aa12d450 [id=c1cd2830fb0648fab33480850c1d2adb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.599906 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 16d2ff8925e342729a71c93db378d031 (DEFAULT_TABLE table=default.Table_f6f977edac0f45bea7e94de5bfa51ee4 [id=4a74b9a60cbb4198b55cee630b2f4bef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.600055 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 840) took 1912 ms. Trace:
I20251211 20:52:40.603984 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16d2ff8925e342729a71c93db378d031. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.603963 1163 rpcz_store.cc:276] 1211 20:52:38.688040 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.701806 (+ 13766us) service_pool.cc:225] Handling call
1211 20:52:40.573070 (+1871264us) ts_tablet_manager.cc:1151] Deleting tablet 6029c1fd563d4023a8cf7cfb8ffa0876
1211 20:52:40.573081 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.600038 (+ 26957us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1870218}
W20251211 20:52:40.604524 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 6029c1fd563d4023a8cf7cfb8ffa0876 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6029c1fd563d4023a8cf7cfb8ffa0876 already in progress: creating tablet
I20251211 20:52:40.606446 757 tablet_service.cc:1505] Processing CreateTablet for tablet ecc347e0b71a48e4ab803280232d139e (DEFAULT_TABLE table=default.Table_86d6443f6a564414a7f024e4aa12d450 [id=c1cd2830fb0648fab33480850c1d2adb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.606882 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecc347e0b71a48e4ab803280232d139e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.607653 884 tablet_service.cc:1505] Processing CreateTablet for tablet 4585e05d06ef42cfb06c4ceb7524fd71 (DEFAULT_TABLE table=default.Table_72e3939e6805448ba5d1f87e6dad3d3a [id=c5967718ed0a431e839130be06a114bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.608767 1015 tablet_service.cc:1505] Processing CreateTablet for tablet ecc347e0b71a48e4ab803280232d139e (DEFAULT_TABLE table=default.Table_86d6443f6a564414a7f024e4aa12d450 [id=c1cd2830fb0648fab33480850c1d2adb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.609090 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecc347e0b71a48e4ab803280232d139e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.609491 1166 ts_tablet_manager.cc:1403] T fed300d31c414f4a9b27c66e78097688 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.058s user 0.003s sys 0.003s
I20251211 20:52:40.610213 1166 raft_consensus.cc:359] T fed300d31c414f4a9b27c66e78097688 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.610357 1166 raft_consensus.cc:385] T fed300d31c414f4a9b27c66e78097688 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.610399 1166 raft_consensus.cc:740] T fed300d31c414f4a9b27c66e78097688 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.610345 894 tablet_service.cc:1505] Processing CreateTablet for tablet 46ddf50b7e5c45e4b9a6483bb4346873 (DEFAULT_TABLE table=default.Table_18ec995eeab648dd8f0399b55e571ece [id=a362f3dbfa82447483dc50751ed0a9b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.611546 756 tablet_service.cc:1505] Processing CreateTablet for tablet 46ddf50b7e5c45e4b9a6483bb4346873 (DEFAULT_TABLE table=default.Table_18ec995eeab648dd8f0399b55e571ece [id=a362f3dbfa82447483dc50751ed0a9b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.611842 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 4585e05d06ef42cfb06c4ceb7524fd71 (DEFAULT_TABLE table=default.Table_72e3939e6805448ba5d1f87e6dad3d3a [id=c5967718ed0a431e839130be06a114bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.611951 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46ddf50b7e5c45e4b9a6483bb4346873. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.612751 1168 tablet_bootstrap.cc:492] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:40.612888 1168 ts_tablet_manager.cc:1403] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.057s user 0.006s sys 0.002s
I20251211 20:52:40.613667 1168 raft_consensus.cc:359] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.613823 1168 raft_consensus.cc:385] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.613883 1168 raft_consensus.cc:740] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.614087 1168 consensus_queue.cc:260] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.614537 754 tablet_service.cc:1505] Processing CreateTablet for tablet 4585e05d06ef42cfb06c4ceb7524fd71 (DEFAULT_TABLE table=default.Table_72e3939e6805448ba5d1f87e6dad3d3a [id=c5967718ed0a431e839130be06a114bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.614614 1168 ts_tablet_manager.cc:1434] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:40.614853 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4585e05d06ef42cfb06c4ceb7524fd71. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.616297 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4585e05d06ef42cfb06c4ceb7524fd71. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.616367 1166 consensus_queue.cc:260] T fed300d31c414f4a9b27c66e78097688 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:40.619740 983 consensus_peers.cc:597] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:40.614864 1168 tablet_bootstrap.cc:492] T 22273287aa9245a2a66131fed2c8861c P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:40.621320 1015 tablet_service.cc:1505] Processing CreateTablet for tablet 46ddf50b7e5c45e4b9a6483bb4346873 (DEFAULT_TABLE table=default.Table_18ec995eeab648dd8f0399b55e571ece [id=a362f3dbfa82447483dc50751ed0a9b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.621470 1168 tablet_bootstrap.cc:654] T 22273287aa9245a2a66131fed2c8861c P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.621730 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46ddf50b7e5c45e4b9a6483bb4346873. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.622720 1166 ts_tablet_manager.cc:1434] T fed300d31c414f4a9b27c66e78097688 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.013s user 0.001s sys 0.000s
I20251211 20:52:40.622982 1166 tablet_bootstrap.cc:492] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:40.624030 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3809a89dfb31442d99455e06733b132e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.624475 1166 tablet_bootstrap.cc:654] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.627147 1163 tablet_replica.cc:333] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:40.627337 1163 raft_consensus.cc:2243] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:40.627669 1163 raft_consensus.cc:2272] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.628424 1163 ts_tablet_manager.cc:1916] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.628484 890 tablet_service.cc:1505] Processing CreateTablet for tablet daf364774dea41bb8e94b49db55b8ea5 (DEFAULT_TABLE table=default.Table_0e243018f79e4e358c4f844e6ba51c35 [id=655a160341774729972a39fb7220a685]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.629395 1160 tablet_bootstrap.cc:492] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:40.629519 1160 ts_tablet_manager.cc:1403] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.163s user 0.003s sys 0.000s
I20251211 20:52:40.630192 1160 raft_consensus.cc:359] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.630364 1160 raft_consensus.cc:385] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.630422 1160 raft_consensus.cc:740] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.630466 1010 tablet_service.cc:1505] Processing CreateTablet for tablet daf364774dea41bb8e94b49db55b8ea5 (DEFAULT_TABLE table=default.Table_0e243018f79e4e358c4f844e6ba51c35 [id=655a160341774729972a39fb7220a685]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.630862 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daf364774dea41bb8e94b49db55b8ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.631379 881 tablet_service.cc:1505] Processing CreateTablet for tablet ae45b0b63c554fd79ff243f14a3820c4 (DEFAULT_TABLE table=default.Table_d8d764377e524faf931940bfe23100be [id=198db6e18dcf4f65bde2405f4f0bad0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.632272 1160 consensus_queue.cc:260] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.632757 1160 ts_tablet_manager.cc:1434] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:40.633728 1017 tablet_service.cc:1505] Processing CreateTablet for tablet ae45b0b63c554fd79ff243f14a3820c4 (DEFAULT_TABLE table=default.Table_d8d764377e524faf931940bfe23100be [id=198db6e18dcf4f65bde2405f4f0bad0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.634111 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae45b0b63c554fd79ff243f14a3820c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.635465 880 tablet_service.cc:1505] Processing CreateTablet for tablet 9f71d67cde1d4ac98ee195c3b81b6e75 (DEFAULT_TABLE table=default.Table_fd043ec094f94011986fc8f5a9f32f1a [id=994990d2a18f405c988452e93cca7bef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.636698 756 tablet_service.cc:1505] Processing CreateTablet for tablet 9f71d67cde1d4ac98ee195c3b81b6e75 (DEFAULT_TABLE table=default.Table_fd043ec094f94011986fc8f5a9f32f1a [id=994990d2a18f405c988452e93cca7bef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.637434 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f71d67cde1d4ac98ee195c3b81b6e75. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.640610 1163 ts_tablet_manager.cc:1929] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:40.640718 1163 log.cc:1199] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/b631e6b3048e428f901af1b5b0b1ab3c
W20251211 20:52:40.641535 710 consensus_peers.cc:597] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 793f543d6b614ad7b3ab8e717db2ceaf. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:40.645627 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 9f71d67cde1d4ac98ee195c3b81b6e75 (DEFAULT_TABLE table=default.Table_fd043ec094f94011986fc8f5a9f32f1a [id=994990d2a18f405c988452e93cca7bef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.645995 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f71d67cde1d4ac98ee195c3b81b6e75. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.646160 746 tablet_service.cc:1505] Processing CreateTablet for tablet 53b4116db0a44dfbb4c1ec5b2acebc67 (DEFAULT_TABLE table=default.Table_6829fee154a74021a1410d7535c0c89f [id=aea8a10b336548f78e1c093ce01a22a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.646590 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53b4116db0a44dfbb4c1ec5b2acebc67. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.646924 739 tablet_service.cc:1505] Processing CreateTablet for tablet daf364774dea41bb8e94b49db55b8ea5 (DEFAULT_TABLE table=default.Table_0e243018f79e4e358c4f844e6ba51c35 [id=655a160341774729972a39fb7220a685]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.647265 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daf364774dea41bb8e94b49db55b8ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.647539 2135 raft_consensus.cc:493] T fed300d31c414f4a9b27c66e78097688 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:40.647711 2135 raft_consensus.cc:515] T fed300d31c414f4a9b27c66e78097688 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.648263 2135 leader_election.cc:290] T fed300d31c414f4a9b27c66e78097688 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:40.648794 591 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_fbf522a147ec4a04950c849072ebb619" } new_table_name: "default.Table_44f971096d1e41f480e594985ffd5c17" modify_external_catalogs: true
I20251211 20:52:40.648900 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fed300d31c414f4a9b27c66e78097688" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:40.648926 879 tablet_service.cc:1505] Processing CreateTablet for tablet 53b4116db0a44dfbb4c1ec5b2acebc67 (DEFAULT_TABLE table=default.Table_6829fee154a74021a1410d7535c0c89f [id=aea8a10b336548f78e1c093ce01a22a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.649097 1015 tablet_service.cc:1505] Processing CreateTablet for tablet 53b4116db0a44dfbb4c1ec5b2acebc67 (DEFAULT_TABLE table=default.Table_6829fee154a74021a1410d7535c0c89f [id=aea8a10b336548f78e1c093ce01a22a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.649395 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53b4116db0a44dfbb4c1ec5b2acebc67. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:40.649518 982 leader_election.cc:343] T fed300d31c414f4a9b27c66e78097688 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:40.651859 1163 ts_tablet_manager.cc:1950] T b631e6b3048e428f901af1b5b0b1ab3c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:40.654822 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 843) took 1939 ms. Trace:
I20251211 20:52:40.654968 1163 rpcz_store.cc:276] 1211 20:52:38.715161 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.715290 (+ 129us) service_pool.cc:225] Handling call
1211 20:52:40.627097 (+1911807us) ts_tablet_manager.cc:1151] Deleting tablet b631e6b3048e428f901af1b5b0b1ab3c
1211 20:52:40.627108 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.638050 (+ 10942us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.640514 (+ 2464us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.654803 (+ 14289us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1911614}
I20251211 20:52:40.655433 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet b631e6b3048e428f901af1b5b0b1ab3c (table default.Table_dd84a1d3fe964c76b146f798f6781baa [id=e80ce2c7f3aa48539b6d2fad357b3b49]) successfully deleted
I20251211 20:52:40.655752 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fed300d31c414f4a9b27c66e78097688" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:40.656394 1160 tablet_bootstrap.cc:492] T c7bd218b3ff340b2a589c9517b49a649 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:40.656708 886 tablet_service.cc:1505] Processing CreateTablet for tablet 78f77d8916d84edabbe95ce17d8df2c1 (DEFAULT_TABLE table=default.Table_f21e4bd4ff9846c8b37a5723d0fc3c3e [id=ddc12ae6d8ed4587b5ef4da3ef81a055]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:40.657775 983 leader_election.cc:343] T fed300d31c414f4a9b27c66e78097688 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:40.657887 983 leader_election.cc:304] T fed300d31c414f4a9b27c66e78097688 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:40.657953 746 tablet_service.cc:1505] Processing CreateTablet for tablet 78f77d8916d84edabbe95ce17d8df2c1 (DEFAULT_TABLE table=default.Table_f21e4bd4ff9846c8b37a5723d0fc3c3e [id=ddc12ae6d8ed4587b5ef4da3ef81a055]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.658239 2135 raft_consensus.cc:2749] T fed300d31c414f4a9b27c66e78097688 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:40.658279 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78f77d8916d84edabbe95ce17d8df2c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.658869 1160 tablet_bootstrap.cc:654] T c7bd218b3ff340b2a589c9517b49a649 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.659849 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 839) took 1971 ms. Trace:
I20251211 20:52:40.659972 1163 rpcz_store.cc:276] 1211 20:52:38.687929 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.791652 (+103723us) service_pool.cc:225] Handling call
1211 20:52:40.659731 (+1868079us) ts_tablet_manager.cc:1151] Deleting tablet 449921ea74f04a9c88eb9e094e8226c8
1211 20:52:40.659741 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.659832 (+ 91us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1867793}
W20251211 20:52:40.660455 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 449921ea74f04a9c88eb9e094e8226c8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 449921ea74f04a9c88eb9e094e8226c8 already in progress: creating tablet
I20251211 20:52:40.660692 883 tablet_service.cc:1505] Processing CreateTablet for tablet eb6ae7c6ed0b40e19300d2922681205c (DEFAULT_TABLE table=default.Table_6a85485dbf204937a1c5a8e0e151fc68 [id=856b47e2455f4727a81e6291ab2ccf38]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.660851 1163 tablet_replica.cc:333] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:40.661077 1163 raft_consensus.cc:2243] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:40.661325 1163 pending_rounds.cc:70] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7: Trying to abort 1 pending ops.
I20251211 20:52:40.661774 1163 pending_rounds.cc:77] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432129436774400 op_type: NO_OP noop_request { }
I20251211 20:52:40.661775 756 tablet_service.cc:1505] Processing CreateTablet for tablet eb6ae7c6ed0b40e19300d2922681205c (DEFAULT_TABLE table=default.Table_6a85485dbf204937a1c5a8e0e151fc68 [id=856b47e2455f4727a81e6291ab2ccf38]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.661942 1163 raft_consensus.cc:2889] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:40.662039 1163 raft_consensus.cc:2272] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.661648 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 78f77d8916d84edabbe95ce17d8df2c1 (DEFAULT_TABLE table=default.Table_f21e4bd4ff9846c8b37a5723d0fc3c3e [id=ddc12ae6d8ed4587b5ef4da3ef81a055]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.662295 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb6ae7c6ed0b40e19300d2922681205c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.662526 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78f77d8916d84edabbe95ce17d8df2c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.663192 1163 ts_tablet_manager.cc:1916] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.669157 1019 tablet_service.cc:1505] Processing CreateTablet for tablet eb6ae7c6ed0b40e19300d2922681205c (DEFAULT_TABLE table=default.Table_6a85485dbf204937a1c5a8e0e151fc68 [id=856b47e2455f4727a81e6291ab2ccf38]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.669545 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb6ae7c6ed0b40e19300d2922681205c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.673231 1160 tablet_bootstrap.cc:492] T c7bd218b3ff340b2a589c9517b49a649 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:40.673434 1160 ts_tablet_manager.cc:1403] T c7bd218b3ff340b2a589c9517b49a649 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.002s
I20251211 20:52:40.674464 1160 raft_consensus.cc:359] T c7bd218b3ff340b2a589c9517b49a649 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.674669 1160 raft_consensus.cc:385] T c7bd218b3ff340b2a589c9517b49a649 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.674727 1160 raft_consensus.cc:740] T c7bd218b3ff340b2a589c9517b49a649 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.679550 1160 consensus_queue.cc:260] T c7bd218b3ff340b2a589c9517b49a649 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.680462 1166 tablet_bootstrap.cc:492] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:40.680608 1168 tablet_bootstrap.cc:492] T 22273287aa9245a2a66131fed2c8861c P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:40.680608 1166 ts_tablet_manager.cc:1403] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.058s user 0.003s sys 0.000s
I20251211 20:52:40.680764 1168 ts_tablet_manager.cc:1403] T 22273287aa9245a2a66131fed2c8861c P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.066s user 0.001s sys 0.001s
I20251211 20:52:40.681311 1166 raft_consensus.cc:359] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.681389 1168 raft_consensus.cc:359] T 22273287aa9245a2a66131fed2c8861c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.681537 1166 raft_consensus.cc:385] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.681540 1168 raft_consensus.cc:385] T 22273287aa9245a2a66131fed2c8861c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.681604 1168 raft_consensus.cc:740] T 22273287aa9245a2a66131fed2c8861c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.681592 1166 raft_consensus.cc:740] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.681834 1166 consensus_queue.cc:260] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.682371 1166 ts_tablet_manager.cc:1434] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:40.682307 1168 consensus_queue.cc:260] T 22273287aa9245a2a66131fed2c8861c P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.682629 1166 tablet_bootstrap.cc:492] T a9bb22dce13f4c9da76ca446715cd99a P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:40.684043 1160 ts_tablet_manager.cc:1434] T c7bd218b3ff340b2a589c9517b49a649 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.011s user 0.000s sys 0.002s
I20251211 20:52:40.684342 1160 tablet_bootstrap.cc:492] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:40.684445 1163 ts_tablet_manager.cc:1929] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:40.684544 1163 log.cc:1199] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/52ef1707f6f2425aafd454f303af0333
I20251211 20:52:40.684698 756 tablet_service.cc:1505] Processing CreateTablet for tablet b5b68f08566e4c01b004ecdab970fffa (DEFAULT_TABLE table=default.Table_4b444339f63048179e9b53a3aec9173c [id=1f67fc9412d2431ab260b87363248dd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.684886 1163 ts_tablet_manager.cc:1950] T 52ef1707f6f2425aafd454f303af0333 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:40.685598 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 850) took 1807 ms. Trace:
I20251211 20:52:40.685639 887 tablet_service.cc:1505] Processing CreateTablet for tablet ca89f09cc0954389b679e8bb8b5e4dfb (DEFAULT_TABLE table=default.Table_52e650ce0b824a41a39fcd4f1b8ba455 [id=748adb1caf074cf18655c4d7af403663]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.685679 1163 rpcz_store.cc:276] 1211 20:52:38.877676 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.878307 (+ 631us) service_pool.cc:225] Handling call
1211 20:52:40.660767 (+1782460us) ts_tablet_manager.cc:1151] Deleting tablet 52ef1707f6f2425aafd454f303af0333
1211 20:52:40.660777 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.665966 (+ 5189us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.684345 (+ 18379us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.685578 (+ 1233us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1753783}
I20251211 20:52:40.685973 1160 tablet_bootstrap.cc:654] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.686058 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 849) took 1808 ms. Trace:
I20251211 20:52:40.686118 1163 rpcz_store.cc:276] 1211 20:52:38.877527 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.886765 (+ 9238us) service_pool.cc:225] Handling call
1211 20:52:40.685939 (+1799174us) ts_tablet_manager.cc:1151] Deleting tablet 80c566822fa94ada983aa7266df4bf9d
1211 20:52:40.685948 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.686044 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1776503}
I20251211 20:52:40.686323 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 52ef1707f6f2425aafd454f303af0333 (table default.Table_cedad3ae249f4796b8c3756c4f4d6f74 [id=c5b8a1dc09f146b782f6d67ac53f0e8c]) successfully deleted
I20251211 20:52:40.686417 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 856) took 1752 ms. Trace:
I20251211 20:52:40.686481 1163 rpcz_store.cc:276] 1211 20:52:38.933748 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.933809 (+ 61us) service_pool.cc:225] Handling call
1211 20:52:40.686326 (+1752517us) ts_tablet_manager.cc:1151] Deleting tablet e95c63b4d99a4dd0b2c0fa384ca9d552
1211 20:52:40.686334 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.686404 (+ 70us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1751644}
I20251211 20:52:40.686746 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 857) took 1659 ms. Trace:
I20251211 20:52:40.686813 1163 rpcz_store.cc:276] 1211 20:52:39.027418 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.027474 (+ 56us) service_pool.cc:225] Handling call
1211 20:52:40.686658 (+1659184us) ts_tablet_manager.cc:1151] Deleting tablet d15eff49af284a38bb1babdb231a35df
1211 20:52:40.686666 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.686733 (+ 67us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1658876}
W20251211 20:52:40.687006 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 80c566822fa94ada983aa7266df4bf9d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 80c566822fa94ada983aa7266df4bf9d already in progress: creating tablet
I20251211 20:52:40.687107 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 858) took 1620 ms. Trace:
I20251211 20:52:40.687160 1166 tablet_bootstrap.cc:654] T a9bb22dce13f4c9da76ca446715cd99a P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:40.687242 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet e95c63b4d99a4dd0b2c0fa384ca9d552 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e95c63b4d99a4dd0b2c0fa384ca9d552 already in progress: creating tablet
W20251211 20:52:40.687708 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet d15eff49af284a38bb1babdb231a35df because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d15eff49af284a38bb1babdb231a35df already in progress: creating tablet
I20251211 20:52:40.688874 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5b68f08566e4c01b004ecdab970fffa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.688978 754 tablet_service.cc:1505] Processing CreateTablet for tablet ae45b0b63c554fd79ff243f14a3820c4 (DEFAULT_TABLE table=default.Table_d8d764377e524faf931940bfe23100be [id=198db6e18dcf4f65bde2405f4f0bad0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.695950 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3809a89dfb31442d99455e06733b132e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.696382 1160 tablet_bootstrap.cc:492] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:40.696517 1160 ts_tablet_manager.cc:1403] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.004s
I20251211 20:52:40.696399 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53b4116db0a44dfbb4c1ec5b2acebc67. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.696871 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae45b0b63c554fd79ff243f14a3820c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.697196 1160 raft_consensus.cc:359] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.697346 1160 raft_consensus.cc:385] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.697392 1160 raft_consensus.cc:740] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.695652 1019 tablet_service.cc:1505] Processing CreateTablet for tablet ca89f09cc0954389b679e8bb8b5e4dfb (DEFAULT_TABLE table=default.Table_52e650ce0b824a41a39fcd4f1b8ba455 [id=748adb1caf074cf18655c4d7af403663]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.697757 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca89f09cc0954389b679e8bb8b5e4dfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.699412 1160 consensus_queue.cc:260] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.699693 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb6ae7c6ed0b40e19300d2922681205c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.700335 1160 ts_tablet_manager.cc:1434] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:40.700579 1160 tablet_bootstrap.cc:492] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:40.702095 1160 tablet_bootstrap.cc:654] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.703253 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78f77d8916d84edabbe95ce17d8df2c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.704852 888 tablet_service.cc:1505] Processing CreateTablet for tablet b5b68f08566e4c01b004ecdab970fffa (DEFAULT_TABLE table=default.Table_4b444339f63048179e9b53a3aec9173c [id=1f67fc9412d2431ab260b87363248dd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.705173 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5b68f08566e4c01b004ecdab970fffa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.707973 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecc347e0b71a48e4ab803280232d139e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.710812 885 tablet_service.cc:1505] Processing CreateTablet for tablet 622e513e55d749f2a0105b5d6f9444c8 (DEFAULT_TABLE table=default.Table_ec0bf98cd1ce42ceacacde30208d5d04 [id=b8fdb6772ace427e8273983d1e791956]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.711174 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 622e513e55d749f2a0105b5d6f9444c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.711720 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daf364774dea41bb8e94b49db55b8ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.713089 875 tablet_service.cc:1505] Processing CreateTablet for tablet aa50ddb2789b4be3bfa9aedef9287109 (DEFAULT_TABLE table=default.Table_fbf522a147ec4a04950c849072ebb619 [id=d4e1349374a346a5b8be180f9bdc329a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.713335 1022 tablet_service.cc:1505] Processing CreateTablet for tablet aa50ddb2789b4be3bfa9aedef9287109 (DEFAULT_TABLE table=default.Table_fbf522a147ec4a04950c849072ebb619 [id=d4e1349374a346a5b8be180f9bdc329a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.713428 875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa50ddb2789b4be3bfa9aedef9287109. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.713686 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa50ddb2789b4be3bfa9aedef9287109. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.715698 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4585e05d06ef42cfb06c4ceb7524fd71. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.719173 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46ddf50b7e5c45e4b9a6483bb4346873. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.719898 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 622e513e55d749f2a0105b5d6f9444c8 (DEFAULT_TABLE table=default.Table_ec0bf98cd1ce42ceacacde30208d5d04 [id=b8fdb6772ace427e8273983d1e791956]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.720240 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 622e513e55d749f2a0105b5d6f9444c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.722692 1813 raft_consensus.cc:493] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:40.722807 1813 raft_consensus.cc:515] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.723342 1813 leader_election.cc:290] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:40.723694 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca89f09cc0954389b679e8bb8b5e4dfb. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:40.724905 847 leader_election.cc:343] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 6a526a6d1bc84cefad2c5d897223292e
W20251211 20:52:40.725219 847 leader_election.cc:343] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 6a526a6d1bc84cefad2c5d897223292e
I20251211 20:52:40.725301 847 leader_election.cc:304] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:40.725881 1813 raft_consensus.cc:2749] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:40.726336 1015 tablet_service.cc:1505] Processing CreateTablet for tablet b5b68f08566e4c01b004ecdab970fffa (DEFAULT_TABLE table=default.Table_4b444339f63048179e9b53a3aec9173c [id=1f67fc9412d2431ab260b87363248dd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.726675 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5b68f08566e4c01b004ecdab970fffa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.727706 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16d2ff8925e342729a71c93db378d031. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.733598 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f71d67cde1d4ac98ee195c3b81b6e75. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.687170 1163 rpcz_store.cc:276] 1211 20:52:39.067035 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.067089 (+ 54us) service_pool.cc:225] Handling call
1211 20:52:40.687018 (+1619929us) ts_tablet_manager.cc:1151] Deleting tablet c97d410272a947de8b5d7fb40fe4777d
1211 20:52:40.687026 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.687092 (+ 66us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1619753}
W20251211 20:52:40.734573 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet c97d410272a947de8b5d7fb40fe4777d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c97d410272a947de8b5d7fb40fe4777d already in progress: creating tablet
I20251211 20:52:40.735045 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 859) took 1309 ms. Trace:
I20251211 20:52:40.735121 1163 rpcz_store.cc:276] 1211 20:52:39.425885 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.425952 (+ 67us) service_pool.cc:225] Handling call
1211 20:52:40.734844 (+1308892us) ts_tablet_manager.cc:1151] Deleting tablet e9ead34065e240eeb1f946319537d32a
1211 20:52:40.734855 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.735026 (+ 171us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1294745}
W20251211 20:52:40.735559 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet e9ead34065e240eeb1f946319537d32a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e9ead34065e240eeb1f946319537d32a already in progress: creating tablet
I20251211 20:52:40.735934 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 861) took 1181 ms. Trace:
I20251211 20:52:40.736001 1163 rpcz_store.cc:276] 1211 20:52:39.554263 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.555809 (+ 1546us) service_pool.cc:225] Handling call
1211 20:52:40.735827 (+1180018us) ts_tablet_manager.cc:1151] Deleting tablet 7bc03ed8c35740f58fe48ddbaa120ea0
1211 20:52:40.735837 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.735918 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1138769}
W20251211 20:52:40.736346 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 7bc03ed8c35740f58fe48ddbaa120ea0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7bc03ed8c35740f58fe48ddbaa120ea0 already in progress: creating tablet
I20251211 20:52:40.736663 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 862) took 1180 ms. Trace:
I20251211 20:52:40.736727 1163 rpcz_store.cc:276] 1211 20:52:39.556124 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.556203 (+ 79us) service_pool.cc:225] Handling call
1211 20:52:40.736557 (+1180354us) ts_tablet_manager.cc:1151] Deleting tablet a295a24e81f14e5fa126a4a58ffd08cf
1211 20:52:40.736566 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.736647 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1139359}
W20251211 20:52:40.737102 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet a295a24e81f14e5fa126a4a58ffd08cf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a295a24e81f14e5fa126a4a58ffd08cf already in progress: creating tablet
I20251211 20:52:40.737387 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 864) took 1177 ms. Trace:
I20251211 20:52:40.737448 1163 rpcz_store.cc:276] 1211 20:52:39.559742 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.559809 (+ 67us) service_pool.cc:225] Handling call
1211 20:52:40.737295 (+1177486us) ts_tablet_manager.cc:1151] Deleting tablet 48820386e41b47129a13bdd5768f021b
1211 20:52:40.737304 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.737373 (+ 69us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1140008}
W20251211 20:52:40.737792 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 48820386e41b47129a13bdd5768f021b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 48820386e41b47129a13bdd5768f021b already in progress: creating tablet
I20251211 20:52:40.738195 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 865) took 1160 ms. Trace:
I20251211 20:52:40.738257 1163 rpcz_store.cc:276] 1211 20:52:39.577355 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.577415 (+ 60us) service_pool.cc:225] Handling call
1211 20:52:40.738106 (+1160691us) ts_tablet_manager.cc:1151] Deleting tablet 4585e05d06ef42cfb06c4ceb7524fd71
1211 20:52:40.738115 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.738182 (+ 67us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1138769}
W20251211 20:52:40.738611 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 4585e05d06ef42cfb06c4ceb7524fd71 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4585e05d06ef42cfb06c4ceb7524fd71 already in progress: creating tablet
I20251211 20:52:40.738916 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 860) took 1195 ms. Trace:
I20251211 20:52:40.738981 1163 rpcz_store.cc:276] 1211 20:52:39.543129 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.543203 (+ 74us) service_pool.cc:225] Handling call
1211 20:52:40.738815 (+1195612us) ts_tablet_manager.cc:1151] Deleting tablet 4585e05d06ef42cfb06c4ceb7524fd71
1211 20:52:40.738832 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.738903 (+ 71us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1139109}
W20251211 20:52:40.739298 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 4585e05d06ef42cfb06c4ceb7524fd71 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4585e05d06ef42cfb06c4ceb7524fd71 already in progress: creating tablet
I20251211 20:52:40.739691 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 863) took 1181 ms. Trace:
I20251211 20:52:40.739758 1163 rpcz_store.cc:276] 1211 20:52:39.557760 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.557811 (+ 51us) service_pool.cc:225] Handling call
1211 20:52:40.739571 (+1181760us) ts_tablet_manager.cc:1151] Deleting tablet 6ad6ab53aad14cc2b6d201acfecf6723
1211 20:52:40.739599 (+ 28us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.739672 (+ 73us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1139764}
W20251211 20:52:40.740137 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 6ad6ab53aad14cc2b6d201acfecf6723 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6ad6ab53aad14cc2b6d201acfecf6723 already in progress: creating tablet
I20251211 20:52:40.740404 1163 tablet_replica.cc:333] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:40.740608 1163 raft_consensus.cc:2243] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:40.740957 1163 raft_consensus.cc:2272] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.745205 1163 ts_tablet_manager.cc:1916] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.748060 1166 tablet_bootstrap.cc:492] T a9bb22dce13f4c9da76ca446715cd99a P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:40.757527 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae45b0b63c554fd79ff243f14a3820c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.761448 1166 ts_tablet_manager.cc:1403] T a9bb22dce13f4c9da76ca446715cd99a P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.079s user 0.007s sys 0.003s
I20251211 20:52:40.764572 1166 raft_consensus.cc:359] T a9bb22dce13f4c9da76ca446715cd99a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.764863 1166 raft_consensus.cc:385] T a9bb22dce13f4c9da76ca446715cd99a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.764981 1166 raft_consensus.cc:740] T a9bb22dce13f4c9da76ca446715cd99a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.765261 1166 consensus_queue.cc:260] T a9bb22dce13f4c9da76ca446715cd99a P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.765977 1166 ts_tablet_manager.cc:1434] T a9bb22dce13f4c9da76ca446715cd99a P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251211 20:52:40.766650 1160 tablet_bootstrap.cc:492] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:40.768087 1166 tablet_bootstrap.cc:492] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:40.768301 1160 ts_tablet_manager.cc:1403] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.068s user 0.003s sys 0.001s
I20251211 20:52:40.769821 1160 raft_consensus.cc:359] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.771426 1166 tablet_bootstrap.cc:654] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.661788 1165 ts_tablet_manager.cc:1929] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:40.779893 1165 log.cc:1199] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/b631e6b3048e428f901af1b5b0b1ab3c
I20251211 20:52:40.780230 1165 ts_tablet_manager.cc:1950] T b631e6b3048e428f901af1b5b0b1ab3c P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:40.778872 1164 ts_tablet_manager.cc:1929] T 33addf3d158941098f240cfcdec45d28 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:40.780579 1164 log.cc:1199] T 33addf3d158941098f240cfcdec45d28 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/33addf3d158941098f240cfcdec45d28
I20251211 20:52:40.780895 1164 ts_tablet_manager.cc:1950] T 33addf3d158941098f240cfcdec45d28 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:40.780936 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 854) took 2388 ms. Trace:
I20251211 20:52:40.781015 1165 rpcz_store.cc:276] 1211 20:52:38.392559 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.392622 (+ 63us) service_pool.cc:225] Handling call
1211 20:52:40.584544 (+2191922us) ts_tablet_manager.cc:1151] Deleting tablet b631e6b3048e428f901af1b5b0b1ab3c
1211 20:52:40.584554 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.603079 (+ 18525us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.661688 (+ 58609us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.780915 (+119227us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2191723}
I20251211 20:52:40.781292 1165 tablet_replica.cc:333] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:40.781447 1165 raft_consensus.cc:2243] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:40.781523 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 33addf3d158941098f240cfcdec45d28 (table default.Table_f6a10556ff104db5bd5ecc9fc380e228 [id=f75813add54e46189ff1bbed82d83ba7]) successfully deleted
I20251211 20:52:40.781701 1165 pending_rounds.cc:70] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919: Trying to abort 1 pending ops.
I20251211 20:52:40.781736 1165 pending_rounds.cc:77] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432124880797696 op_type: NO_OP noop_request { }
I20251211 20:52:40.781809 1165 raft_consensus.cc:2889] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:40.781894 1165 raft_consensus.cc:2272] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.782356 1165 ts_tablet_manager.cc:1916] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.789131 1160 raft_consensus.cc:385] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.789249 1160 raft_consensus.cc:740] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
W20251211 20:52:40.790030 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 757ec913dff840f992a41a38b05f8d41 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 757ec913dff840f992a41a38b05f8d41 already in progress: creating tablet
I20251211 20:52:40.790531 1160 consensus_queue.cc:260] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.791066 1160 ts_tablet_manager.cc:1434] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.022s user 0.000s sys 0.001s
I20251211 20:52:40.791363 1160 tablet_bootstrap.cc:492] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:40.792974 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 4776c456b4d74c90b1fc71d57b62a461
I20251211 20:52:40.793008 1160 tablet_bootstrap.cc:654] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.774109 2135 raft_consensus.cc:493] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:40.794373 2135 raft_consensus.cc:515] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.794992 2135 leader_election.cc:290] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:40.795570 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "260bdeb73439437cbf234d20f7264ab3" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:40.796139 983 leader_election.cc:343] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:40.796271 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e841712540dc46ffa1821599e1f10e86" } modify_external_catalogs: true
I20251211 20:52:40.799129 1015 tablet_service.cc:1552] Processing DeleteTablet for tablet 4776c456b4d74c90b1fc71d57b62a461 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:40.803444 710 consensus_peers.cc:597] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2197822e0f11484f9679c4858626ac2c. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:40.730543 757 tablet_service.cc:1505] Processing CreateTablet for tablet ca89f09cc0954389b679e8bb8b5e4dfb (DEFAULT_TABLE table=default.Table_52e650ce0b824a41a39fcd4f1b8ba455 [id=748adb1caf074cf18655c4d7af403663]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.806852 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 4776c456b4d74c90b1fc71d57b62a461 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:40.807071 881 tablet_service.cc:1552] Processing DeleteTablet for tablet 68be21a25c6743f5b86d518d98fbc35e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:40.733248 2286 raft_consensus.cc:493] T 22273287aa9245a2a66131fed2c8861c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:40.807226 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca89f09cc0954389b679e8bb8b5e4dfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:40.807245 2286 raft_consensus.cc:515] T 22273287aa9245a2a66131fed2c8861c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.807863 2286 leader_election.cc:290] T 22273287aa9245a2a66131fed2c8861c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:40.807972 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "260bdeb73439437cbf234d20f7264ab3" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:40.810359 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_e841712540dc46ffa1821599e1f10e86" } modify_external_catalogs: true
I20251211 20:52:40.812379 593 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet a56f1a7bd5464d289bfd018c826ee574
I20251211 20:52:40.813040 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "22273287aa9245a2a66131fed2c8861c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:40.813575 711 leader_election.cc:343] T 22273287aa9245a2a66131fed2c8861c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:40.813727 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "22273287aa9245a2a66131fed2c8861c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
W20251211 20:52:40.814355 982 leader_election.cc:343] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:40.814487 982 leader_election.cc:304] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:40.814677 710 leader_election.cc:343] T 22273287aa9245a2a66131fed2c8861c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:40.814777 710 leader_election.cc:304] T 22273287aa9245a2a66131fed2c8861c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:40.814929 2135 raft_consensus.cc:2749] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:40.814477 1163 ts_tablet_manager.cc:1929] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:40.815186 2286 raft_consensus.cc:2749] T 22273287aa9245a2a66131fed2c8861c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:40.815274 1163 log.cc:1199] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/bb0fc179a337469eae056c5bb3815e6b
I20251211 20:52:40.815649 1163 ts_tablet_manager.cc:1950] T bb0fc179a337469eae056c5bb3815e6b P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:40.816285 593 catalog_manager.cc:6724] ReplaceTablet: tablet a56f1a7bd5464d289bfd018c826ee574 deleted and replaced by tablet 284e34970f634abcb2288c864f73e963
I20251211 20:52:40.816622 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 869) took 1179 ms. Trace:
I20251211 20:52:40.816632 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_d87c2c33f4ff4aee9ff82b0653d4cc43"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:40.816766 1163 rpcz_store.cc:276] 1211 20:52:39.637060 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.637147 (+ 87us) service_pool.cc:225] Handling call
1211 20:52:40.740363 (+1103216us) ts_tablet_manager.cc:1151] Deleting tablet bb0fc179a337469eae056c5bb3815e6b
1211 20:52:40.740371 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.769054 (+ 28683us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.814356 (+ 45302us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.816591 (+ 2235us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1103015}
I20251211 20:52:40.817564 881 tablet_service.cc:1552] Processing DeleteTablet for tablet a56f1a7bd5464d289bfd018c826ee574 with delete_type TABLET_DATA_DELETED (replaced by tablet 284e34970f634abcb2288c864f73e963) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:40.817646 1163 tablet_replica.cc:333] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:40.817811 1163 raft_consensus.cc:2243] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:40.817929 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet a56f1a7bd5464d289bfd018c826ee574 with delete_type TABLET_DATA_DELETED (replaced by tablet 284e34970f634abcb2288c864f73e963) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:40.818893 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d87c2c33f4ff4aee9ff82b0653d4cc43 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:40.819370 711 consensus_peers.cc:597] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:40.819626 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 84bb3583857848459e0e865283de623d consensus state has no leader
I20251211 20:52:40.819816 1166 tablet_bootstrap.cc:492] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:40.819965 1166 ts_tablet_manager.cc:1403] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.052s user 0.002s sys 0.002s
I20251211 20:52:40.820720 1166 raft_consensus.cc:359] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.820899 1166 raft_consensus.cc:385] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.820955 1166 raft_consensus.cc:740] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.821159 1166 consensus_queue.cc:260] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.818703 589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_6bc2fc890a7a485ba23c3214f772b758"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:40.757100 741 tablet_service.cc:1505] Processing CreateTablet for tablet 622e513e55d749f2a0105b5d6f9444c8 (DEFAULT_TABLE table=default.Table_ec0bf98cd1ce42ceacacde30208d5d04 [id=b8fdb6772ace427e8273983d1e791956]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.825048 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 622e513e55d749f2a0105b5d6f9444c8. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:40.826759 591 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet aa50ddb2789b4be3bfa9aedef9287109 consensus state has no leader
I20251211 20:52:40.689412 1168 ts_tablet_manager.cc:1434] T 22273287aa9245a2a66131fed2c8861c P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20251211 20:52:40.828040 1168 tablet_bootstrap.cc:492] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:40.829670 1168 tablet_bootstrap.cc:654] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.829973 1166 ts_tablet_manager.cc:1434] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.010s user 0.001s sys 0.000s
I20251211 20:52:40.830243 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_717a9e3f73be421585b11e3cdad48b64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:40.831082 589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6bc2fc890a7a485ba23c3214f772b758 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:40.831327 595 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 676faec7f3d1488f99b220f8e968e9a0 consensus state has no leader
I20251211 20:52:40.831362 588 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 2630ad57845a4c31b20c8a104dd27335
I20251211 20:52:40.831996 590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_087c001a257049babda598744c761be8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:40.834635 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_af66deaa476a4ea986b403ef120654ff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:40.835472 1160 tablet_bootstrap.cc:492] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:40.835645 1160 ts_tablet_manager.cc:1403] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.044s user 0.003s sys 0.000s
W20251211 20:52:40.835981 590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_087c001a257049babda598744c761be8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:40.836133 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_3db64a0e2af24640883c6413a68001b0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:40.836447 1160 raft_consensus.cc:359] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.836614 1160 raft_consensus.cc:385] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.836669 1160 raft_consensus.cc:740] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.836921 594 catalog_manager.cc:5936] Not sending DeleteTablet requests; no consensus state for tablet a739873b94364c6c9d96cb5a1aea0609
I20251211 20:52:40.817968 1163 raft_consensus.cc:2272] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.838721 1163 ts_tablet_manager.cc:1916] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.777961 747 tablet_service.cc:1505] Processing CreateTablet for tablet aa50ddb2789b4be3bfa9aedef9287109 (DEFAULT_TABLE table=default.Table_fbf522a147ec4a04950c849072ebb619 [id=d4e1349374a346a5b8be180f9bdc329a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:40.839849 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa50ddb2789b4be3bfa9aedef9287109. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:40.840320 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3db64a0e2af24640883c6413a68001b0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:40.840444 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet bb0fc179a337469eae056c5bb3815e6b (table default.Table_c27589191ac249f4a9460f5bbae2611b [id=461d63ce3b4f4d12b5f395bb8c1cdbb8]) successfully deleted
I20251211 20:52:40.841053 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_60f0bdc6a1c849fd851ca0d31eeeaf79"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:40.841954 596 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_717a9e3f73be421585b11e3cdad48b64 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:40.841362 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 2630ad57845a4c31b20c8a104dd27335 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:40.843961 1166 tablet_bootstrap.cc:492] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:40.837970 881 tablet_service.cc:1552] Processing DeleteTablet for tablet 2630ad57845a4c31b20c8a104dd27335 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:40.844951 1160 consensus_queue.cc:260] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.845489 1166 tablet_bootstrap.cc:654] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.846084 1160 ts_tablet_manager.cc:1434] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20251211 20:52:40.851728 1163 ts_tablet_manager.cc:1929] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:40.851939 1163 log.cc:1199] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/7ec261837cf04bf88a48ff8add1b0d66
I20251211 20:52:40.837921 2325 consensus_queue.cc:1048] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:40.852576 1163 ts_tablet_manager.cc:1950] T 7ec261837cf04bf88a48ff8add1b0d66 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:40.853288 591 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_bdb9aa9507d94416bbc3c8fe9cb11fa7" } modify_external_catalogs: true
I20251211 20:52:40.853497 591 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_bdb9aa9507d94416bbc3c8fe9cb11fa7" } modify_external_catalogs: true
I20251211 20:52:40.853564 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 868) took 1216 ms. Trace:
I20251211 20:52:40.853756 1163 rpcz_store.cc:276] 1211 20:52:39.636944 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.637109 (+ 165us) service_pool.cc:225] Handling call
1211 20:52:40.817585 (+1180476us) ts_tablet_manager.cc:1151] Deleting tablet 7ec261837cf04bf88a48ff8add1b0d66
1211 20:52:40.817605 (+ 20us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.848617 (+ 31012us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.851558 (+ 2941us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.853543 (+ 1985us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1151821}
I20251211 20:52:40.855077 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 7ec261837cf04bf88a48ff8add1b0d66 (table default.Table_e6e48a22fc3d43b69d3dd391b9d1c60b [id=1921595e79a7402dad44c6aee6417e78]) successfully deleted
I20251211 20:52:40.855334 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet bb4004212441473483a209d23c0800ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:40.855657 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_60f0bdc6a1c849fd851ca0d31eeeaf79 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:40.855780 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 871) took 1108 ms. Trace:
I20251211 20:52:40.855852 1163 rpcz_store.cc:276] 1211 20:52:39.747766 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.747823 (+ 57us) service_pool.cc:225] Handling call
1211 20:52:40.855651 (+1107828us) ts_tablet_manager.cc:1151] Deleting tablet 1944267def4e46949fd680ab0a369be9
1211 20:52:40.855663 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.855763 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1107597}
W20251211 20:52:40.856272 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 1944267def4e46949fd680ab0a369be9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1944267def4e46949fd680ab0a369be9 already in progress: creating tablet
I20251211 20:52:40.856622 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 872) took 1079 ms. Trace:
I20251211 20:52:40.856689 1163 rpcz_store.cc:276] 1211 20:52:39.777431 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.777484 (+ 53us) service_pool.cc:225] Handling call
1211 20:52:40.856508 (+1079024us) ts_tablet_manager.cc:1151] Deleting tablet 7d4e63f54d8443ae8322d4659a165e76
1211 20:52:40.856516 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.856607 (+ 91us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1078802}
W20251211 20:52:40.857107 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 7d4e63f54d8443ae8322d4659a165e76 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d4e63f54d8443ae8322d4659a165e76 already in progress: creating tablet
I20251211 20:52:40.857475 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 873) took 1075 ms. Trace:
I20251211 20:52:40.857576 1163 rpcz_store.cc:276] 1211 20:52:39.781658 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.781714 (+ 56us) service_pool.cc:225] Handling call
1211 20:52:40.857378 (+1075664us) ts_tablet_manager.cc:1151] Deleting tablet 0f8808bfd68841e4ae30818018040cf9
1211 20:52:40.857387 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.857461 (+ 74us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1075434}
W20251211 20:52:40.857985 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 0f8808bfd68841e4ae30818018040cf9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0f8808bfd68841e4ae30818018040cf9 already in progress: creating tablet
I20251211 20:52:40.858590 594 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f6f977edac0f45bea7e94de5bfa51ee4" } new_table_name: "default.Table_34e6ed4f13da437ea50d349b92e83bd6" modify_external_catalogs: true
I20251211 20:52:40.860152 1163 tablet_replica.cc:333] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:40.860410 1163 raft_consensus.cc:2243] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:40.860638 1163 raft_consensus.cc:2272] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.860893 1168 tablet_bootstrap.cc:492] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:40.861085 1168 ts_tablet_manager.cc:1403] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.002s
I20251211 20:52:40.861876 1168 raft_consensus.cc:359] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.862109 1168 raft_consensus.cc:385] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.862216 1168 raft_consensus.cc:740] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.862246 1163 ts_tablet_manager.cc:1916] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.862509 1168 consensus_queue.cc:260] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.862994 1168 ts_tablet_manager.cc:1434] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:40.863261 1168 tablet_bootstrap.cc:492] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:40.865012 1168 tablet_bootstrap.cc:654] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.865573 1166 tablet_bootstrap.cc:492] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:40.865708 1166 ts_tablet_manager.cc:1403] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20251211 20:52:40.866377 1166 raft_consensus.cc:359] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.866533 1166 raft_consensus.cc:385] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.866591 1166 raft_consensus.cc:740] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.866832 1166 consensus_queue.cc:260] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.867357 1166 ts_tablet_manager.cc:1434] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:40.867646 1166 tablet_bootstrap.cc:492] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:40.870682 592 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_af66deaa476a4ea986b403ef120654ff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:40.871258 1166 tablet_bootstrap.cc:654] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.880945 2135 raft_consensus.cc:493] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:40.881104 2135 raft_consensus.cc:515] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.881368 2322 raft_consensus.cc:493] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:40.881459 1163 ts_tablet_manager.cc:1929] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:40.881524 2322 raft_consensus.cc:515] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.881757 2135 leader_election.cc:290] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:40.882126 2322 leader_election.cc:290] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:40.881561 1163 log.cc:1199] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/4df68d1b6a91458e9f212ae81aa25438
I20251211 20:52:40.883675 1163 ts_tablet_manager.cc:1950] T 4df68d1b6a91458e9f212ae81aa25438 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:40.883826 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb4004212441473483a209d23c0800ce" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:40.884611 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 875) took 1096 ms. Trace:
W20251211 20:52:40.884629 847 leader_election.cc:343] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 68be21a25c6743f5b86d518d98fbc35e
I20251211 20:52:40.884827 1163 rpcz_store.cc:276] 1211 20:52:39.788627 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.788681 (+ 54us) service_pool.cc:225] Handling call
1211 20:52:40.858316 (+1069635us) ts_tablet_manager.cc:1151] Deleting tablet 4df68d1b6a91458e9f212ae81aa25438
1211 20:52:40.860106 (+ 1790us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.868523 (+ 8417us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.881344 (+ 12821us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.884594 (+ 3250us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1069393}
W20251211 20:52:40.884994 983 leader_election.cc:343] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:40.885288 847 leader_election.cc:343] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 68be21a25c6743f5b86d518d98fbc35e
I20251211 20:52:40.885391 847 leader_election.cc:304] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:40.885514 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 4df68d1b6a91458e9f212ae81aa25438 (table default.Table_f574e04a625b4f00bd2e2e8693a7f496 [id=e266c3e0cd804232bcecc4db812ca13b]) successfully deleted
I20251211 20:52:40.885844 1163 tablet_replica.cc:333] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:40.885900 2322 raft_consensus.cc:2749] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:40.886195 1163 raft_consensus.cc:2243] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:40.886387 1163 raft_consensus.cc:2272] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.886973 1163 ts_tablet_manager.cc:1916] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.889987 1165 ts_tablet_manager.cc:1929] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:40.890333 1165 log.cc:1199] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/bb112b6431af42a6a7b328dfdefbc2a2
I20251211 20:52:40.896487 1166 tablet_bootstrap.cc:492] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:40.896713 1166 ts_tablet_manager.cc:1403] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.003s
I20251211 20:52:40.897550 1166 raft_consensus.cc:359] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.897881 1166 raft_consensus.cc:385] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.898017 1166 raft_consensus.cc:740] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.898309 1166 consensus_queue.cc:260] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.898859 1166 ts_tablet_manager.cc:1434] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:40.899174 1166 tablet_bootstrap.cc:492] T 2e7e158809b9471fbd6e6ee9fb9ac569 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:40.900817 1166 tablet_bootstrap.cc:654] T 2e7e158809b9471fbd6e6ee9fb9ac569 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.891021 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb4004212441473483a209d23c0800ce" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:40.902346 1165 ts_tablet_manager.cc:1950] T bb112b6431af42a6a7b328dfdefbc2a2 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:40.903927 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_bae5e6fe430247c99eae40ae86192b39"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:40.904515 593 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bae5e6fe430247c99eae40ae86192b39 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:40.904631 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_1c2d691fc52b40dcb94acbe811aefe97" } modify_external_catalogs: true
I20251211 20:52:40.904783 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_1c2d691fc52b40dcb94acbe811aefe97" } modify_external_catalogs: true
I20251211 20:52:40.908026 2135 raft_consensus.cc:493] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:40.908182 2135 raft_consensus.cc:515] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:40.908371 982 leader_election.cc:343] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:40.908459 982 leader_election.cc:304] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:40.895521 1163 ts_tablet_manager.cc:1929] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:40.908669 2216 raft_consensus.cc:2749] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:40.908733 2135 leader_election.cc:290] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:40.908857 1163 log.cc:1199] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/7d4bba0aa7ea403e8dfcb74cec2e1f9b
I20251211 20:52:40.909173 1163 ts_tablet_manager.cc:1950] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:40.909273 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2207af105bc3471ca930d35a91d38e70" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:40.909778 983 leader_election.cc:343] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:40.910522 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 874) took 1122 ms. Trace:
I20251211 20:52:40.910614 1163 rpcz_store.cc:276] 1211 20:52:39.788485 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.788967 (+ 482us) service_pool.cc:225] Handling call
1211 20:52:40.885791 (+1096824us) ts_tablet_manager.cc:1151] Deleting tablet 7d4bba0aa7ea403e8dfcb74cec2e1f9b
1211 20:52:40.885802 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.892622 (+ 6820us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.895414 (+ 2792us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.910503 (+ 15089us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1096637}
I20251211 20:52:40.911875 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 7d4bba0aa7ea403e8dfcb74cec2e1f9b (table default.Table_6bfc3963d4e346858acdfd9b9016245d [id=4ba40e6e45c7463f9730a2380b5ff856]) successfully deleted
I20251211 20:52:40.912364 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 870) took 1195 ms. Trace:
I20251211 20:52:40.912523 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2207af105bc3471ca930d35a91d38e70" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:40.912791 1163 rpcz_store.cc:276] 1211 20:52:39.717090 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.825117 (+108027us) service_pool.cc:225] Handling call
1211 20:52:40.912236 (+1087119us) ts_tablet_manager.cc:1151] Deleting tablet 66cdc6f297b24cfba3a3a0ae9f45d12d
1211 20:52:40.912247 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.912347 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1085504}
W20251211 20:52:40.913085 982 leader_election.cc:343] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:40.913183 982 leader_election.cc:304] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:40.913319 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 66cdc6f297b24cfba3a3a0ae9f45d12d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66cdc6f297b24cfba3a3a0ae9f45d12d already in progress: creating tablet
I20251211 20:52:40.913383 2135 raft_consensus.cc:2749] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:40.914001 1163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47004 (request call id 876) took 1063 ms. Trace:
I20251211 20:52:40.914304 2135 raft_consensus.cc:493] T a9bb22dce13f4c9da76ca446715cd99a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:40.914415 2135 raft_consensus.cc:515] T a9bb22dce13f4c9da76ca446715cd99a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.914992 2135 leader_election.cc:290] T a9bb22dce13f4c9da76ca446715cd99a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:40.914088 1163 rpcz_store.cc:276] 1211 20:52:39.850169 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.850875 (+ 706us) service_pool.cc:225] Handling call
1211 20:52:40.913882 (+1063007us) ts_tablet_manager.cc:1151] Deleting tablet fbc2d4aea084492c8270ad1d87096e58
1211 20:52:40.913893 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.913985 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1062701}
W20251211 20:52:40.916134 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet fbc2d4aea084492c8270ad1d87096e58 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fbc2d4aea084492c8270ad1d87096e58 already in progress: creating tablet
I20251211 20:52:40.916290 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9bb22dce13f4c9da76ca446715cd99a" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:40.916870 983 leader_election.cc:343] T a9bb22dce13f4c9da76ca446715cd99a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:40.916887 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 4810f8d8b53a45eb939bf3c8bf1b4e77 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4810f8d8b53a45eb939bf3c8bf1b4e77 already in progress: creating tablet
W20251211 20:52:40.922701 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 0add14ccd2af42aeb02fad7a28ed4520 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0add14ccd2af42aeb02fad7a28ed4520 already in progress: creating tablet
I20251211 20:52:40.922899 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9bb22dce13f4c9da76ca446715cd99a" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:40.923367 982 leader_election.cc:343] T a9bb22dce13f4c9da76ca446715cd99a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:40.923399 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 877aa237fe43452cbdbb4dac46f613ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 877aa237fe43452cbdbb4dac46f613ae already in progress: creating tablet
I20251211 20:52:40.923480 982 leader_election.cc:304] T a9bb22dce13f4c9da76ca446715cd99a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:40.923682 1163 tablet_replica.cc:333] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:40.923888 2135 raft_consensus.cc:2749] T a9bb22dce13f4c9da76ca446715cd99a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:40.923887 1163 raft_consensus.cc:2243] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:40.923678 1166 tablet_bootstrap.cc:492] T 2e7e158809b9471fbd6e6ee9fb9ac569 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:40.924156 1163 raft_consensus.cc:2272] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.924239 1166 ts_tablet_manager.cc:1403] T 2e7e158809b9471fbd6e6ee9fb9ac569 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.004s
I20251211 20:52:40.924644 1163 ts_tablet_manager.cc:1916] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.924979 1166 raft_consensus.cc:359] T 2e7e158809b9471fbd6e6ee9fb9ac569 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.925139 1166 raft_consensus.cc:385] T 2e7e158809b9471fbd6e6ee9fb9ac569 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.925190 1166 raft_consensus.cc:740] T 2e7e158809b9471fbd6e6ee9fb9ac569 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.925370 1166 consensus_queue.cc:260] T 2e7e158809b9471fbd6e6ee9fb9ac569 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.925806 1166 ts_tablet_manager.cc:1434] T 2e7e158809b9471fbd6e6ee9fb9ac569 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:40.926437 1166 tablet_bootstrap.cc:492] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:40.930400 1166 tablet_bootstrap.cc:654] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.933375 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 850) took 2541 ms. Trace:
I20251211 20:52:40.933573 1165 rpcz_store.cc:276] 1211 20:52:38.392229 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.393022 (+ 793us) service_pool.cc:225] Handling call
1211 20:52:40.781245 (+2388223us) ts_tablet_manager.cc:1151] Deleting tablet bb112b6431af42a6a7b328dfdefbc2a2
1211 20:52:40.781255 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:40.823428 (+ 42173us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.889874 (+ 66446us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:40.933345 (+ 43471us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2388030}
I20251211 20:52:40.934039 1165 tablet_replica.cc:333] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:40.934475 1165 raft_consensus.cc:2243] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:40.936851 1163 ts_tablet_manager.cc:1929] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:40.936978 1163 log.cc:1199] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/2207af105bc3471ca930d35a91d38e70
I20251211 20:52:40.937367 1163 ts_tablet_manager.cc:1950] T 2207af105bc3471ca930d35a91d38e70 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:40.938422 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 2207af105bc3471ca930d35a91d38e70 (table default.Table_bd9633c55bd242c18aac3a7f2c18a329 [id=767f50bee5464316a18f73e64bab817a]) successfully deleted
W20251211 20:52:40.939196 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 757ec913dff840f992a41a38b05f8d41 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 757ec913dff840f992a41a38b05f8d41 already in progress: creating tablet
I20251211 20:52:40.939639 1163 tablet_replica.cc:333] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:40.939872 1163 raft_consensus.cc:2243] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:40.940081 1163 raft_consensus.cc:2272] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.940662 1163 ts_tablet_manager.cc:1916] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.942402 589 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet 8308ad46551447439a248d20ed1e5922
I20251211 20:52:40.943327 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 8308ad46551447439a248d20ed1e5922 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:47004
I20251211 20:52:40.943740 881 tablet_service.cc:1552] Processing DeleteTablet for tablet 8308ad46551447439a248d20ed1e5922 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:40.952086 1166 tablet_bootstrap.cc:492] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:40.952344 1166 ts_tablet_manager.cc:1403] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.026s user 0.001s sys 0.005s
I20251211 20:52:40.953125 1166 raft_consensus.cc:359] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.953375 1166 raft_consensus.cc:385] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.953475 1166 raft_consensus.cc:740] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.953747 1166 consensus_queue.cc:260] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.954291 1166 ts_tablet_manager.cc:1434] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:40.954582 1166 tablet_bootstrap.cc:492] T 3de5b34b96404cdc9c191fed7f055638 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:40.955168 1163 ts_tablet_manager.cc:1929] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:40.955289 1163 log.cc:1199] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/68792ccef54944a09af9b5456e9ff1fe
I20251211 20:52:40.957054 1163 ts_tablet_manager.cc:1950] T 68792ccef54944a09af9b5456e9ff1fe P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:40.958182 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 68792ccef54944a09af9b5456e9ff1fe (table default.Table_00bd643f7bb84217970178aefb485a69 [id=1b90cdd71d064d27948a1a9a7f3f852b]) successfully deleted
I20251211 20:52:40.958624 1163 tablet_replica.cc:333] T 4ce495233a5e4f3792faf44680df54ba P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:40.959769 1163 raft_consensus.cc:2243] T 4ce495233a5e4f3792faf44680df54ba P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:40.960000 1163 raft_consensus.cc:2272] T 4ce495233a5e4f3792faf44680df54ba P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.959545 1166 tablet_bootstrap.cc:654] T 3de5b34b96404cdc9c191fed7f055638 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.962132 1163 ts_tablet_manager.cc:1916] T 4ce495233a5e4f3792faf44680df54ba P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.944496 1168 tablet_bootstrap.cc:492] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:40.963932 1168 ts_tablet_manager.cc:1403] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.101s user 0.005s sys 0.000s
I20251211 20:52:40.964056 1165 pending_rounds.cc:70] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919: Trying to abort 1 pending ops.
I20251211 20:52:40.964102 1165 pending_rounds.cc:77] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432126888054784 op_type: NO_OP noop_request { }
I20251211 20:52:40.964200 1165 raft_consensus.cc:2889] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:40.964278 1165 raft_consensus.cc:2272] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.964709 1168 raft_consensus.cc:359] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.964843 1165 ts_tablet_manager.cc:1916] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.964851 1168 raft_consensus.cc:385] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:40.965123 1168 raft_consensus.cc:740] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:40.965304 1168 consensus_queue.cc:260] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.965760 1168 ts_tablet_manager.cc:1434] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:40.966041 1168 tablet_bootstrap.cc:492] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:40.971122 711 consensus_peers.cc:597] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:40.976130 1168 tablet_bootstrap.cc:654] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.977670 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet b631e6b3048e428f901af1b5b0b1ab3c (table default.Table_dd84a1d3fe964c76b146f798f6781baa [id=e80ce2c7f3aa48539b6d2fad357b3b49]) successfully deleted
I20251211 20:52:40.979203 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet bb112b6431af42a6a7b328dfdefbc2a2 (table default.Table_0c2f5186505f4a14b5c9bba07ca14178 [id=b4d34bd45a214d3b898045da7537ee10]) successfully deleted
I20251211 20:52:40.979982 2325 raft_consensus.cc:493] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:40.980123 2325 raft_consensus.cc:515] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:40.980717 2325 leader_election.cc:290] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:40.982260 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7d4bba0aa7ea403e8dfcb74cec2e1f9b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:40.983748 711 leader_election.cc:343] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:40.985250 710 consensus_peers.cc:597] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4f807596fa6a4e66b89566b5d5dd1080. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:40.985414 710 consensus_peers.cc:597] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e6ba12f94f5f4d688f4222a4c2fb5436. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:40.985549 710 leader_election.cc:343] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 7d4bba0aa7ea403e8dfcb74cec2e1f9b
I20251211 20:52:40.985646 710 leader_election.cc:304] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:40.986271 2286 raft_consensus.cc:2749] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:40.986840 1163 ts_tablet_manager.cc:1929] T 4ce495233a5e4f3792faf44680df54ba P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:40.986958 1163 log.cc:1199] T 4ce495233a5e4f3792faf44680df54ba P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/4ce495233a5e4f3792faf44680df54ba
I20251211 20:52:40.987248 1163 ts_tablet_manager.cc:1950] T 4ce495233a5e4f3792faf44680df54ba P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
W20251211 20:52:40.988529 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 79d92736bb1c46d2bfa5678b2d396ec2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79d92736bb1c46d2bfa5678b2d396ec2 already in progress: creating tablet
I20251211 20:52:40.989900 1164 tablet_replica.cc:333] T 648f43e95e034f2188a20e1b837f2818 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:40.990095 1164 raft_consensus.cc:2243] T 648f43e95e034f2188a20e1b837f2818 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:40.990254 1164 raft_consensus.cc:2272] T 648f43e95e034f2188a20e1b837f2818 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:40.990777 1164 ts_tablet_manager.cc:1916] T 648f43e95e034f2188a20e1b837f2818 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:40.995891 1160 tablet_bootstrap.cc:492] T 1a77e3e744ff42ccb68897704b4c2f1d P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:40.997627 1160 tablet_bootstrap.cc:654] T 1a77e3e744ff42ccb68897704b4c2f1d P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:40.999296 1168 tablet_bootstrap.cc:492] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:40.999454 1168 ts_tablet_manager.cc:1403] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.033s user 0.008s sys 0.000s
I20251211 20:52:41.000198 1168 raft_consensus.cc:359] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.000547 1168 raft_consensus.cc:385] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.000682 1168 raft_consensus.cc:740] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.000970 1168 consensus_queue.cc:260] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.001514 1168 ts_tablet_manager.cc:1434] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251211 20:52:41.002887 1164 ts_tablet_manager.cc:1929] T 648f43e95e034f2188a20e1b837f2818 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:41.003602 1164 log.cc:1199] T 648f43e95e034f2188a20e1b837f2818 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/648f43e95e034f2188a20e1b837f2818
I20251211 20:52:41.004065 1168 tablet_bootstrap.cc:492] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:41.004109 1164 ts_tablet_manager.cc:1950] T 648f43e95e034f2188a20e1b837f2818 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:41.005164 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 648f43e95e034f2188a20e1b837f2818 (table default.Table_fc80da3fac43460dabeea7e43bf098df [id=75a5a167f3f84710afbf197ea1eff3cc]) successfully deleted
I20251211 20:52:41.005838 1168 tablet_bootstrap.cc:654] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:41.006234 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 4776c456b4d74c90b1fc71d57b62a461 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4776c456b4d74c90b1fc71d57b62a461 already in progress: creating tablet
I20251211 20:52:41.007537 1164 tablet_replica.cc:333] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:41.008385 1164 raft_consensus.cc:2243] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:41.008803 1164 raft_consensus.cc:2272] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:41.009806 1166 tablet_bootstrap.cc:492] T 3de5b34b96404cdc9c191fed7f055638 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:41.010015 1166 ts_tablet_manager.cc:1403] T 3de5b34b96404cdc9c191fed7f055638 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.055s user 0.004s sys 0.003s
I20251211 20:52:41.010922 1166 raft_consensus.cc:359] T 3de5b34b96404cdc9c191fed7f055638 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.011163 1166 raft_consensus.cc:385] T 3de5b34b96404cdc9c191fed7f055638 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.011248 1166 raft_consensus.cc:740] T 3de5b34b96404cdc9c191fed7f055638 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.011518 1166 consensus_queue.cc:260] T 3de5b34b96404cdc9c191fed7f055638 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.012254 1166 ts_tablet_manager.cc:1434] T 3de5b34b96404cdc9c191fed7f055638 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:41.012333 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_87f092378445464ab48c155fae469f29" } modify_external_catalogs: true
I20251211 20:52:41.012542 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_87f092378445464ab48c155fae469f29" } modify_external_catalogs: true
I20251211 20:52:41.012615 1166 tablet_bootstrap.cc:492] T c7147571483e41ad970e573a08c368ca P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:41.013249 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_4ecbff2ab6c243f994ef4fae46276e92"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:41.013900 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4ecbff2ab6c243f994ef4fae46276e92 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:41.014353 592 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_c676a5b83a30404b9e0e9b189701845a" } new_table_name: "default.Table_92efd469dd324f7d886e2cbc90ece61c" modify_external_catalogs: true
I20251211 20:52:41.015153 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 4ce495233a5e4f3792faf44680df54ba (table default.Table_488913be2ade428593d6c0934e54a8f0 [id=f8adc6a8019d4b6b949eb34bde21031a]) successfully deleted
I20251211 20:52:41.015475 2322 raft_consensus.cc:493] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.015650 2322 raft_consensus.cc:515] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.016178 1163 tablet_replica.cc:333] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:41.016273 2322 leader_election.cc:290] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:41.016353 1163 raft_consensus.cc:2243] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:41.016510 1163 raft_consensus.cc:2272] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:41.017539 847 leader_election.cc:343] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 87863daf77304a399318cdf56da9f9bb
W20251211 20:52:41.017736 847 leader_election.cc:343] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 87863daf77304a399318cdf56da9f9bb
I20251211 20:52:41.017804 847 leader_election.cc:304] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:41.018077 1164 ts_tablet_manager.cc:1916] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:41.018151 2322 raft_consensus.cc:2749] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:41.021828 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_ca4526eee78f4811a98c7a64f76431d8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:41.022467 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ca4526eee78f4811a98c7a64f76431d8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:41.024114 1163 ts_tablet_manager.cc:1916] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:41.032248 1166 tablet_bootstrap.cc:654] T c7147571483e41ad970e573a08c368ca P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.039013 590 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f21e4bd4ff9846c8b37a5723d0fc3c3e" } new_table_name: "default.Table_e2af0e2fbd3d46d595136c7a7bf24e4c" modify_external_catalogs: true
I20251211 20:52:41.039224 1160 tablet_bootstrap.cc:492] T 1a77e3e744ff42ccb68897704b4c2f1d P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:41.039353 1160 ts_tablet_manager.cc:1403] T 1a77e3e744ff42ccb68897704b4c2f1d P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.044s user 0.003s sys 0.005s
I20251211 20:52:41.040148 1160 raft_consensus.cc:359] T 1a77e3e744ff42ccb68897704b4c2f1d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.040442 1160 raft_consensus.cc:385] T 1a77e3e744ff42ccb68897704b4c2f1d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.040532 1160 raft_consensus.cc:740] T 1a77e3e744ff42ccb68897704b4c2f1d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.040793 1160 consensus_queue.cc:260] T 1a77e3e744ff42ccb68897704b4c2f1d P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.044607 1168 tablet_bootstrap.cc:492] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:41.044839 1168 ts_tablet_manager.cc:1403] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.041s user 0.002s sys 0.003s
I20251211 20:52:41.046026 1160 ts_tablet_manager.cc:1434] T 1a77e3e744ff42ccb68897704b4c2f1d P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.007s user 0.000s sys 0.001s
I20251211 20:52:41.045708 1168 raft_consensus.cc:359] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.046226 1168 raft_consensus.cc:385] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.046320 1168 raft_consensus.cc:740] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.046557 1168 consensus_queue.cc:260] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.047969 1160 tablet_bootstrap.cc:492] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:41.049396 1168 ts_tablet_manager.cc:1434] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251211 20:52:41.049706 1168 tablet_bootstrap.cc:492] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:41.052906 757 tablet_service.cc:1552] Processing DeleteTablet for tablet a56f1a7bd5464d289bfd018c826ee574 with delete_type TABLET_DATA_DELETED (replaced by tablet 284e34970f634abcb2288c864f73e963) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:41.053586 1160 tablet_bootstrap.cc:654] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.053663 741 tablet_service.cc:1552] Processing DeleteTablet for tablet 4776c456b4d74c90b1fc71d57b62a461 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:41.054883 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 2630ad57845a4c31b20c8a104dd27335 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:37886
W20251211 20:52:41.055012 711 consensus_peers.cc:597] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:41.055088 739 tablet_service.cc:1552] Processing DeleteTablet for tablet 8308ad46551447439a248d20ed1e5922 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:41.057117 1166 tablet_bootstrap.cc:492] T c7147571483e41ad970e573a08c368ca P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:41.058264 1166 ts_tablet_manager.cc:1403] T c7147571483e41ad970e573a08c368ca P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.046s user 0.004s sys 0.000s
W20251211 20:52:41.059026 711 consensus_peers.cc:597] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:41.060113 1166 raft_consensus.cc:359] T c7147571483e41ad970e573a08c368ca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.061915 1166 raft_consensus.cc:385] T c7147571483e41ad970e573a08c368ca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.062057 1166 raft_consensus.cc:740] T c7147571483e41ad970e573a08c368ca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.062325 1166 consensus_queue.cc:260] T c7147571483e41ad970e573a08c368ca P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.063702 1166 ts_tablet_manager.cc:1434] T c7147571483e41ad970e573a08c368ca P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251211 20:52:41.065749 1164 ts_tablet_manager.cc:1929] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:41.066951 1168 tablet_bootstrap.cc:654] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.069901 1164 log.cc:1199] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/68be21a25c6743f5b86d518d98fbc35e
I20251211 20:52:41.070981 1164 ts_tablet_manager.cc:1950] T 68be21a25c6743f5b86d518d98fbc35e P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:41.071653 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 68be21a25c6743f5b86d518d98fbc35e (table default.Table_afd3c9ee35ad41caa1717a299443409e [id=b0ebaab8cd5a4deb9a5ea1e7bd3ccec3]) successfully deleted
I20251211 20:52:41.071732 1163 ts_tablet_manager.cc:1929] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:41.071834 1163 log.cc:1199] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/260bdeb73439437cbf234d20f7264ab3
I20251211 20:52:41.072149 1163 ts_tablet_manager.cc:1950] T 260bdeb73439437cbf234d20f7264ab3 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
W20251211 20:52:41.073741 582 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet a56f1a7bd5464d289bfd018c826ee574 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a56f1a7bd5464d289bfd018c826ee574 already in progress: creating tablet
I20251211 20:52:41.074676 1164 tablet_replica.cc:333] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:41.075228 1168 tablet_bootstrap.cc:492] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:41.075286 1164 raft_consensus.cc:2243] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:41.075493 1164 raft_consensus.cc:2272] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:41.076781 1168 ts_tablet_manager.cc:1403] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.002s
I20251211 20:52:41.077044 1164 ts_tablet_manager.cc:1916] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:41.077924 1168 raft_consensus.cc:359] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.079141 1166 tablet_bootstrap.cc:492] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:41.090655 1168 raft_consensus.cc:385] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.091552 1166 tablet_bootstrap.cc:654] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.093523 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 260bdeb73439437cbf234d20f7264ab3 (table default.Table_10e55a82b5d1413b8b68da9df264da06 [id=7c485abbfb12494c89c81b0b51965923]) successfully deleted
W20251211 20:52:41.094405 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 79d92736bb1c46d2bfa5678b2d396ec2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79d92736bb1c46d2bfa5678b2d396ec2 already in progress: creating tablet
I20251211 20:52:41.094709 1168 raft_consensus.cc:740] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.094995 1164 ts_tablet_manager.cc:1929] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20251211 20:52:41.095094 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 4776c456b4d74c90b1fc71d57b62a461 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4776c456b4d74c90b1fc71d57b62a461 already in progress: creating tablet
I20251211 20:52:41.095238 1168 consensus_queue.cc:260] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:41.095780 581 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet a56f1a7bd5464d289bfd018c826ee574 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a56f1a7bd5464d289bfd018c826ee574 already in progress: creating tablet
I20251211 20:52:41.096097 1163 tablet_replica.cc:333] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:41.096098 1168 ts_tablet_manager.cc:1434] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.019s user 0.001s sys 0.001s
I20251211 20:52:41.096344 1163 raft_consensus.cc:2243] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:41.096868 1163 raft_consensus.cc:2272] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:41.097509 1163 ts_tablet_manager.cc:1916] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:41.098193 1164 log.cc:1199] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/2630ad57845a4c31b20c8a104dd27335
I20251211 20:52:41.098793 1164 ts_tablet_manager.cc:1950] T 2630ad57845a4c31b20c8a104dd27335 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:41.100255 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 2630ad57845a4c31b20c8a104dd27335 (table default.Table_4d70f35e05524680b3d72b2e488adbd5 [id=61b2de16fee74487823292913de1b685]) successfully deleted
I20251211 20:52:41.101052 1164 tablet_replica.cc:333] T 8308ad46551447439a248d20ed1e5922 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:41.101487 1164 raft_consensus.cc:2243] T 8308ad46551447439a248d20ed1e5922 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:41.101953 1164 raft_consensus.cc:2272] T 8308ad46551447439a248d20ed1e5922 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:41.103885 1164 ts_tablet_manager.cc:1916] T 8308ad46551447439a248d20ed1e5922 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:41.106056 1166 tablet_bootstrap.cc:492] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:41.106276 1166 ts_tablet_manager.cc:1403] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.027s user 0.005s sys 0.000s
I20251211 20:52:41.107050 1166 raft_consensus.cc:359] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.108045 1166 raft_consensus.cc:385] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.108161 1166 raft_consensus.cc:740] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.108476 1166 consensus_queue.cc:260] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.114971 2135 raft_consensus.cc:493] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.115150 2135 raft_consensus.cc:515] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.115849 2135 leader_election.cc:290] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:41.116765 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5710fc2f24764f818f7e5c86c6fd985e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:41.117434 983 leader_election.cc:343] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.118408 1160 tablet_bootstrap.cc:492] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:41.118603 1160 ts_tablet_manager.cc:1403] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.071s user 0.006s sys 0.009s
I20251211 20:52:41.119410 1160 raft_consensus.cc:359] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.119647 1160 raft_consensus.cc:385] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.119740 1160 raft_consensus.cc:740] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.119911 1166 ts_tablet_manager.cc:1434] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
I20251211 20:52:41.120012 1160 consensus_queue.cc:260] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.120504 1160 ts_tablet_manager.cc:1434] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:41.120836 1160 tablet_bootstrap.cc:492] T 8229b704e7464b30b7d2902a938ce33b P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:41.122395 1160 tablet_bootstrap.cc:654] T 8229b704e7464b30b7d2902a938ce33b P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.125597 589 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_b158d0ce6ac74ce9a3a5f207a42ec4f3" } new_table_name: "default.Table_f0a70914acc94231a4e84155c3e4d13f" modify_external_catalogs: true
I20251211 20:52:41.129747 1164 ts_tablet_manager.cc:1929] T 8308ad46551447439a248d20ed1e5922 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251211 20:52:41.129877 1164 log.cc:1199] T 8308ad46551447439a248d20ed1e5922 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/8308ad46551447439a248d20ed1e5922
W20251211 20:52:41.131534 711 consensus_peers.cc:597] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:41.136677 881 tablet_service.cc:1552] Processing DeleteTablet for tablet 68be21a25c6743f5b86d518d98fbc35e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:41.136936 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 6c64fd8dd49b4e5d91c1ffb0fc2559e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:41.136677 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a526a6d1bc84cefad2c5d897223292e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:41.137890 1166 tablet_bootstrap.cc:492] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:41.140715 1166 tablet_bootstrap.cc:654] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.141115 2352 tablet.cc:1722] T cf1201bba4b6421f8d7ce593f4bd892b P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:41.144300 1163 ts_tablet_manager.cc:1929] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251211 20:52:41.144459 1163 log.cc:1199] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/2630ad57845a4c31b20c8a104dd27335
I20251211 20:52:41.144817 1163 ts_tablet_manager.cc:1950] T 2630ad57845a4c31b20c8a104dd27335 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:41.146066 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 2630ad57845a4c31b20c8a104dd27335 (table default.Table_4d70f35e05524680b3d72b2e488adbd5 [id=61b2de16fee74487823292913de1b685]) successfully deleted
I20251211 20:52:41.147173 1160 tablet_bootstrap.cc:492] T 8229b704e7464b30b7d2902a938ce33b P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:41.147387 1160 ts_tablet_manager.cc:1403] T 8229b704e7464b30b7d2902a938ce33b P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.001s
I20251211 20:52:41.148124 1160 raft_consensus.cc:359] T 8229b704e7464b30b7d2902a938ce33b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.148360 1160 raft_consensus.cc:385] T 8229b704e7464b30b7d2902a938ce33b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.148440 1160 raft_consensus.cc:740] T 8229b704e7464b30b7d2902a938ce33b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.148685 1160 consensus_queue.cc:260] T 8229b704e7464b30b7d2902a938ce33b P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.151731 1164 ts_tablet_manager.cc:1950] T 8308ad46551447439a248d20ed1e5922 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
W20251211 20:52:41.152667 590 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 78f77d8916d84edabbe95ce17d8df2c1 consensus state has no leader
I20251211 20:52:41.152758 1160 ts_tablet_manager.cc:1434] T 8229b704e7464b30b7d2902a938ce33b P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20251211 20:52:41.153045 1160 tablet_bootstrap.cc:492] T c7a2610f20df4c1ca81eae2fc7dba6bb P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:41.153551 1164 tablet_replica.cc:333] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:41.153728 1164 raft_consensus.cc:2243] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:41.153885 1164 raft_consensus.cc:2272] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:41.155457 597 master_service.cc:954] ReplaceTablet: received request to replace tablet c8e17645a26947ab8c01f9b093fa422c from {username='slave'} at 127.0.0.1:44772
I20251211 20:52:41.156562 1164 ts_tablet_manager.cc:1916] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:41.156877 1160 tablet_bootstrap.cc:654] T c7a2610f20df4c1ca81eae2fc7dba6bb P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.157205 588 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_2282dc6429c345098eeaf08b63262e8b" } new_table_name: "default.Table_e70873b71ba24d7eae548e2acfd243b3" modify_external_catalogs: true
W20251211 20:52:41.159631 592 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3003ef71a3554c71a3b24cdfb9c293bc consensus state has no leader
I20251211 20:52:41.160006 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a5b944a25f4c45cf8c70765ea43d1ac2" } modify_external_catalogs: true
I20251211 20:52:41.160183 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_a5b944a25f4c45cf8c70765ea43d1ac2" } modify_external_catalogs: true
I20251211 20:52:41.161211 1163 tablet_replica.cc:333] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:41.161871 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 8308ad46551447439a248d20ed1e5922 (table default.Table_1c2d691fc52b40dcb94acbe811aefe97 [id=17b2359416c04980857565ab4df594bd]) successfully deleted
W20251211 20:52:41.162178 582 catalog_manager.cc:4952] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 68be21a25c6743f5b86d518d98fbc35e because the tablet was not found. No further retry: Not found: Tablet not found: 68be21a25c6743f5b86d518d98fbc35e
I20251211 20:52:41.162477 1163 raft_consensus.cc:2243] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:41.163151 1163 raft_consensus.cc:2272] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:41.163414 589 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7ca98b43863a47c091baae0a6ce65630 consensus state has no leader
I20251211 20:52:41.163880 589 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_52e650ce0b824a41a39fcd4f1b8ba455" } modify_external_catalogs: true
I20251211 20:52:41.164196 1166 tablet_bootstrap.cc:492] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:41.164170 589 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_52e650ce0b824a41a39fcd4f1b8ba455" } modify_external_catalogs: true
I20251211 20:52:41.164433 1166 ts_tablet_manager.cc:1403] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.027s user 0.003s sys 0.002s
I20251211 20:52:41.165273 1166 raft_consensus.cc:359] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.165518 1166 raft_consensus.cc:385] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.165637 1166 raft_consensus.cc:740] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.165926 1166 consensus_queue.cc:260] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.167294 1164 ts_tablet_manager.cc:1929] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:41.173923 1166 ts_tablet_manager.cc:1434] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20251211 20:52:41.166754 1163 ts_tablet_manager.cc:1916] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:41.182348 597 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Sending DeleteTablet for 3 replicas of tablet c8e17645a26947ab8c01f9b093fa422c
I20251211 20:52:41.182929 597 catalog_manager.cc:6724] ReplaceTablet: tablet c8e17645a26947ab8c01f9b093fa422c deleted and replaced by tablet 171d40a8a9124f9bb1b38797fef8e5d5
I20251211 20:52:41.183167 739 tablet_service.cc:1552] Processing DeleteTablet for tablet c8e17645a26947ab8c01f9b093fa422c with delete_type TABLET_DATA_DELETED (replaced by tablet 171d40a8a9124f9bb1b38797fef8e5d5) from {username='slave'} at 127.0.0.1:37886
I20251211 20:52:41.183292 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_ebf689834b574c408bb00777fac24c84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:41.183779 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet c8e17645a26947ab8c01f9b093fa422c with delete_type TABLET_DATA_DELETED (replaced by tablet 171d40a8a9124f9bb1b38797fef8e5d5) from {username='slave'} at 127.0.0.1:47004
W20251211 20:52:41.183897 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ebf689834b574c408bb00777fac24c84 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:41.184669 1166 tablet_bootstrap.cc:492] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:41.184652 1164 log.cc:1199] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/6c64fd8dd49b4e5d91c1ffb0fc2559e0
I20251211 20:52:41.185009 1164 ts_tablet_manager.cc:1950] T 6c64fd8dd49b4e5d91c1ffb0fc2559e0 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
W20251211 20:52:41.185647 588 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1f4eb1a174c648c897ae2b7612e44585 consensus state has no leader
I20251211 20:52:41.186055 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b8e180cc27234500a1c4c7bdd2b93ac8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:41.186601 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b8e180cc27234500a1c4c7bdd2b93ac8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:41.187115 1164 tablet_replica.cc:333] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:41.187320 1166 tablet_bootstrap.cc:654] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.188818 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 6c64fd8dd49b4e5d91c1ffb0fc2559e0 (table default.Table_cc48404bbbf24a7caae7641a8d42c247 [id=048a032339b3462ba676bcb44f4e2082]) successfully deleted
I20251211 20:52:41.188320 894 tablet_service.cc:1552] Processing DeleteTablet for tablet c8e17645a26947ab8c01f9b093fa422c with delete_type TABLET_DATA_DELETED (replaced by tablet 171d40a8a9124f9bb1b38797fef8e5d5) from {username='slave'} at 127.0.0.1:43888
I20251211 20:52:41.188449 1160 tablet_bootstrap.cc:492] T c7a2610f20df4c1ca81eae2fc7dba6bb P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:41.189188 1160 ts_tablet_manager.cc:1403] T c7a2610f20df4c1ca81eae2fc7dba6bb P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.036s user 0.003s sys 0.002s
I20251211 20:52:41.189369 1164 raft_consensus.cc:2243] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:41.189528 1164 raft_consensus.cc:2272] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:41.190212 1160 raft_consensus.cc:359] T c7a2610f20df4c1ca81eae2fc7dba6bb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.190428 1160 raft_consensus.cc:385] T c7a2610f20df4c1ca81eae2fc7dba6bb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.190511 1160 raft_consensus.cc:740] T c7a2610f20df4c1ca81eae2fc7dba6bb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.190888 1160 consensus_queue.cc:260] T c7a2610f20df4c1ca81eae2fc7dba6bb P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.195485 1160 ts_tablet_manager.cc:1434] T c7a2610f20df4c1ca81eae2fc7dba6bb P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20251211 20:52:41.195498 2135 raft_consensus.cc:493] T c7147571483e41ad970e573a08c368ca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.195704 2135 raft_consensus.cc:515] T c7147571483e41ad970e573a08c368ca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.196348 2135 leader_election.cc:290] T c7147571483e41ad970e573a08c368ca P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:41.197255 1160 tablet_bootstrap.cc:492] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:41.197705 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7147571483e41ad970e573a08c368ca" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:41.198244 983 leader_election.cc:343] T c7147571483e41ad970e573a08c368ca P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.200785 1160 tablet_bootstrap.cc:654] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.201329 1164 ts_tablet_manager.cc:1916] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:41.207146 1163 ts_tablet_manager.cc:1929] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:41.208135 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_e3de355d0a69496c842b4a35daa99b90"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:41.208806 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e3de355d0a69496c842b4a35daa99b90 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:41.209870 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_b1d96cf6ac344427a92df55d1dfcee90"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:41.210422 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b1d96cf6ac344427a92df55d1dfcee90 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:41.211827 1164 ts_tablet_manager.cc:1929] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:41.223971 1163 log.cc:1199] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/bb4004212441473483a209d23c0800ce
I20251211 20:52:41.224946 1163 ts_tablet_manager.cc:1950] T bb4004212441473483a209d23c0800ce P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:41.226082 1164 log.cc:1199] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/6a526a6d1bc84cefad2c5d897223292e
I20251211 20:52:41.226503 1164 ts_tablet_manager.cc:1950] T 6a526a6d1bc84cefad2c5d897223292e P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:41.227945 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet bb4004212441473483a209d23c0800ce (table default.Table_55b2abeacec041418b6359c21e87b3d8 [id=75609081b0c7488eb388faac157db3e5]) successfully deleted
I20251211 20:52:41.226883 1160 tablet_bootstrap.cc:492] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:41.228199 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 6a526a6d1bc84cefad2c5d897223292e (table default.Table_254d66657cb14d0f8bbf713f794806f6 [id=f96a54c77528480ebf36ecc5741f7e17]) successfully deleted
I20251211 20:52:41.228559 1164 tablet_replica.cc:333] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:41.228739 1163 tablet_replica.cc:333] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:41.229079 1163 raft_consensus.cc:2243] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:41.229360 1163 raft_consensus.cc:2272] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:41.229918 1164 raft_consensus.cc:2243] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:41.229988 1163 ts_tablet_manager.cc:1916] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:41.228585 1160 ts_tablet_manager.cc:1403] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.031s user 0.001s sys 0.005s
I20251211 20:52:41.230165 1164 raft_consensus.cc:2272] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:41.231738 1164 ts_tablet_manager.cc:1916] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:41.230769 1160 raft_consensus.cc:359] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.233384 1160 raft_consensus.cc:385] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.233486 1160 raft_consensus.cc:740] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.233781 1160 consensus_queue.cc:260] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.239048 1160 ts_tablet_manager.cc:1434] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20251211 20:52:41.239130 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_06dc3002fa354d78ab4d5a90fd0fb3ab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:41.240103 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_06dc3002fa354d78ab4d5a90fd0fb3ab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:41.232272 1166 tablet_bootstrap.cc:492] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:41.240906 1166 ts_tablet_manager.cc:1403] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.056s user 0.005s sys 0.001s
I20251211 20:52:41.242038 1166 raft_consensus.cc:359] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.242329 1166 raft_consensus.cc:385] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.242455 1166 raft_consensus.cc:740] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.242908 1166 consensus_queue.cc:260] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.249127 1166 ts_tablet_manager.cc:1434] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
W20251211 20:52:41.249161 710 consensus_peers.cc:597] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 14e2650f8ec1482ea9511b989fdc963c. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:41.249686 1166 tablet_bootstrap.cc:492] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:41.237133 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_fadb0061961d4706af19ddfb2f79d521"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:41.252988 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fadb0061961d4706af19ddfb2f79d521 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:41.241592 2322 raft_consensus.cc:493] T c7bd218b3ff340b2a589c9517b49a649 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.253849 2322 raft_consensus.cc:515] T c7bd218b3ff340b2a589c9517b49a649 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.254523 2322 leader_election.cc:290] T c7bd218b3ff340b2a589c9517b49a649 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:41.255551 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7bd218b3ff340b2a589c9517b49a649" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:41.255963 1164 ts_tablet_manager.cc:1929] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251211 20:52:41.257897 1164 log.cc:1199] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/c8e17645a26947ab8c01f9b093fa422c
I20251211 20:52:41.258359 1164 ts_tablet_manager.cc:1950] T c8e17645a26947ab8c01f9b093fa422c P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:41.262257 588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_440d3f9c240c448bb5bf5bfbdaf35d00"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:41.262959 1163 ts_tablet_manager.cc:1929] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251211 20:52:41.263082 1163 log.cc:1199] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/8308ad46551447439a248d20ed1e5922
W20251211 20:52:41.263159 588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_440d3f9c240c448bb5bf5bfbdaf35d00 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:41.263482 1166 tablet_bootstrap.cc:654] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.263521 1163 ts_tablet_manager.cc:1950] T 8308ad46551447439a248d20ed1e5922 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:41.266016 1163 tablet_replica.cc:333] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:41.266283 1163 raft_consensus.cc:2243] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:41.266498 1163 raft_consensus.cc:2272] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:41.266995 1163 ts_tablet_manager.cc:1916] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:41.273495 1163 ts_tablet_manager.cc:1929] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251211 20:52:41.273777 1163 log.cc:1199] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/c8e17645a26947ab8c01f9b093fa422c
I20251211 20:52:41.274255 1163 ts_tablet_manager.cc:1950] T c8e17645a26947ab8c01f9b093fa422c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:41.283982 2353 tablet.cc:1722] T cf1201bba4b6421f8d7ce593f4bd892b P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:41.289842 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7147571483e41ad970e573a08c368ca" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:41.290253 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7bd218b3ff340b2a589c9517b49a649" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:41.290560 982 leader_election.cc:343] T c7147571483e41ad970e573a08c368ca P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.290716 982 leader_election.cc:304] T c7147571483e41ad970e573a08c368ca P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:41.290827 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5710fc2f24764f818f7e5c86c6fd985e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:41.291148 2135 raft_consensus.cc:2749] T c7147571483e41ad970e573a08c368ca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:41.291626 847 leader_election.cc:304] T c7bd218b3ff340b2a589c9517b49a649 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:41.292040 2322 raft_consensus.cc:2749] T c7bd218b3ff340b2a589c9517b49a649 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:41.292507 982 leader_election.cc:343] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.292632 982 leader_election.cc:304] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:41.293046 2135 raft_consensus.cc:2749] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:41.303759 2338 consensus_queue.cc:1048] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:41.303893 1168 tablet_bootstrap.cc:492] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:41.313894 711 consensus_peers.cc:597] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:41.315141 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44772:
name: "default.Table_cd866c64dfc7477782129c897b849771"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:41.315788 597 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cd866c64dfc7477782129c897b849771 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:41.320196 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 8308ad46551447439a248d20ed1e5922 (table default.Table_1c2d691fc52b40dcb94acbe811aefe97 [id=17b2359416c04980857565ab4df594bd]) successfully deleted
I20251211 20:52:41.320403 581 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet c8e17645a26947ab8c01f9b093fa422c (table default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5]) successfully deleted
I20251211 20:52:41.322300 2135 raft_consensus.cc:493] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.322480 2135 raft_consensus.cc:515] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.323133 2135 leader_election.cc:290] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:41.323941 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28ddb8a59a214576a25d613d42c2fef5" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:41.324563 588 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f84cd3de863b446894133bb40aaf1a39" } modify_external_catalogs: true
W20251211 20:52:41.324661 983 leader_election.cc:343] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.325366 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28ddb8a59a214576a25d613d42c2fef5" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:41.325513 588 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44772:
table { table_name: "default.Table_f84cd3de863b446894133bb40aaf1a39" } modify_external_catalogs: true
W20251211 20:52:41.326028 982 leader_election.cc:343] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.326157 982 leader_election.cc:304] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:41.326647 2135 raft_consensus.cc:2749] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:41.328197 1168 tablet_bootstrap.cc:654] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.328600 1166 tablet_bootstrap.cc:492] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:41.328831 1166 ts_tablet_manager.cc:1403] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.079s user 0.025s sys 0.037s
I20251211 20:52:41.329418 1160 tablet_bootstrap.cc:492] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:41.329653 582 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet c8e17645a26947ab8c01f9b093fa422c (table default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5]) successfully deleted
I20251211 20:52:41.330518 1166 raft_consensus.cc:359] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.330789 1166 raft_consensus.cc:385] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.330912 1166 raft_consensus.cc:740] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.331092 1160 tablet_bootstrap.cc:654] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.331269 1166 consensus_queue.cc:260] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.331976 1166 ts_tablet_manager.cc:1434] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:41.332304 1166 tablet_bootstrap.cc:492] T 50f9f29c064c4f938a0f20f302ab5e8a P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:41.334028 1166 tablet_bootstrap.cc:654] T 50f9f29c064c4f938a0f20f302ab5e8a P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.334724 2322 raft_consensus.cc:493] T c7a2610f20df4c1ca81eae2fc7dba6bb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.334887 2322 raft_consensus.cc:515] T c7a2610f20df4c1ca81eae2fc7dba6bb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.335493 2322 leader_election.cc:290] T c7a2610f20df4c1ca81eae2fc7dba6bb P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:41.337818 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7a2610f20df4c1ca81eae2fc7dba6bb" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:41.342283 1166 tablet_bootstrap.cc:492] T 50f9f29c064c4f938a0f20f302ab5e8a P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:41.342514 1166 ts_tablet_manager.cc:1403] T 50f9f29c064c4f938a0f20f302ab5e8a P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.003s
I20251211 20:52:41.343362 1166 raft_consensus.cc:359] T 50f9f29c064c4f938a0f20f302ab5e8a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.343647 1166 raft_consensus.cc:385] T 50f9f29c064c4f938a0f20f302ab5e8a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.343820 1166 raft_consensus.cc:740] T 50f9f29c064c4f938a0f20f302ab5e8a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.328316 1165 ts_tablet_manager.cc:1929] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:41.336231 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7a2610f20df4c1ca81eae2fc7dba6bb" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:41.344172 1166 consensus_queue.cc:260] T 50f9f29c064c4f938a0f20f302ab5e8a P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.344568 1165 log.cc:1199] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/a2058b67ddf1413db338bfa2f64e7a77
I20251211 20:52:41.344764 1166 ts_tablet_manager.cc:1434] T 50f9f29c064c4f938a0f20f302ab5e8a P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:41.344920 847 leader_election.cc:304] T c7a2610f20df4c1ca81eae2fc7dba6bb P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:41.345181 1166 tablet_bootstrap.cc:492] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:41.345289 1165 ts_tablet_manager.cc:1950] T a2058b67ddf1413db338bfa2f64e7a77 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:41.345328 2322 raft_consensus.cc:2749] T c7a2610f20df4c1ca81eae2fc7dba6bb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:41.346081 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 851) took 2953 ms. Trace:
I20251211 20:52:41.346181 1165 rpcz_store.cc:276] 1211 20:52:38.392326 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.393244 (+ 918us) service_pool.cc:225] Handling call
1211 20:52:40.933984 (+2540740us) ts_tablet_manager.cc:1151] Deleting tablet a2058b67ddf1413db338bfa2f64e7a77
1211 20:52:40.933997 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:41.101723 (+167726us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:41.328200 (+226477us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:41.346056 (+ 17856us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2540561}
I20251211 20:52:41.346675 581 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet a2058b67ddf1413db338bfa2f64e7a77 (table default.Table_91453272b5a341baab304eedbc5b2b34 [id=e1b92959cd2f46469c90526286f30b27]) successfully deleted
I20251211 20:52:41.346884 1166 tablet_bootstrap.cc:654] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.349167 1166 tablet_bootstrap.cc:492] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:41.349380 1166 ts_tablet_manager.cc:1403] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251211 20:52:41.350183 1166 raft_consensus.cc:359] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.350471 1166 raft_consensus.cc:385] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.350579 1166 raft_consensus.cc:740] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.350895 1166 consensus_queue.cc:260] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.351544 1166 ts_tablet_manager.cc:1434] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:41.351998 1166 tablet_bootstrap.cc:492] T 5262c2241e524395a73c0eb368917169 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:41.353195 710 consensus_peers.cc:597] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8308ad46551447439a248d20ed1e5922. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:41.354300 1166 tablet_bootstrap.cc:654] T 5262c2241e524395a73c0eb368917169 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.355871 1168 tablet_bootstrap.cc:492] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:41.356060 1168 ts_tablet_manager.cc:1403] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.052s user 0.004s sys 0.000s
I20251211 20:52:41.356844 1168 raft_consensus.cc:359] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.357089 1168 raft_consensus.cc:385] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.357185 1168 raft_consensus.cc:740] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.357451 1168 consensus_queue.cc:260] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.358206 1168 ts_tablet_manager.cc:1434] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:41.358520 1168 tablet_bootstrap.cc:492] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:41.363401 1168 tablet_bootstrap.cc:654] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.364125 2338 raft_consensus.cc:493] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.364267 2338 raft_consensus.cc:515] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.364902 2338 leader_election.cc:290] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:41.366313 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "676faec7f3d1488f99b220f8e968e9a0" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:41.366611 1042 raft_consensus.cc:2468] T 676faec7f3d1488f99b220f8e968e9a0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:41.367129 710 leader_election.cc:304] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:41.367514 2338 raft_consensus.cc:2804] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:41.367630 2338 raft_consensus.cc:493] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:41.367697 2338 raft_consensus.cc:3060] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:41.370628 2338 raft_consensus.cc:515] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.371927 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "676faec7f3d1488f99b220f8e968e9a0" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:41.372665 711 leader_election.cc:343] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.373634 2338 leader_election.cc:290] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:41.373744 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "676faec7f3d1488f99b220f8e968e9a0" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:41.373960 1048 raft_consensus.cc:3060] T 676faec7f3d1488f99b220f8e968e9a0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:41.375766 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "676faec7f3d1488f99b220f8e968e9a0" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:41.376421 711 leader_election.cc:343] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.376605 1048 raft_consensus.cc:2468] T 676faec7f3d1488f99b220f8e968e9a0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:41.377046 710 leader_election.cc:304] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:41.377485 2338 raft_consensus.cc:2804] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:41.377730 2338 raft_consensus.cc:697] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:41.378026 2338 consensus_queue.cc:237] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.379163 1160 tablet_bootstrap.cc:492] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:41.379315 1160 ts_tablet_manager.cc:1403] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.050s user 0.000s sys 0.009s
I20251211 20:52:41.380071 1160 raft_consensus.cc:359] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.380239 1160 raft_consensus.cc:385] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.380286 1160 raft_consensus.cc:740] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.380508 1160 consensus_queue.cc:260] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.380980 1160 ts_tablet_manager.cc:1434] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:41.403270 2135 raft_consensus.cc:493] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.403496 2135 raft_consensus.cc:515] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.404253 2135 leader_election.cc:290] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:41.404778 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f5916a4d9b24d7b8db3fcf028bbe3c9" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:41.412773 416 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-task5TWsQc/build/asan/bin/kudu pid 564
I20251211 20:52:41.413780 627 generic_service.cc:196] Checking for leaks (request via RPC)
I20251211 20:52:41.414722 2371 raft_consensus.cc:493] T 1a77e3e744ff42ccb68897704b4c2f1d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.415856 1165 tablet_replica.cc:333] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:41.416082 1165 raft_consensus.cc:2243] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:41.416252 1165 raft_consensus.cc:2272] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:41.415727 2371 raft_consensus.cc:515] T 1a77e3e744ff42ccb68897704b4c2f1d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.417435 1165 ts_tablet_manager.cc:1916] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:41.416617 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f5916a4d9b24d7b8db3fcf028bbe3c9" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:41.419116 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a77e3e744ff42ccb68897704b4c2f1d" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:41.419701 2371 leader_election.cc:290] T 1a77e3e744ff42ccb68897704b4c2f1d P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:41.421846 983 leader_election.cc:343] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.427910 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a77e3e744ff42ccb68897704b4c2f1d" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:41.432052 914 raft_consensus.cc:3060] T c7bd218b3ff340b2a589c9517b49a649 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:41.435016 2325 raft_consensus.cc:493] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.435169 2325 raft_consensus.cc:515] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.435801 2325 leader_election.cc:290] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:41.436635 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b793241ee3b42b1a38e9f2f05e1173b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:41.436846 1048 raft_consensus.cc:2468] T 8b793241ee3b42b1a38e9f2f05e1173b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:41.436830 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b793241ee3b42b1a38e9f2f05e1173b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:41.437423 710 leader_election.cc:304] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:41.437784 1160 tablet_bootstrap.cc:492] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:41.440207 1160 tablet_bootstrap.cc:654] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.442539 2361 tablet.cc:1722] T cf1201bba4b6421f8d7ce593f4bd892b P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:41.460528 2325 raft_consensus.cc:2804] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:41.460652 2325 raft_consensus.cc:493] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20251211 20:52:41.461763 711 leader_election.cc:343] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.465890 847 leader_election.cc:304] T 1a77e3e744ff42ccb68897704b4c2f1d P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:41.466382 2322 raft_consensus.cc:2749] T 1a77e3e744ff42ccb68897704b4c2f1d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:41.467312 2325 raft_consensus.cc:3060] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:41.470014 2325 raft_consensus.cc:515] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.470654 2325 leader_election.cc:290] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:41.471391 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b793241ee3b42b1a38e9f2f05e1173b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:41.471693 1045 raft_consensus.cc:3060] T 8b793241ee3b42b1a38e9f2f05e1173b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:41.474273 1045 raft_consensus.cc:2468] T 8b793241ee3b42b1a38e9f2f05e1173b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:41.475991 1160 tablet_bootstrap.cc:492] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:41.483758 1160 ts_tablet_manager.cc:1403] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.046s user 0.002s sys 0.003s
I20251211 20:52:41.484606 1160 raft_consensus.cc:359] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.484885 1160 raft_consensus.cc:385] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.484974 1160 raft_consensus.cc:740] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.485241 1160 consensus_queue.cc:260] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:41.485878 711 consensus_peers.cc:597] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2630ad57845a4c31b20c8a104dd27335. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:41.496094 710 leader_election.cc:304] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:41.480018 2375 raft_consensus.cc:493] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.488302 1166 tablet_bootstrap.cc:492] T 5262c2241e524395a73c0eb368917169 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:41.496508 2325 raft_consensus.cc:2804] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:41.496639 1166 ts_tablet_manager.cc:1403] T 5262c2241e524395a73c0eb368917169 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.145s user 0.019s sys 0.024s
I20251211 20:52:41.496613 2375 raft_consensus.cc:515] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.496867 2325 raft_consensus.cc:697] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:41.497167 2325 consensus_queue.cc:237] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.497488 1166 raft_consensus.cc:359] T 5262c2241e524395a73c0eb368917169 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.497637 1166 raft_consensus.cc:385] T 5262c2241e524395a73c0eb368917169 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.497697 1166 raft_consensus.cc:740] T 5262c2241e524395a73c0eb368917169 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.497725 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2e33606930774fe4aafc6e3819618af8" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:41.497885 1166 consensus_queue.cc:260] T 5262c2241e524395a73c0eb368917169 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.497635 2375 leader_election.cc:290] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:41.498389 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2e33606930774fe4aafc6e3819618af8" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:41.499697 982 leader_election.cc:343] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.501312 909 raft_consensus.cc:3060] T 8229b704e7464b30b7d2902a938ce33b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:41.501922 983 leader_election.cc:343] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.502027 983 leader_election.cc:304] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:41.502441 2375 raft_consensus.cc:2749] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:41.503724 2375 raft_consensus.cc:493] T 645b08b33676420c96e880de3d460829 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.503858 2375 raft_consensus.cc:515] T 645b08b33676420c96e880de3d460829 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.504411 2375 leader_election.cc:290] T 645b08b33676420c96e880de3d460829 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:41.505507 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "645b08b33676420c96e880de3d460829" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:41.506225 982 leader_election.cc:343] T 645b08b33676420c96e880de3d460829 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.507431 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "645b08b33676420c96e880de3d460829" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:41.507994 983 leader_election.cc:343] T 645b08b33676420c96e880de3d460829 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.508100 983 leader_election.cc:304] T 645b08b33676420c96e880de3d460829 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:41.508245 1166 ts_tablet_manager.cc:1434] T 5262c2241e524395a73c0eb368917169 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.012s user 0.001s sys 0.005s
I20251211 20:52:41.508528 1166 tablet_bootstrap.cc:492] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:41.509893 1166 tablet_bootstrap.cc:654] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.510571 2376 raft_consensus.cc:2749] T 645b08b33676420c96e880de3d460829 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:41.477082 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b793241ee3b42b1a38e9f2f05e1173b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:41.516119 711 leader_election.cc:343] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.519132 1166 tablet_bootstrap.cc:492] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:41.519281 1166 ts_tablet_manager.cc:1403] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.005s
I20251211 20:52:41.520066 1166 raft_consensus.cc:359] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.520242 1166 raft_consensus.cc:385] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.520303 1166 raft_consensus.cc:740] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.521047 1160 ts_tablet_manager.cc:1434] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.037s user 0.001s sys 0.000s
I20251211 20:52:41.521422 1160 tablet_bootstrap.cc:492] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:41.523383 870 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:43876 (request call id 157) took 1091 ms. Trace:
I20251211 20:52:41.523540 870 rpcz_store.cc:276] 1211 20:52:40.432433 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:40.432500 (+ 67us) service_pool.cc:225] Handling call
1211 20:52:41.523353 (+1090853us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20251211 20:52:41.524053 1166 consensus_queue.cc:260] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:41.524470 982 leader_election.cc:343] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.524590 982 leader_election.cc:304] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:41.524642 1166 ts_tablet_manager.cc:1434] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251211 20:52:41.524847 2376 raft_consensus.cc:2749] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:41.524904 1166 tablet_bootstrap.cc:492] T 861d2d70a24946699dccd65cab05195e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:41.525933 1168 tablet_bootstrap.cc:492] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:41.526078 1168 ts_tablet_manager.cc:1403] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.168s user 0.032s sys 0.019s
I20251211 20:52:41.526221 1166 tablet_bootstrap.cc:654] T 861d2d70a24946699dccd65cab05195e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.526746 1168 raft_consensus.cc:359] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.526927 1168 raft_consensus.cc:385] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.526980 1168 raft_consensus.cc:740] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.527247 1168 consensus_queue.cc:260] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.527755 1168 ts_tablet_manager.cc:1434] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:41.528062 1168 tablet_bootstrap.cc:492] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:41.529677 1168 tablet_bootstrap.cc:654] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:41.549329 710 consensus_peers.cc:597] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2630ad57845a4c31b20c8a104dd27335. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:41.549418 2353 tablet.cc:1722] T 8229b704e7464b30b7d2902a938ce33b P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20251211 20:52:41.552664 2135 raft_consensus.cc:493] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.552908 2135 raft_consensus.cc:515] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.553333 2322 raft_consensus.cc:493] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.553464 2322 raft_consensus.cc:515] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.553957 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f3855ffb51a74101b26fd28429e65bc2" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:41.554030 2322 leader_election.cc:290] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:41.554397 983 leader_election.cc:343] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.554684 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f807596fa6a4e66b89566b5d5dd1080" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:41.555153 847 leader_election.cc:343] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 4f807596fa6a4e66b89566b5d5dd1080
I20251211 20:52:41.555543 847 leader_election.cc:304] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:41.556008 2322 raft_consensus.cc:2749] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:41.570024 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f3855ffb51a74101b26fd28429e65bc2" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:41.570580 1165 ts_tablet_manager.cc:1929] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20251211 20:52:41.570629 982 leader_election.cc:343] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.570760 982 leader_election.cc:304] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:41.570978 2376 raft_consensus.cc:2749] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:41.572218 1168 tablet_bootstrap.cc:492] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:41.575074 1168 ts_tablet_manager.cc:1403] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.047s user 0.002s sys 0.002s
I20251211 20:52:41.575922 1168 raft_consensus.cc:359] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.576164 1168 raft_consensus.cc:385] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.576263 1168 raft_consensus.cc:740] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.576570 1168 consensus_queue.cc:260] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.576866 1165 log.cc:1199] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/52ef1707f6f2425aafd454f303af0333
I20251211 20:52:41.577190 1168 ts_tablet_manager.cc:1434] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:41.577490 1168 tablet_bootstrap.cc:492] T 4df68d1b6a91458e9f212ae81aa25438 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:41.578154 2135 leader_election.cc:290] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:41.578830 1168 tablet_bootstrap.cc:654] T 4df68d1b6a91458e9f212ae81aa25438 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.584744 1165 ts_tablet_manager.cc:1950] T 52ef1707f6f2425aafd454f303af0333 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:41.587397 1160 tablet_bootstrap.cc:654] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.595657 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 852) took 3203 ms. Trace:
I20251211 20:52:41.599246 1165 rpcz_store.cc:276] 1211 20:52:38.392404 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.393450 (+ 1046us) service_pool.cc:225] Handling call
1211 20:52:41.415790 (+3022340us) ts_tablet_manager.cc:1151] Deleting tablet 52ef1707f6f2425aafd454f303af0333
1211 20:52:41.415805 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:41.489464 (+ 73659us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:41.570464 (+ 81000us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:41.595630 (+ 25166us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3022174}
I20251211 20:52:41.600023 1165 tablet_replica.cc:333] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:41.601943 1168 tablet_bootstrap.cc:492] T 4df68d1b6a91458e9f212ae81aa25438 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:41.603825 1165 raft_consensus.cc:2243] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:41.603852 1168 ts_tablet_manager.cc:1403] T 4df68d1b6a91458e9f212ae81aa25438 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.026s user 0.005s sys 0.007s
I20251211 20:52:41.605237 1165 raft_consensus.cc:2272] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:41.605911 1168 raft_consensus.cc:359] T 4df68d1b6a91458e9f212ae81aa25438 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.606159 1168 raft_consensus.cc:385] T 4df68d1b6a91458e9f212ae81aa25438 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.606240 1168 raft_consensus.cc:740] T 4df68d1b6a91458e9f212ae81aa25438 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.606490 1168 consensus_queue.cc:260] T 4df68d1b6a91458e9f212ae81aa25438 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.606717 1160 tablet_bootstrap.cc:492] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:41.606983 1168 ts_tablet_manager.cc:1434] T 4df68d1b6a91458e9f212ae81aa25438 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:41.607126 1160 ts_tablet_manager.cc:1403] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.086s user 0.004s sys 0.001s
I20251211 20:52:41.608075 1160 raft_consensus.cc:359] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.608374 2135 raft_consensus.cc:493] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.608570 2135 raft_consensus.cc:515] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.610421 1160 raft_consensus.cc:385] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.610563 1160 raft_consensus.cc:740] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.610903 1160 consensus_queue.cc:260] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.610263 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6a61cf8f5de14adaa4a1de5bddde9a2f" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:41.611792 982 leader_election.cc:343] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.612185 1168 tablet_bootstrap.cc:492] T 2207af105bc3471ca930d35a91d38e70 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:41.615146 2376 raft_consensus.cc:493] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.615305 2376 raft_consensus.cc:515] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.616039 2376 leader_election.cc:290] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:41.616693 1168 tablet_bootstrap.cc:654] T 2207af105bc3471ca930d35a91d38e70 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.617933 1165 ts_tablet_manager.cc:1916] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:41.619065 1166 tablet_bootstrap.cc:492] T 861d2d70a24946699dccd65cab05195e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:41.619199 1166 ts_tablet_manager.cc:1403] T 861d2d70a24946699dccd65cab05195e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.094s user 0.021s sys 0.012s
I20251211 20:52:41.619179 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "22273287aa9245a2a66131fed2c8861c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:41.619402 776 raft_consensus.cc:2468] T 22273287aa9245a2a66131fed2c8861c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 0.
I20251211 20:52:41.619889 1166 raft_consensus.cc:359] T 861d2d70a24946699dccd65cab05195e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.619978 982 leader_election.cc:304] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:41.620042 1166 raft_consensus.cc:385] T 861d2d70a24946699dccd65cab05195e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.620086 1166 raft_consensus.cc:740] T 861d2d70a24946699dccd65cab05195e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.620281 1166 consensus_queue.cc:260] T 861d2d70a24946699dccd65cab05195e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.620380 2376 raft_consensus.cc:2804] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:41.620460 2376 raft_consensus.cc:493] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:41.620529 2376 raft_consensus.cc:3060] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:41.623016 2376 raft_consensus.cc:515] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.624547 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "22273287aa9245a2a66131fed2c8861c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:41.625330 983 leader_election.cc:343] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.627171 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "22273287aa9245a2a66131fed2c8861c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:41.627748 983 leader_election.cc:343] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.627843 2273 raft_consensus.cc:493] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.628021 2273 raft_consensus.cc:515] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.628432 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "22273287aa9245a2a66131fed2c8861c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:41.628595 776 raft_consensus.cc:3060] T 22273287aa9245a2a66131fed2c8861c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:41.629042 2273 leader_election.cc:290] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:41.631042 776 raft_consensus.cc:2468] T 22273287aa9245a2a66131fed2c8861c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 1.
I20251211 20:52:41.631785 982 leader_election.cc:304] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:41.632066 2375 raft_consensus.cc:2804] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:41.632294 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fed300d31c414f4a9b27c66e78097688" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:41.632473 1045 raft_consensus.cc:2468] T fed300d31c414f4a9b27c66e78097688 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:41.633081 710 leader_election.cc:304] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:41.633463 2273 raft_consensus.cc:2804] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:41.633574 2273 raft_consensus.cc:493] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:41.633678 2273 raft_consensus.cc:3060] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:41.636199 2273 raft_consensus.cc:515] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.637395 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fed300d31c414f4a9b27c66e78097688" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:41.637432 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fed300d31c414f4a9b27c66e78097688" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:41.637578 1045 raft_consensus.cc:3060] T fed300d31c414f4a9b27c66e78097688 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:41.637972 711 leader_election.cc:343] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.638360 1166 ts_tablet_manager.cc:1434] T 861d2d70a24946699dccd65cab05195e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.019s user 0.002s sys 0.000s
I20251211 20:52:41.640687 1045 raft_consensus.cc:2468] T fed300d31c414f4a9b27c66e78097688 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:41.640887 1166 tablet_bootstrap.cc:492] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:41.641330 710 leader_election.cc:304] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:41.642016 2286 raft_consensus.cc:2804] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:41.642136 2286 raft_consensus.cc:697] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:41.642271 1166 tablet_bootstrap.cc:654] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.642367 2286 consensus_queue.cc:237] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.643810 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fed300d31c414f4a9b27c66e78097688" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:41.644245 2376 leader_election.cc:290] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
W20251211 20:52:41.644591 711 leader_election.cc:343] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.644639 2376 raft_consensus.cc:493] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.645062 2376 raft_consensus.cc:515] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.646368 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b1ce6d2a12a4c7ea044b349412b4bf5" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:41.646950 982 leader_election.cc:343] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.647655 2376 leader_election.cc:290] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:41.648653 2273 leader_election.cc:290] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:41.650513 2135 leader_election.cc:290] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:41.651649 1160 ts_tablet_manager.cc:1434] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.044s user 0.002s sys 0.000s
I20251211 20:52:41.654274 1166 tablet_bootstrap.cc:492] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:41.654410 1166 ts_tablet_manager.cc:1403] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.001s
I20251211 20:52:41.655109 1166 raft_consensus.cc:359] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.655272 1166 raft_consensus.cc:385] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.655324 1166 raft_consensus.cc:740] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.656322 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6a61cf8f5de14adaa4a1de5bddde9a2f" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:41.656623 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b1ce6d2a12a4c7ea044b349412b4bf5" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:41.656868 983 leader_election.cc:343] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.656970 983 leader_election.cc:304] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:41.657338 983 leader_election.cc:343] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.657491 983 leader_election.cc:304] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:41.657868 2135 raft_consensus.cc:2749] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:41.658077 2135 raft_consensus.cc:2749] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:41.660387 1165 ts_tablet_manager.cc:1929] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:41.660568 1165 log.cc:1199] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/92a71dff4ea8445d8484fc2747471148
I20251211 20:52:41.660913 1165 ts_tablet_manager.cc:1950] T 92a71dff4ea8445d8484fc2747471148 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:41.661607 2286 raft_consensus.cc:493] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.661693 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 853) took 3269 ms. Trace:
I20251211 20:52:41.661844 2286 raft_consensus.cc:515] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.661906 1165 rpcz_store.cc:276] 1211 20:52:38.392484 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.393637 (+ 1153us) service_pool.cc:225] Handling call
1211 20:52:41.599969 (+3206332us) ts_tablet_manager.cc:1151] Deleting tablet 92a71dff4ea8445d8484fc2747471148
1211 20:52:41.599981 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:41.626430 (+ 26449us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:41.660274 (+ 33844us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:41.661673 (+ 1399us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":12107,"tablet-delete.queue_time_us":3206187}
I20251211 20:52:41.662494 1165 tablet_replica.cc:333] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:41.662719 1165 raft_consensus.cc:2243] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:41.662914 1165 raft_consensus.cc:2272] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:41.663102 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a3b1aadf17ea4bf0a896d23cd543fca7" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:41.662833 2286 leader_election.cc:290] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:41.663298 1045 raft_consensus.cc:2468] T a3b1aadf17ea4bf0a896d23cd543fca7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:41.663656 1165 ts_tablet_manager.cc:1916] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:41.663689 710 leader_election.cc:304] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:41.664546 1168 tablet_bootstrap.cc:492] T 2207af105bc3471ca930d35a91d38e70 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:41.664721 1168 ts_tablet_manager.cc:1403] T 2207af105bc3471ca930d35a91d38e70 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.053s user 0.006s sys 0.002s
I20251211 20:52:41.665375 1168 raft_consensus.cc:359] T 2207af105bc3471ca930d35a91d38e70 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.665684 1168 raft_consensus.cc:385] T 2207af105bc3471ca930d35a91d38e70 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.665762 1168 raft_consensus.cc:740] T 2207af105bc3471ca930d35a91d38e70 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.665992 1168 consensus_queue.cc:260] T 2207af105bc3471ca930d35a91d38e70 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.666482 1168 ts_tablet_manager.cc:1434] T 2207af105bc3471ca930d35a91d38e70 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:41.666778 1168 tablet_bootstrap.cc:492] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:41.668244 1168 tablet_bootstrap.cc:654] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.668787 1160 tablet_bootstrap.cc:492] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:41.665619 2286 raft_consensus.cc:2804] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:41.668964 2286 raft_consensus.cc:493] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:41.669087 2286 raft_consensus.cc:3060] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:41.670413 1160 tablet_bootstrap.cc:654] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.670905 2375 raft_consensus.cc:697] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 LEADER]: Becoming Leader. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Running, Role: LEADER
I20251211 20:52:41.671180 2375 consensus_queue.cc:237] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.671859 2286 raft_consensus.cc:515] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.672303 1166 consensus_queue.cc:260] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.672792 1166 ts_tablet_manager.cc:1434] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.018s user 0.001s sys 0.001s
I20251211 20:52:41.673076 1166 tablet_bootstrap.cc:492] T 41c30bded6504bbc9f0af8f34dc81e8d P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:41.673010 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a3b1aadf17ea4bf0a896d23cd543fca7" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:41.673182 2286 leader_election.cc:290] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:41.673290 1045 raft_consensus.cc:3060] T a3b1aadf17ea4bf0a896d23cd543fca7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:41.674347 1166 tablet_bootstrap.cc:654] T 41c30bded6504bbc9f0af8f34dc81e8d P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.675917 1045 raft_consensus.cc:2468] T a3b1aadf17ea4bf0a896d23cd543fca7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:41.676429 710 leader_election.cc:304] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:41.676694 2286 raft_consensus.cc:2804] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:41.676802 2286 raft_consensus.cc:697] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:41.676999 2286 consensus_queue.cc:237] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.678088 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a3b1aadf17ea4bf0a896d23cd543fca7" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:41.678645 711 leader_election.cc:343] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.680502 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a3b1aadf17ea4bf0a896d23cd543fca7" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:41.681099 711 leader_election.cc:343] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.690721 1165 ts_tablet_manager.cc:1929] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:41.692016 1168 tablet_bootstrap.cc:492] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:41.692569 1168 ts_tablet_manager.cc:1403] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.002s
I20251211 20:52:41.693306 1168 raft_consensus.cc:359] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.693529 1168 raft_consensus.cc:385] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.693621 1168 raft_consensus.cc:740] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.693892 1168 consensus_queue.cc:260] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.694384 1168 ts_tablet_manager.cc:1434] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:41.694761 1168 tablet_bootstrap.cc:492] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:41.696249 1168 tablet_bootstrap.cc:654] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.699685 1166 tablet_bootstrap.cc:492] T 41c30bded6504bbc9f0af8f34dc81e8d P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:41.699129 1165 log.cc:1199] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/4201da4241aa48dfb670a713e4453ca5
I20251211 20:52:41.699841 1166 ts_tablet_manager.cc:1403] T 41c30bded6504bbc9f0af8f34dc81e8d P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.002s
I20251211 20:52:41.700174 1165 ts_tablet_manager.cc:1950] T 4201da4241aa48dfb670a713e4453ca5 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:41.700584 1166 raft_consensus.cc:359] T 41c30bded6504bbc9f0af8f34dc81e8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.700754 1166 raft_consensus.cc:385] T 41c30bded6504bbc9f0af8f34dc81e8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.700807 1166 raft_consensus.cc:740] T 41c30bded6504bbc9f0af8f34dc81e8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.701045 1166 consensus_queue.cc:260] T 41c30bded6504bbc9f0af8f34dc81e8d P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.701314 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 855) took 3308 ms. Trace:
I20251211 20:52:41.701424 1165 rpcz_store.cc:276] 1211 20:52:38.392652 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.393815 (+ 1163us) service_pool.cc:225] Handling call
1211 20:52:41.662445 (+3268630us) ts_tablet_manager.cc:1151] Deleting tablet 4201da4241aa48dfb670a713e4453ca5
1211 20:52:41.662456 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:41.675640 (+ 13184us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:41.690611 (+ 14971us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:41.701293 (+ 10682us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3268493}
I20251211 20:52:41.701612 1166 ts_tablet_manager.cc:1434] T 41c30bded6504bbc9f0af8f34dc81e8d P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:41.701884 1166 tablet_bootstrap.cc:492] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:41.702054 1165 tablet_replica.cc:333] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:41.702256 1165 raft_consensus.cc:2243] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:41.702458 1165 raft_consensus.cc:2272] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:41.703003 1165 ts_tablet_manager.cc:1916] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:41.703217 1166 tablet_bootstrap.cc:654] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.706295 1160 tablet_bootstrap.cc:492] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:41.706452 1160 ts_tablet_manager.cc:1403] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.000s
I20251211 20:52:41.707648 1160 raft_consensus.cc:359] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.707860 1160 raft_consensus.cc:385] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.707932 1160 raft_consensus.cc:740] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.708145 1160 consensus_queue.cc:260] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.710569 1160 ts_tablet_manager.cc:1434] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
W20251211 20:52:41.716480 710 consensus_peers.cc:597] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c8e17645a26947ab8c01f9b093fa422c. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:41.716511 1168 tablet_bootstrap.cc:492] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:41.716837 1168 ts_tablet_manager.cc:1403] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.000s
I20251211 20:52:41.717585 1168 raft_consensus.cc:359] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.717787 1168 raft_consensus.cc:385] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.717847 1168 raft_consensus.cc:740] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.714977 1160 tablet_bootstrap.cc:492] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:41.721791 1166 tablet_bootstrap.cc:492] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:41.721967 1166 ts_tablet_manager.cc:1403] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20251211 20:52:41.722947 1166 raft_consensus.cc:359] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.723143 1166 raft_consensus.cc:385] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.723212 1166 raft_consensus.cc:740] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.723425 1166 consensus_queue.cc:260] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.724006 1166 ts_tablet_manager.cc:1434] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:41.730329 2375 raft_consensus.cc:493] T 5262c2241e524395a73c0eb368917169 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.730473 2375 raft_consensus.cc:515] T 5262c2241e524395a73c0eb368917169 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.731047 2375 leader_election.cc:290] T 5262c2241e524395a73c0eb368917169 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:41.731283 1166 tablet_bootstrap.cc:492] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:41.733204 1166 tablet_bootstrap.cc:654] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.734551 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5262c2241e524395a73c0eb368917169" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:41.735356 982 leader_election.cc:343] T 5262c2241e524395a73c0eb368917169 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.736543 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5262c2241e524395a73c0eb368917169" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:41.737756 983 leader_election.cc:343] T 5262c2241e524395a73c0eb368917169 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.737875 983 leader_election.cc:304] T 5262c2241e524395a73c0eb368917169 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:41.738536 2375 raft_consensus.cc:2749] T 5262c2241e524395a73c0eb368917169 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:41.742909 1166 tablet_bootstrap.cc:492] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:41.743072 1166 ts_tablet_manager.cc:1403] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.003s
I20251211 20:52:41.727521 1165 ts_tablet_manager.cc:1929] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:41.744163 1165 log.cc:1199] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/0b267c7feaac43f6aacb7229759c7ec3
I20251211 20:52:41.744110 1166 raft_consensus.cc:359] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.744297 1166 raft_consensus.cc:385] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.744369 1166 raft_consensus.cc:740] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.744582 1166 consensus_queue.cc:260] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.744705 1165 ts_tablet_manager.cc:1950] T 0b267c7feaac43f6aacb7229759c7ec3 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:41.745539 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 856) took 3352 ms. Trace:
I20251211 20:52:41.745677 1165 rpcz_store.cc:276] 1211 20:52:38.392726 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.393977 (+ 1251us) service_pool.cc:225] Handling call
1211 20:52:41.702007 (+3308030us) ts_tablet_manager.cc:1151] Deleting tablet 0b267c7feaac43f6aacb7229759c7ec3
1211 20:52:41.702017 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:41.719451 (+ 17434us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:41.727411 (+ 7960us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:41.745516 (+ 18105us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3307890}
I20251211 20:52:41.746208 1165 tablet_replica.cc:333] T e4b5a7553110424d901788c7a2ecdce6 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:41.746429 1165 raft_consensus.cc:2243] T e4b5a7553110424d901788c7a2ecdce6 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:41.747380 1166 ts_tablet_manager.cc:1434] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251211 20:52:41.747756 1166 tablet_bootstrap.cc:492] T c91ff29847d54e55b19321652a857e9f P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:41.750113 1166 tablet_bootstrap.cc:654] T c91ff29847d54e55b19321652a857e9f P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:41.750635 711 consensus_peers.cc:597] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c8e17645a26947ab8c01f9b093fa422c. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:41.757157 1165 raft_consensus.cc:2272] T e4b5a7553110424d901788c7a2ecdce6 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:41.761638 2375 raft_consensus.cc:493] T 50f9f29c064c4f938a0f20f302ab5e8a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.761786 2375 raft_consensus.cc:515] T 50f9f29c064c4f938a0f20f302ab5e8a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.762344 2375 leader_election.cc:290] T 50f9f29c064c4f938a0f20f302ab5e8a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:41.763053 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "50f9f29c064c4f938a0f20f302ab5e8a" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:41.763746 982 leader_election.cc:343] T 50f9f29c064c4f938a0f20f302ab5e8a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.763943 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "50f9f29c064c4f938a0f20f302ab5e8a" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:41.764443 983 leader_election.cc:343] T 50f9f29c064c4f938a0f20f302ab5e8a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.764546 983 leader_election.cc:304] T 50f9f29c064c4f938a0f20f302ab5e8a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:41.764992 2375 raft_consensus.cc:2749] T 50f9f29c064c4f938a0f20f302ab5e8a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:41.765619 1168 consensus_queue.cc:260] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.766297 1168 ts_tablet_manager.cc:1434] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.049s user 0.001s sys 0.000s
I20251211 20:52:41.719532 1160 tablet_bootstrap.cc:654] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.768151 1168 tablet_bootstrap.cc:492] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:41.768729 909 raft_consensus.cc:3060] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:41.769949 1168 tablet_bootstrap.cc:654] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.773967 2286 raft_consensus.cc:493] T 2207af105bc3471ca930d35a91d38e70 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.774112 2286 raft_consensus.cc:515] T 2207af105bc3471ca930d35a91d38e70 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.774695 2286 leader_election.cc:290] T 2207af105bc3471ca930d35a91d38e70 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:41.775538 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2207af105bc3471ca930d35a91d38e70" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:41.776201 711 leader_election.cc:343] T 2207af105bc3471ca930d35a91d38e70 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:41.779567 710 leader_election.cc:343] T 2207af105bc3471ca930d35a91d38e70 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 2207af105bc3471ca930d35a91d38e70
I20251211 20:52:41.779722 710 leader_election.cc:304] T 2207af105bc3471ca930d35a91d38e70 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:41.780109 2286 raft_consensus.cc:2749] T 2207af105bc3471ca930d35a91d38e70 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:41.781553 1160 tablet_bootstrap.cc:492] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:41.781734 1160 ts_tablet_manager.cc:1403] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.067s user 0.003s sys 0.000s
I20251211 20:52:41.782399 1160 raft_consensus.cc:359] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.782570 1160 raft_consensus.cc:385] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.782631 1160 raft_consensus.cc:740] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.782855 1160 consensus_queue.cc:260] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.783308 1160 ts_tablet_manager.cc:1434] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251211 20:52:41.783622 1165 ts_tablet_manager.cc:1916] T e4b5a7553110424d901788c7a2ecdce6 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:41.794351 1160 tablet_bootstrap.cc:492] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:41.796880 1165 ts_tablet_manager.cc:1929] T e4b5a7553110424d901788c7a2ecdce6 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:41.803185 1160 tablet_bootstrap.cc:654] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.810243 1165 log.cc:1199] T e4b5a7553110424d901788c7a2ecdce6 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/e4b5a7553110424d901788c7a2ecdce6
I20251211 20:52:41.810724 1165 ts_tablet_manager.cc:1950] T e4b5a7553110424d901788c7a2ecdce6 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:41.811573 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 863) took 3342 ms. Trace:
I20251211 20:52:41.812243 1165 rpcz_store.cc:276] 1211 20:52:38.468796 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.469034 (+ 238us) service_pool.cc:225] Handling call
1211 20:52:41.746160 (+3277126us) ts_tablet_manager.cc:1151] Deleting tablet e4b5a7553110424d901788c7a2ecdce6
1211 20:52:41.746171 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:41.786547 (+ 40376us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:41.796772 (+ 10225us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:41.811553 (+ 14781us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3276945}
I20251211 20:52:41.822487 1166 tablet_bootstrap.cc:492] T c91ff29847d54e55b19321652a857e9f P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:41.822546 1160 tablet_bootstrap.cc:492] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:41.822681 1160 ts_tablet_manager.cc:1403] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.039s user 0.003s sys 0.000s
I20251211 20:52:41.822695 1166 ts_tablet_manager.cc:1403] T c91ff29847d54e55b19321652a857e9f P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.075s user 0.016s sys 0.013s
I20251211 20:52:41.823381 1160 raft_consensus.cc:359] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.823555 1160 raft_consensus.cc:385] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.823453 1166 raft_consensus.cc:359] T c91ff29847d54e55b19321652a857e9f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.823629 1160 raft_consensus.cc:740] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.823666 1166 raft_consensus.cc:385] T c91ff29847d54e55b19321652a857e9f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.823729 1166 raft_consensus.cc:740] T c91ff29847d54e55b19321652a857e9f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.823820 1160 consensus_queue.cc:260] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.824003 1166 consensus_queue.cc:260] T c91ff29847d54e55b19321652a857e9f P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.824633 1166 ts_tablet_manager.cc:1434] T c91ff29847d54e55b19321652a857e9f P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:41.824937 1166 tablet_bootstrap.cc:492] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:41.826907 1165 tablet_replica.cc:333] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:41.827316 1165 raft_consensus.cc:2243] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:41.827970 1165 raft_consensus.cc:2272] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:41.828943 1165 ts_tablet_manager.cc:1916] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:41.831912 1166 tablet_bootstrap.cc:654] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.837359 2402 raft_consensus.cc:493] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9230f94a6a1a49cead51aeea9f069087)
I20251211 20:52:41.838616 2402 raft_consensus.cc:515] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:41.839134 1160 ts_tablet_manager.cc:1434] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.016s user 0.002s sys 0.000s
I20251211 20:52:41.840384 2322 raft_consensus.cc:493] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.840509 2322 raft_consensus.cc:515] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.841048 2322 leader_election.cc:290] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:41.841567 1350 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11f5f94732f049d0be4c97cebfffcb05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1197 } } ignore_live_leader: false dest_uuid: "1830b0c58cfe495989f3d911dda0409a" is_pre_election: true
I20251211 20:52:41.841888 1350 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f5f94732f049d0be4c97cebfffcb05 in term 1.
W20251211 20:52:41.842176 847 leader_election.cc:343] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: e28d66ce4711430cb09a9650e99fbe9e
I20251211 20:52:41.842620 1786 leader_election.cc:304] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f5f94732f049d0be4c97cebfffcb05, 1830b0c58cfe495989f3d911dda0409a; no voters:
W20251211 20:52:41.843179 847 leader_election.cc:343] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: e28d66ce4711430cb09a9650e99fbe9e
I20251211 20:52:41.843299 847 leader_election.cc:304] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:41.843704 2322 raft_consensus.cc:2749] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:41.848289 1160 tablet_bootstrap.cc:492] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:41.848726 1168 tablet_bootstrap.cc:492] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:41.848896 1168 ts_tablet_manager.cc:1403] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.081s user 0.008s sys 0.000s
I20251211 20:52:41.849802 1160 tablet_bootstrap.cc:654] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.850319 1168 raft_consensus.cc:359] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.850517 1168 raft_consensus.cc:385] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.850580 1168 raft_consensus.cc:740] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.850863 1168 consensus_queue.cc:260] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.851534 1168 ts_tablet_manager.cc:1434] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20251211 20:52:41.852186 2402 leader_election.cc:290] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1830b0c58cfe495989f3d911dda0409a (127.0.104.62:41123), 9230f94a6a1a49cead51aeea9f069087 (127.0.104.60:42087)
I20251211 20:52:41.852384 2402 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251211 20:52:41.852344 2403 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9230f94a6a1a49cead51aeea9f069087)
I20251211 20:52:41.852478 2402 raft_consensus.cc:493] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9230f94a6a1a49cead51aeea9f069087)
I20251211 20:52:41.852571 2402 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 1 FOLLOWER]: Advancing to term 2
I20251211 20:52:41.853500 2375 raft_consensus.cc:493] T c91ff29847d54e55b19321652a857e9f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.853638 2375 raft_consensus.cc:515] T c91ff29847d54e55b19321652a857e9f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.854286 2375 leader_election.cc:290] T c91ff29847d54e55b19321652a857e9f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:41.855185 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c91ff29847d54e55b19321652a857e9f" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:41.853644 1166 tablet_bootstrap.cc:492] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:41.855405 1166 ts_tablet_manager.cc:1403] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.031s user 0.001s sys 0.004s
W20251211 20:52:41.855726 983 leader_election.cc:343] T c91ff29847d54e55b19321652a857e9f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.856206 1166 raft_consensus.cc:359] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.856374 1166 raft_consensus.cc:385] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.856431 1166 raft_consensus.cc:740] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.856416 1165 ts_tablet_manager.cc:1929] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:41.856516 1165 log.cc:1199] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/4f807596fa6a4e66b89566b5d5dd1080
I20251211 20:52:41.852669 2403 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:41.856840 1165 ts_tablet_manager.cc:1950] T 4f807596fa6a4e66b89566b5d5dd1080 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:41.863734 2322 raft_consensus.cc:493] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.863893 2322 raft_consensus.cc:515] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.864442 2322 leader_election.cc:290] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:41.864883 2375 raft_consensus.cc:493] T 3de5b34b96404cdc9c191fed7f055638 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.865013 2375 raft_consensus.cc:515] T 3de5b34b96404cdc9c191fed7f055638 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.865542 2375 leader_election.cc:290] T 3de5b34b96404cdc9c191fed7f055638 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
W20251211 20:52:41.865804 847 leader_election.cc:343] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 0b267c7feaac43f6aacb7229759c7ec3
I20251211 20:52:41.867092 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3de5b34b96404cdc9c191fed7f055638" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:41.867460 847 leader_election.cc:343] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 0b267c7feaac43f6aacb7229759c7ec3
I20251211 20:52:41.867561 847 leader_election.cc:304] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:41.860972 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c91ff29847d54e55b19321652a857e9f" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:41.868157 982 leader_election.cc:343] T c91ff29847d54e55b19321652a857e9f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.868229 2322 raft_consensus.cc:2749] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:41.868273 982 leader_election.cc:304] T c91ff29847d54e55b19321652a857e9f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:41.868157 983 leader_election.cc:343] T 3de5b34b96404cdc9c191fed7f055638 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.868633 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3de5b34b96404cdc9c191fed7f055638" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:41.868743 2375 raft_consensus.cc:2749] T c91ff29847d54e55b19321652a857e9f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:41.869081 982 leader_election.cc:343] T 3de5b34b96404cdc9c191fed7f055638 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.869169 982 leader_election.cc:304] T 3de5b34b96404cdc9c191fed7f055638 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:41.869410 1160 tablet_bootstrap.cc:492] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:41.869565 2375 raft_consensus.cc:2749] T 3de5b34b96404cdc9c191fed7f055638 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:41.869637 1160 ts_tablet_manager.cc:1403] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.000s
I20251211 20:52:41.870308 1160 raft_consensus.cc:359] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.870466 1160 raft_consensus.cc:385] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.870581 1160 raft_consensus.cc:740] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.870833 1160 consensus_queue.cc:260] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.867687 1168 tablet_bootstrap.cc:492] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:41.872287 1166 consensus_queue.cc:260] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.876099 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 859) took 3461 ms. Trace:
I20251211 20:52:41.876806 1165 rpcz_store.cc:276] 1211 20:52:38.414981 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.475675 (+ 60694us) service_pool.cc:225] Handling call
1211 20:52:41.812771 (+3337096us) ts_tablet_manager.cc:1151] Deleting tablet 4f807596fa6a4e66b89566b5d5dd1080
1211 20:52:41.826856 (+ 14085us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:41.834703 (+ 7847us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:41.856310 (+ 21607us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:41.876075 (+ 19765us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3336549}
I20251211 20:52:41.877665 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 867) took 3384 ms. Trace:
I20251211 20:52:41.878095 1165 rpcz_store.cc:276] 1211 20:52:38.492880 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.493059 (+ 179us) service_pool.cc:225] Handling call
1211 20:52:41.877529 (+3384470us) ts_tablet_manager.cc:1151] Deleting tablet 5bcc94ec9630484cb7a3e8bf6d4f378a
1211 20:52:41.877542 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:41.877649 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3384273}
I20251211 20:52:41.880873 1168 tablet_bootstrap.cc:654] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.878108 1166 ts_tablet_manager.cc:1434] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.023s user 0.000s sys 0.002s
I20251211 20:52:41.880478 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 868) took 3321 ms. Trace:
I20251211 20:52:41.881814 1165 rpcz_store.cc:276] 1211 20:52:38.559089 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.584151 (+ 25062us) service_pool.cc:225] Handling call
1211 20:52:41.880332 (+3296181us) ts_tablet_manager.cc:1151] Deleting tablet 75dd170c5828409fa69bd68fdf28a72b
1211 20:52:41.880347 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:41.880454 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3295961}
I20251211 20:52:41.882550 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 872) took 3281 ms. Trace:
I20251211 20:52:41.882938 1165 rpcz_store.cc:276] 1211 20:52:38.601157 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.601216 (+ 59us) service_pool.cc:225] Handling call
1211 20:52:41.882444 (+3281228us) ts_tablet_manager.cc:1151] Deleting tablet 449921ea74f04a9c88eb9e094e8226c8
1211 20:52:41.882454 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:41.882536 (+ 82us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3281036}
I20251211 20:52:41.883890 909 raft_consensus.cc:3060] T 1a77e3e744ff42ccb68897704b4c2f1d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:41.884548 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 873) took 3266 ms. Trace:
I20251211 20:52:41.884629 1165 rpcz_store.cc:276] 1211 20:52:38.618548 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.618657 (+ 109us) service_pool.cc:225] Handling call
1211 20:52:41.884447 (+3265790us) ts_tablet_manager.cc:1151] Deleting tablet 6029c1fd563d4023a8cf7cfb8ffa0876
1211 20:52:41.884456 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:41.884533 (+ 77us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3265575}
I20251211 20:52:41.885735 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 886) took 3012 ms. Trace:
I20251211 20:52:41.885800 1165 rpcz_store.cc:276] 1211 20:52:38.873752 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.873811 (+ 59us) service_pool.cc:225] Handling call
1211 20:52:41.885637 (+3011826us) ts_tablet_manager.cc:1151] Deleting tablet 80c566822fa94ada983aa7266df4bf9d
1211 20:52:41.885647 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:41.885719 (+ 72us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3011550}
I20251211 20:52:41.887198 1166 tablet_bootstrap.cc:492] T 3e6f62c2deff4c11b137fc0cba5ee234 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:41.887194 2402 raft_consensus.cc:515] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:41.887627 1165 tablet_replica.cc:333] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:41.888224 1165 raft_consensus.cc:2243] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:41.888898 1166 tablet_bootstrap.cc:654] T 3e6f62c2deff4c11b137fc0cba5ee234 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.889420 1160 ts_tablet_manager.cc:1434] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.020s user 0.001s sys 0.000s
I20251211 20:52:41.889555 2403 leader_election.cc:290] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 11f5f94732f049d0be4c97cebfffcb05 (127.0.104.61:40749), 9230f94a6a1a49cead51aeea9f069087 (127.0.104.60:42087)
I20251211 20:52:41.889648 1165 pending_rounds.cc:70] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919: Trying to abort 1 pending ops.
I20251211 20:52:41.889750 1160 tablet_bootstrap.cc:492] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:41.889025 1786 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.62:41123, user_credentials={real_user=slave}} blocked reactor thread for 46348us
I20251211 20:52:41.890831 1165 pending_rounds.cc:77] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432127262052352 op_type: NO_OP noop_request { }
I20251211 20:52:41.890933 1350 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11f5f94732f049d0be4c97cebfffcb05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1197 } } ignore_live_leader: false dest_uuid: "1830b0c58cfe495989f3d911dda0409a"
I20251211 20:52:41.891043 1165 raft_consensus.cc:2889] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:41.891132 1165 raft_consensus.cc:2272] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:41.891122 1350 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 1 FOLLOWER]: Advancing to term 2
I20251211 20:52:41.891726 1165 ts_tablet_manager.cc:1916] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:41.892048 2402 leader_election.cc:290] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [CANDIDATE]: Term 2 election: Requested vote from peers 1830b0c58cfe495989f3d911dda0409a (127.0.104.62:41123), 9230f94a6a1a49cead51aeea9f069087 (127.0.104.60:42087)
I20251211 20:52:41.896787 1350 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f5f94732f049d0be4c97cebfffcb05 in term 2.
I20251211 20:52:41.897486 2273 raft_consensus.cc:493] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.897634 2273 raft_consensus.cc:515] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.898208 2273 leader_election.cc:290] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:41.898821 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28ddb8a59a214576a25d613d42c2fef5" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:41.899046 1045 raft_consensus.cc:2468] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:41.899717 710 leader_election.cc:304] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:41.900595 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28ddb8a59a214576a25d613d42c2fef5" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:41.901185 711 leader_election.cc:343] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.901374 2353 tablet.cc:1722] T 1a77e3e744ff42ccb68897704b4c2f1d P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:41.901556 2273 raft_consensus.cc:2804] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:41.901635 2273 raft_consensus.cc:493] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:41.901690 2273 raft_consensus.cc:3060] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:41.904214 1160 tablet_bootstrap.cc:654] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.904822 2273 raft_consensus.cc:515] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.906226 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28ddb8a59a214576a25d613d42c2fef5" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:41.906850 711 leader_election.cc:343] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.908586 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28ddb8a59a214576a25d613d42c2fef5" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:41.908779 1045 raft_consensus.cc:3060] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:41.909669 2375 raft_consensus.cc:493] T 2e7e158809b9471fbd6e6ee9fb9ac569 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.909806 2375 raft_consensus.cc:515] T 2e7e158809b9471fbd6e6ee9fb9ac569 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.910380 2375 leader_election.cc:290] T 2e7e158809b9471fbd6e6ee9fb9ac569 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:41.910760 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2e7e158809b9471fbd6e6ee9fb9ac569" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:41.910912 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2e7e158809b9471fbd6e6ee9fb9ac569" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:41.911208 983 leader_election.cc:343] T 2e7e158809b9471fbd6e6ee9fb9ac569 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:41.911325 982 leader_election.cc:343] T 2e7e158809b9471fbd6e6ee9fb9ac569 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.911403 982 leader_election.cc:304] T 2e7e158809b9471fbd6e6ee9fb9ac569 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:41.911453 1045 raft_consensus.cc:2468] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:41.911871 2375 raft_consensus.cc:2749] T 2e7e158809b9471fbd6e6ee9fb9ac569 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:41.913512 710 leader_election.cc:304] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:41.913777 2286 raft_consensus.cc:2804] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:41.913869 2286 raft_consensus.cc:697] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:41.914075 2286 consensus_queue.cc:237] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.915572 2273 leader_election.cc:290] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:41.917996 1786 leader_election.cc:304] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f5f94732f049d0be4c97cebfffcb05, 1830b0c58cfe495989f3d911dda0409a; no voters:
I20251211 20:52:41.921823 1168 tablet_bootstrap.cc:492] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:41.922312 1168 ts_tablet_manager.cc:1403] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.055s user 0.004s sys 0.000s
I20251211 20:52:41.923230 1168 raft_consensus.cc:359] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.924714 1160 tablet_bootstrap.cc:492] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:41.924860 1160 ts_tablet_manager.cc:1403] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.035s user 0.004s sys 0.000s
I20251211 20:52:41.925530 1160 raft_consensus.cc:359] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.925693 1160 raft_consensus.cc:385] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.925736 1160 raft_consensus.cc:740] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.925925 1160 consensus_queue.cc:260] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.926390 1160 ts_tablet_manager.cc:1434] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251211 20:52:41.931552 2402 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 2 FOLLOWER]: Leader election won for term 2
I20251211 20:52:41.933558 1160 tablet_bootstrap.cc:492] T 3b2c6064c5eb4c0f8979c485f5d85a75 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:41.934677 1168 raft_consensus.cc:385] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.934861 1168 raft_consensus.cc:740] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.935319 2402 raft_consensus.cc:697] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [term 2 LEADER]: Becoming Leader. State: Replica: 11f5f94732f049d0be4c97cebfffcb05, State: Running, Role: LEADER
I20251211 20:52:41.935950 2402 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1197, Committed index: 1197, Last appended: 1.1197, Last appended by leader: 1197, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:41.936787 1168 consensus_queue.cc:260] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.940721 1829 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1830b0c58cfe495989f3d911dda0409a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1197 } } ignore_live_leader: false dest_uuid: "11f5f94732f049d0be4c97cebfffcb05" is_pre_election: true
I20251211 20:52:41.940979 1160 tablet_bootstrap.cc:654] T 3b2c6064c5eb4c0f8979c485f5d85a75 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.943567 1166 tablet_bootstrap.cc:492] T 3e6f62c2deff4c11b137fc0cba5ee234 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:41.943835 1166 ts_tablet_manager.cc:1403] T 3e6f62c2deff4c11b137fc0cba5ee234 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.057s user 0.013s sys 0.006s
I20251211 20:52:41.944648 1166 raft_consensus.cc:359] T 3e6f62c2deff4c11b137fc0cba5ee234 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.944916 1166 raft_consensus.cc:385] T 3e6f62c2deff4c11b137fc0cba5ee234 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.945008 1166 raft_consensus.cc:740] T 3e6f62c2deff4c11b137fc0cba5ee234 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.945252 1166 consensus_queue.cc:260] T 3e6f62c2deff4c11b137fc0cba5ee234 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.947942 2273 raft_consensus.cc:493] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.948076 2273 raft_consensus.cc:515] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.948618 2273 leader_election.cc:290] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:41.949297 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9bb22dce13f4c9da76ca446715cd99a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:41.949820 711 leader_election.cc:343] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.954916 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9bb22dce13f4c9da76ca446715cd99a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:41.955138 1045 raft_consensus.cc:2468] T a9bb22dce13f4c9da76ca446715cd99a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:41.955695 710 leader_election.cc:304] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:41.956312 2286 raft_consensus.cc:2804] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:41.956442 2286 raft_consensus.cc:493] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:41.956553 2286 raft_consensus.cc:3060] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:41.958077 2405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11f5f94732f049d0be4c97cebfffcb05. Latest consensus state: current_term: 2 leader_uuid: "11f5f94732f049d0be4c97cebfffcb05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } } }
I20251211 20:52:41.958493 2405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [sys.catalog]: This master's current role is: LEADER
I20251211 20:52:41.959149 1168 ts_tablet_manager.cc:1434] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.037s user 0.001s sys 0.000s
I20251211 20:52:41.960022 1166 ts_tablet_manager.cc:1434] T 3e6f62c2deff4c11b137fc0cba5ee234 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.016s user 0.001s sys 0.000s
I20251211 20:52:41.965538 1166 tablet_bootstrap.cc:492] T 4a39870c7eea4a379b284ea2e5ddf760 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:41.967391 1166 tablet_bootstrap.cc:654] T 4a39870c7eea4a379b284ea2e5ddf760 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.968442 2286 raft_consensus.cc:515] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.969621 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9bb22dce13f4c9da76ca446715cd99a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:41.969911 1168 tablet_bootstrap.cc:492] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:41.970343 711 leader_election.cc:343] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:41.971454 1168 tablet_bootstrap.cc:654] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.974313 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9bb22dce13f4c9da76ca446715cd99a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:41.974505 1045 raft_consensus.cc:3060] T a9bb22dce13f4c9da76ca446715cd99a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:41.977344 1045 raft_consensus.cc:2468] T a9bb22dce13f4c9da76ca446715cd99a P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:41.977524 1168 tablet_bootstrap.cc:492] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:41.977679 1168 ts_tablet_manager.cc:1403] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20251211 20:52:41.978005 710 leader_election.cc:304] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:41.978310 2273 raft_consensus.cc:2804] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:41.978440 2273 raft_consensus.cc:697] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:41.978461 2286 leader_election.cc:290] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:41.978693 2273 consensus_queue.cc:237] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.990620 1166 tablet_bootstrap.cc:492] T 4a39870c7eea4a379b284ea2e5ddf760 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:41.990823 1166 ts_tablet_manager.cc:1403] T 4a39870c7eea4a379b284ea2e5ddf760 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.000s
I20251211 20:52:41.991693 1166 raft_consensus.cc:359] T 4a39870c7eea4a379b284ea2e5ddf760 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.991912 1166 raft_consensus.cc:385] T 4a39870c7eea4a379b284ea2e5ddf760 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:41.991966 1166 raft_consensus.cc:740] T 4a39870c7eea4a379b284ea2e5ddf760 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:41.992183 1166 consensus_queue.cc:260] T 4a39870c7eea4a379b284ea2e5ddf760 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.992676 1166 ts_tablet_manager.cc:1434] T 4a39870c7eea4a379b284ea2e5ddf760 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:41.993022 1166 tablet_bootstrap.cc:492] T 75dd170c5828409fa69bd68fdf28a72b P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:41.994727 1166 tablet_bootstrap.cc:654] T 75dd170c5828409fa69bd68fdf28a72b P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:41.995967 2325 raft_consensus.cc:493] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.996122 2325 raft_consensus.cc:515] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.996747 2325 leader_election.cc:290] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:41.997570 2385 raft_consensus.cc:493] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.997707 2385 raft_consensus.cc:515] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.997856 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68792ccef54944a09af9b5456e9ff1fe" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:41.998189 2375 raft_consensus.cc:493] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:41.998292 2375 raft_consensus.cc:515] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:41.998324 2385 leader_election.cc:290] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:41.998785 2375 leader_election.cc:290] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
W20251211 20:52:41.999291 711 leader_election.cc:343] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:41.999681 710 leader_election.cc:343] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 68792ccef54944a09af9b5456e9ff1fe
I20251211 20:52:41.999814 710 leader_election.cc:304] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:41.999914 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3acfef6f082641a199b80eefcfc772ac" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:42.000578 983 leader_election.cc:343] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.000612 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "940dfd8781de444a9caa78dd363c7451" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:42.000857 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3acfef6f082641a199b80eefcfc772ac" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:42.001112 982 leader_election.cc:343] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:42.001466 982 leader_election.cc:343] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.001569 982 leader_election.cc:304] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.001928 2375 raft_consensus.cc:2749] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.002157 2325 raft_consensus.cc:2749] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.003883 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "940dfd8781de444a9caa78dd363c7451" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:42.004379 983 leader_election.cc:343] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.004483 983 leader_election.cc:304] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.004914 2375 raft_consensus.cc:2749] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.005679 2415 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251211 20:52:42.006174 1168 raft_consensus.cc:359] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.006863 1160 tablet_bootstrap.cc:492] T 3b2c6064c5eb4c0f8979c485f5d85a75 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:42.007014 1160 ts_tablet_manager.cc:1403] T 3b2c6064c5eb4c0f8979c485f5d85a75 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.074s user 0.022s sys 0.002s
I20251211 20:52:42.007010 1168 raft_consensus.cc:385] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.007177 1168 raft_consensus.cc:740] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.007727 1160 raft_consensus.cc:359] T 3b2c6064c5eb4c0f8979c485f5d85a75 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.007894 1160 raft_consensus.cc:385] T 3b2c6064c5eb4c0f8979c485f5d85a75 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.007943 1160 raft_consensus.cc:740] T 3b2c6064c5eb4c0f8979c485f5d85a75 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.008128 1160 consensus_queue.cc:260] T 3b2c6064c5eb4c0f8979c485f5d85a75 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.008381 1168 consensus_queue.cc:260] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.008589 1160 ts_tablet_manager.cc:1434] T 3b2c6064c5eb4c0f8979c485f5d85a75 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251211 20:52:42.007436 1165 ts_tablet_manager.cc:1929] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:42.008788 1165 log.cc:1199] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/c74427cb9d6f4a679e43eae796ff8c3f
I20251211 20:52:42.009208 1160 tablet_bootstrap.cc:492] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:42.010828 1160 tablet_bootstrap.cc:654] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.011181 2322 raft_consensus.cc:493] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.011325 2322 raft_consensus.cc:515] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.011973 2322 leader_election.cc:290] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:42.013976 847 leader_election.cc:343] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 25454e67437541d9ac2bfab29bad55f9
W20251211 20:52:42.014202 847 leader_election.cc:343] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 25454e67437541d9ac2bfab29bad55f9
I20251211 20:52:42.014221 2273 raft_consensus.cc:493] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.014299 847 leader_election.cc:304] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:42.014488 2273 raft_consensus.cc:515] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.015151 2322 raft_consensus.cc:2749] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.018734 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4ce495233a5e4f3792faf44680df54ba" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:42.019351 711 leader_election.cc:343] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:42.019526 710 leader_election.cc:343] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 4ce495233a5e4f3792faf44680df54ba
I20251211 20:52:42.019672 710 leader_election.cc:304] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.019948 2325 raft_consensus.cc:2749] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.024329 2273 leader_election.cc:290] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.025797 1160 tablet_bootstrap.cc:492] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:42.026002 1160 ts_tablet_manager.cc:1403] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.001s
I20251211 20:52:42.026693 1160 raft_consensus.cc:359] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.026917 1160 raft_consensus.cc:385] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.026996 1160 raft_consensus.cc:740] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.027240 1160 consensus_queue.cc:260] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.028115 1168 ts_tablet_manager.cc:1434] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.050s user 0.002s sys 0.000s
I20251211 20:52:42.028244 1165 ts_tablet_manager.cc:1950] T c74427cb9d6f4a679e43eae796ff8c3f P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:42.029282 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 887) took 3155 ms. Trace:
I20251211 20:52:42.029999 1160 ts_tablet_manager.cc:1434] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251211 20:52:42.030323 1160 tablet_bootstrap.cc:492] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:42.032025 1160 tablet_bootstrap.cc:654] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.039254 1165 rpcz_store.cc:276] 1211 20:52:38.874100 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.874192 (+ 92us) service_pool.cc:225] Handling call
1211 20:52:41.887551 (+3013359us) ts_tablet_manager.cc:1151] Deleting tablet c74427cb9d6f4a679e43eae796ff8c3f
1211 20:52:41.887560 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:41.933926 (+ 46366us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:42.007308 (+ 73382us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:42.029255 (+ 21947us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3013086}
I20251211 20:52:42.039743 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_11d5dfcc3a7d4db59119bf9c9666c50a [id=011d18ce4d764af8b3e399443dc217e9]
I20251211 20:52:42.036650 1168 tablet_bootstrap.cc:492] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:42.040705 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_fba99384e9f14160b3e3504f8d1d0b07 [id=01bba75f854d4c338d6c09a446dd7499]
I20251211 20:52:42.041312 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 892) took 3113 ms. Trace:
I20251211 20:52:42.041401 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_bf7d402a36be44cfb6b79041906d26a1 [id=0327e686e7ed4475a3e8cda8721a3a3a]
I20251211 20:52:42.042276 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_e2163380d2474e9a93f10e69de094195 [id=041f6b72446c42f68aa2fcaa8438ad0c]
I20251211 20:52:42.042423 1165 rpcz_store.cc:276] 1211 20:52:38.927505 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:38.927560 (+ 55us) service_pool.cc:225] Handling call
1211 20:52:42.041172 (+3113612us) ts_tablet_manager.cc:1151] Deleting tablet e95c63b4d99a4dd0b2c0fa384ca9d552
1211 20:52:42.041183 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:42.041291 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3049929}
I20251211 20:52:42.043078 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_d87f970fdabb4d10871f55bb8193bd4d [id=052edcbde60c41449d468c8a449fc3f1]
I20251211 20:52:42.043476 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 893) took 3016 ms. Trace:
I20251211 20:52:42.043625 1165 rpcz_store.cc:276] 1211 20:52:39.027156 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.027239 (+ 83us) service_pool.cc:225] Handling call
1211 20:52:42.043350 (+3016111us) ts_tablet_manager.cc:1151] Deleting tablet d15eff49af284a38bb1babdb231a35df
1211 20:52:42.043361 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:42.043459 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2953161}
I20251211 20:52:42.044417 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 894) took 2977 ms. Trace:
I20251211 20:52:42.044559 1165 rpcz_store.cc:276] 1211 20:52:39.066835 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.066888 (+ 53us) service_pool.cc:225] Handling call
1211 20:52:42.044310 (+2977422us) ts_tablet_manager.cc:1151] Deleting tablet c97d410272a947de8b5d7fb40fe4777d
1211 20:52:42.044320 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:42.044404 (+ 84us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2954062}
I20251211 20:52:42.044061 1168 tablet_bootstrap.cc:654] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.044955 1166 tablet_bootstrap.cc:492] T 75dd170c5828409fa69bd68fdf28a72b P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:42.045102 1166 ts_tablet_manager.cc:1403] T 75dd170c5828409fa69bd68fdf28a72b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.052s user 0.004s sys 0.005s
I20251211 20:52:42.045850 1166 raft_consensus.cc:359] T 75dd170c5828409fa69bd68fdf28a72b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.046038 1166 raft_consensus.cc:385] T 75dd170c5828409fa69bd68fdf28a72b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.046098 1166 raft_consensus.cc:740] T 75dd170c5828409fa69bd68fdf28a72b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.046005 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 898) took 2620 ms. Trace:
I20251211 20:52:42.046291 1166 consensus_queue.cc:260] T 75dd170c5828409fa69bd68fdf28a72b P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.046283 1165 rpcz_store.cc:276] 1211 20:52:39.425940 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.426004 (+ 64us) service_pool.cc:225] Handling call
1211 20:52:42.045879 (+2619875us) ts_tablet_manager.cc:1151] Deleting tablet e9ead34065e240eeb1f946319537d32a
1211 20:52:42.045888 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:42.045988 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2619633}
I20251211 20:52:42.046844 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 899) took 2504 ms. Trace:
I20251211 20:52:42.046973 1165 rpcz_store.cc:276] 1211 20:52:39.542604 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.542673 (+ 69us) service_pool.cc:225] Handling call
1211 20:52:42.046732 (+2504059us) ts_tablet_manager.cc:1151] Deleting tablet 4585e05d06ef42cfb06c4ceb7524fd71
1211 20:52:42.046741 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:42.046829 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2503848}
I20251211 20:52:42.047508 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 900) took 2493 ms. Trace:
I20251211 20:52:42.047649 1165 rpcz_store.cc:276] 1211 20:52:39.553822 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.553891 (+ 69us) service_pool.cc:225] Handling call
1211 20:52:42.047412 (+2493521us) ts_tablet_manager.cc:1151] Deleting tablet 7bc03ed8c35740f58fe48ddbaa120ea0
1211 20:52:42.047421 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:42.047494 (+ 73us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2493370}
I20251211 20:52:42.048089 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 901) took 2492 ms. Trace:
I20251211 20:52:42.048209 1165 rpcz_store.cc:276] 1211 20:52:39.555707 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.555760 (+ 53us) service_pool.cc:225] Handling call
1211 20:52:42.047998 (+2492238us) ts_tablet_manager.cc:1151] Deleting tablet a295a24e81f14e5fa126a4a58ffd08cf
1211 20:52:42.048006 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:42.048074 (+ 68us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2492053}
I20251211 20:52:42.048640 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 902) took 2491 ms. Trace:
I20251211 20:52:42.048750 1165 rpcz_store.cc:276] 1211 20:52:39.557363 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.557415 (+ 52us) service_pool.cc:225] Handling call
1211 20:52:42.048545 (+2491130us) ts_tablet_manager.cc:1151] Deleting tablet 6ad6ab53aad14cc2b6d201acfecf6723
1211 20:52:42.048554 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:42.048628 (+ 74us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2490969}
I20251211 20:52:42.049256 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 903) took 2489 ms. Trace:
I20251211 20:52:42.049279 2375 raft_consensus.cc:493] T 861d2d70a24946699dccd65cab05195e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.049401 1165 rpcz_store.cc:276] 1211 20:52:39.559339 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.559409 (+ 70us) service_pool.cc:225] Handling call
1211 20:52:42.049164 (+2489755us) ts_tablet_manager.cc:1151] Deleting tablet 48820386e41b47129a13bdd5768f021b
1211 20:52:42.049172 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:42.049242 (+ 70us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2489595}
I20251211 20:52:42.049430 2375 raft_consensus.cc:515] T 861d2d70a24946699dccd65cab05195e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.049893 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 904) took 2473 ms. Trace:
I20251211 20:52:42.050036 2375 leader_election.cc:290] T 861d2d70a24946699dccd65cab05195e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:42.050000 1165 rpcz_store.cc:276] 1211 20:52:39.576111 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.576171 (+ 60us) service_pool.cc:225] Handling call
1211 20:52:42.049794 (+2473623us) ts_tablet_manager.cc:1151] Deleting tablet 4585e05d06ef42cfb06c4ceb7524fd71
1211 20:52:42.049802 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:42.049878 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2473422}
I20251211 20:52:42.050614 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "861d2d70a24946699dccd65cab05195e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:42.051020 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "861d2d70a24946699dccd65cab05195e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:42.051092 982 leader_election.cc:343] T 861d2d70a24946699dccd65cab05195e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.051420 1165 tablet_replica.cc:333] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
W20251211 20:52:42.051435 983 leader_election.cc:343] T 861d2d70a24946699dccd65cab05195e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.051535 983 leader_election.cc:304] T 861d2d70a24946699dccd65cab05195e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.051770 1165 raft_consensus.cc:2243] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:42.051970 2375 raft_consensus.cc:2749] T 861d2d70a24946699dccd65cab05195e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.052330 1165 pending_rounds.cc:70] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919: Trying to abort 1 pending ops.
I20251211 20:52:42.052492 1165 pending_rounds.cc:77] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432129466204160 op_type: NO_OP noop_request { }
I20251211 20:52:42.052783 1165 raft_consensus.cc:2889] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:42.053025 1165 raft_consensus.cc:2272] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:42.053701 1165 ts_tablet_manager.cc:1916] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:42.055848 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_37fd7b6c68b64deaa3fb0ac9a1965eff [id=05826ee49d07424aa08e105c165c497c]
I20251211 20:52:42.056423 2273 raft_consensus.cc:493] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.056468 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_68b9d0d7258a4b5caa1779a4769bdd7f [id=06f467911b6f45849e99f0b6fe615163]
I20251211 20:52:42.056581 2273 raft_consensus.cc:515] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.057204 2273 leader_election.cc:290] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.057413 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_c15e1d9cc43c4d0b951d51344fb6ad09 [id=09a33d8a7c7c46c2b8b737fc9a8543b8]
I20251211 20:52:42.057559 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "645b08b33676420c96e880de3d460829" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:42.057906 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "645b08b33676420c96e880de3d460829" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:42.058084 1045 raft_consensus.cc:2468] T 645b08b33676420c96e880de3d460829 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
W20251211 20:52:42.058264 711 leader_election.cc:343] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.058491 1166 ts_tablet_manager.cc:1434] T 75dd170c5828409fa69bd68fdf28a72b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.013s user 0.001s sys 0.001s
I20251211 20:52:42.058595 710 leader_election.cc:304] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.059089 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_087c001a257049babda598744c761be8 [id=0e1a601c016d4b0e8e7d05969a4ea590]
I20251211 20:52:42.059306 2325 raft_consensus.cc:2804] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:42.059406 2325 raft_consensus.cc:493] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:42.059494 2325 raft_consensus.cc:3060] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:42.060365 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_a5b944a25f4c45cf8c70765ea43d1ac2 [id=0ee0170e570a4d9aa25ea2858e9d9e2a]
I20251211 20:52:42.062427 2325 raft_consensus.cc:515] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.063158 2325 leader_election.cc:290] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.063469 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "645b08b33676420c96e880de3d460829" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:42.063784 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "645b08b33676420c96e880de3d460829" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:42.063951 1045 raft_consensus.cc:3060] T 645b08b33676420c96e880de3d460829 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:42.066395 1166 tablet_bootstrap.cc:492] T bbe3529c301a4a65b58b634a443e1fb6 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:42.066747 1045 raft_consensus.cc:2468] T 645b08b33676420c96e880de3d460829 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
W20251211 20:52:42.064013 711 leader_election.cc:343] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.067487 710 leader_election.cc:304] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.068034 2273 raft_consensus.cc:2804] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:42.068161 2273 raft_consensus.cc:697] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:42.068344 1160 tablet_bootstrap.cc:492] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:42.068475 1160 ts_tablet_manager.cc:1403] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.038s user 0.003s sys 0.000s
I20251211 20:52:42.068416 2273 consensus_queue.cc:237] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.068607 1166 tablet_bootstrap.cc:654] T bbe3529c301a4a65b58b634a443e1fb6 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.069144 1160 raft_consensus.cc:359] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.069283 1160 raft_consensus.cc:385] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.069321 1160 raft_consensus.cc:740] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.069479 1160 consensus_queue.cc:260] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.069931 1160 ts_tablet_manager.cc:1434] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251211 20:52:42.070194 1160 tablet_bootstrap.cc:492] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:42.073750 1160 tablet_bootstrap.cc:654] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.076365 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_87f092378445464ab48c155fae469f29 [id=0f029363985443529615cdd1f8bbd78d]
I20251211 20:52:42.077662 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_b0ac4ccf9e224d32af03dcaa1b97330a [id=10bf2a53ba594b5e8b8a37d2ba0235b9]
I20251211 20:52:42.079064 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_defdeab28749411590cf75b5251f604b [id=11352566d1744193a40a8bb605aa3912]
I20251211 20:52:42.080047 1160 tablet_bootstrap.cc:492] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:42.080242 1160 ts_tablet_manager.cc:1403] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.001s
I20251211 20:52:42.081007 1160 raft_consensus.cc:359] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.081213 1160 raft_consensus.cc:385] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.081290 1160 raft_consensus.cc:740] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.081547 1160 consensus_queue.cc:260] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.083870 1160 ts_tablet_manager.cc:1434] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251211 20:52:42.084214 1160 tablet_bootstrap.cc:492] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:42.085080 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_434ed373bf494a40b55b1f275d648d28 [id=1224a8665eaf45c1a4ab549024bb4200]
I20251211 20:52:42.085865 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_06dc3002fa354d78ab4d5a90fd0fb3ab [id=13db407cd0aa40889e554244cbcc3616]
I20251211 20:52:42.086617 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_4721ef8d42714525ba28461e11e30dec [id=146fd60d3483482c8f46de7c44f3cc90]
I20251211 20:52:42.087273 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_25ddd02b2a6640c4a6898d1796553a8b [id=14985ae63d8740f4b692dea1578be3ce]
I20251211 20:52:42.088079 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_c1f9dca5c2144d50b67681dcc6800e77 [id=14fa8debfe5443ce81587f02dfa15847]
I20251211 20:52:42.088618 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_227fbf051d5643528315ca00194583bc [id=1515561ab78d46f6b5aa37a20d90d3f5]
I20251211 20:52:42.090006 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_fa3f2f115cd043a9aa6be2b070d320cb [id=15fad5e1cf8d4ea08dae8275a29a11dd]
I20251211 20:52:42.090325 1166 tablet_bootstrap.cc:492] T bbe3529c301a4a65b58b634a443e1fb6 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:42.090489 1166 ts_tablet_manager.cc:1403] T bbe3529c301a4a65b58b634a443e1fb6 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.005s
I20251211 20:52:42.090934 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_ee37a9ef24e548ebb2b6b55b54fa6f91 [id=162bfc57131c46c9bbf360299e331097]
I20251211 20:52:42.091325 1160 tablet_bootstrap.cc:654] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.091239 1166 raft_consensus.cc:359] T bbe3529c301a4a65b58b634a443e1fb6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.091406 1166 raft_consensus.cc:385] T bbe3529c301a4a65b58b634a443e1fb6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.091459 1166 raft_consensus.cc:740] T bbe3529c301a4a65b58b634a443e1fb6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.091717 1166 consensus_queue.cc:260] T bbe3529c301a4a65b58b634a443e1fb6 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.092833 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_9fb97b3359de4806bf0557739a5da4fd [id=17c25e0db2f9436688a7338067e09aac]
I20251211 20:52:42.095803 1166 ts_tablet_manager.cc:1434] T bbe3529c301a4a65b58b634a443e1fb6 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251211 20:52:42.096216 2375 raft_consensus.cc:493] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.096323 2375 raft_consensus.cc:515] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.096896 2375 leader_election.cc:290] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:42.097926 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_731826f26f9940b8afe425d694b642d9 [id=17dd9cea9ac844128ff9f82adb3f919f]
I20251211 20:52:42.098436 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_e00092673b0e4f09b8d7eec63e1ee034 [id=18497e06d7254441a4655ce1cdc5a9bf]
I20251211 20:52:42.098440 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d9cf0d92ccc7419cb6e7694e8e5c7bfb" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:42.098883 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_2cd4051c3df74bbc9b5d715b770ee926 [id=188ac848ba6b41e88df38554246be155]
W20251211 20:52:42.099052 982 leader_election.cc:343] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.099603 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d9cf0d92ccc7419cb6e7694e8e5c7bfb" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:42.100109 983 leader_election.cc:343] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.100215 983 leader_election.cc:304] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.100759 2375 raft_consensus.cc:2749] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.100793 1166 tablet_bootstrap.cc:492] T 04086eaa32a74edd84235d3b28f9f965 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:42.102654 1166 tablet_bootstrap.cc:654] T 04086eaa32a74edd84235d3b28f9f965 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.099706 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_e6f3a819faad48aabfad3a235c3c1b95 [id=18f8d722faaa4293a112525ff0c96216]
I20251211 20:52:42.104204 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_d8d764377e524faf931940bfe23100be [id=198db6e18dcf4f65bde2405f4f0bad0e]
I20251211 20:52:42.104368 1168 tablet_bootstrap.cc:492] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:42.104635 1168 ts_tablet_manager.cc:1403] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.068s user 0.003s sys 0.004s
I20251211 20:52:42.105639 1168 raft_consensus.cc:359] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.105919 1168 raft_consensus.cc:385] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.106038 1168 raft_consensus.cc:740] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.106359 1168 consensus_queue.cc:260] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.109810 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_574610b44ec44cbfa5a7071d1956f2f6 [id=1a595087add94970af8aca08c75bd834]
I20251211 20:52:42.110483 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_868d96d23ca040158aaea84effd252cb [id=1aaf20327cd24cc682293c44cb5295f2]
I20251211 20:52:42.110941 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_948151057bf94b75a251b99e3906f4a4 [id=1ab6db436e504bfabeb44e435a5453b8]
I20251211 20:52:42.111387 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_2eeb4b33f8424175a9e15a54e38099a2 [id=1b0b021139bc4dc4b08a90403f2d58d0]
I20251211 20:52:42.112120 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_91f5eb149cce4e40a34b953dac528702 [id=1bb8d23d94f44ee88ef0aa71624480db]
I20251211 20:52:42.112731 914 raft_consensus.cc:3060] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:42.112869 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_3482fbfc5b144a7198a7d01aa003cb6b [id=1bdaf5ce0f6b4b99a038f9f9efb828f6]
I20251211 20:52:42.113770 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_d887c953026046dc883ede774a58fd8e [id=1d8a79f716a0447cb0393f4759178598]
I20251211 20:52:42.114362 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_6cce646a13f6405c96fd61f193add5fa [id=1ea52ab9ffc74cb18c6e24e4f4d1cf88]
I20251211 20:52:42.115201 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_4b444339f63048179e9b53a3aec9173c [id=1f67fc9412d2431ab260b87363248dd7]
I20251211 20:52:42.115643 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_b94ece9030c84d37a4ed8d28c2a12a91 [id=2026c3e019fb4c85bb9911107c1cd648]
I20251211 20:52:42.115762 1168 ts_tablet_manager.cc:1434] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20251211 20:52:42.119637 2273 raft_consensus.cc:493] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.119854 2273 raft_consensus.cc:515] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.120481 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_51015ac894894382b4c0d1931b24f4d4 [id=2118ffd1623f491e95ecf3719fa44130]
I20251211 20:52:42.120832 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5710fc2f24764f818f7e5c86c6fd985e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:42.120993 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5710fc2f24764f818f7e5c86c6fd985e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:42.121179 1045 raft_consensus.cc:2468] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
W20251211 20:52:42.121352 711 leader_election.cc:343] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.121652 710 leader_election.cc:304] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.121821 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_6460313002684bba93b30ad8631f0f49 [id=22347b15537d451d9927a17e8e5a202a]
I20251211 20:52:42.121974 2325 raft_consensus.cc:2804] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:42.122090 2325 raft_consensus.cc:493] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:42.122193 2325 raft_consensus.cc:3060] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:42.122421 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_9fa0ba597863437e94957c10e3a68b4b [id=24012f9fe7eb4ba4ac585e3dea3b1425]
I20251211 20:52:42.123644 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_e9b10d4bc5db49289ea753470f1585a6 [id=26d30d9b3c3d46fd9fbffa0f0124a63e]
I20251211 20:52:42.125602 2325 raft_consensus.cc:515] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.126708 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5710fc2f24764f818f7e5c86c6fd985e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:42.126837 1160 tablet_bootstrap.cc:492] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:42.127192 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_440d3f9c240c448bb5bf5bfbdaf35d00 [id=276a75404b564adcb63880b9bbed2359]
I20251211 20:52:42.127225 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5710fc2f24764f818f7e5c86c6fd985e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
W20251211 20:52:42.127328 711 leader_election.cc:343] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.127413 1045 raft_consensus.cc:3060] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:42.127663 1160 ts_tablet_manager.cc:1403] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.043s user 0.006s sys 0.004s
I20251211 20:52:42.127964 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_d812d6f6c90842e4821d33226a54ec68 [id=276b9b5c545849dcb1ed38324b7ad540]
I20251211 20:52:42.128443 1160 raft_consensus.cc:359] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.128608 1160 raft_consensus.cc:385] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.128669 1160 raft_consensus.cc:740] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.128868 1160 consensus_queue.cc:260] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.129225 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_7730b7f512954bcab4c83c9f272d9a59 [id=2783ef72c8944e8188a8646565a6df13]
I20251211 20:52:42.129346 1160 ts_tablet_manager.cc:1434] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:42.130002 1045 raft_consensus.cc:2468] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:42.130584 710 leader_election.cc:304] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.130996 2286 raft_consensus.cc:2804] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:42.131129 2286 raft_consensus.cc:697] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:42.131389 2286 consensus_queue.cc:237] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.135068 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_7506ffecc2b64648a9ba736e5298ddf4 [id=279f39d718ba45318f52f95e8aa13652]
I20251211 20:52:42.135615 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_d87c2c33f4ff4aee9ff82b0653d4cc43 [id=2876f1fc09634b03afe5664e82c017d6]
I20251211 20:52:42.136291 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_819c4206e91e41358ac71e82185b4140 [id=28be836b71ed4107af7adc3fcb948b5e]
I20251211 20:52:42.136783 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_595d8c100e69422ab5e6d44cce9e733c [id=28f3ebe9b7a64b5e89404b0407b0e5ae]
I20251211 20:52:42.137254 1160 tablet_bootstrap.cc:492] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:42.137254 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_c504f8f3796f43b697488bd604268543 [id=2af76b9d32094a4ca656060a4478e08a]
I20251211 20:52:42.138866 1160 tablet_bootstrap.cc:654] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.139456 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_0d3816dfd69844518591138a3a844a9a [id=2b6bb7b58170467abe6463a912f608c8]
I20251211 20:52:42.139685 2325 leader_election.cc:290] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.139981 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]
I20251211 20:52:42.140487 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_8c30c4151d484db0bb9e2b9ca262248d [id=2d489cd7d73e403d84bf10960e6a2dfb]
I20251211 20:52:42.144742 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_b8ccf690baa544a8a6f41b0c6eea6b38 [id=2ee7c64dbda8471ea684bd515d4f2f4a]
I20251211 20:52:42.147141 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_bd71cd88b0354c88aa03799a4362f1b1 [id=2f4a7d71bf18442fa8f3ed7421512ff2]
I20251211 20:52:42.147740 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_3c3cd48c8fe5411e96cdf811d492925d [id=30620a01c27f456bac05fa7c02a2cb4d]
I20251211 20:52:42.148254 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_907c596bdddc4edca25d14cbacebb23f [id=321235280d1d4fa0bae5cc0333860f85]
I20251211 20:52:42.149380 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_12f41f708f1949598d6176d89777aa69 [id=3309205faabf4a42acadc43766e4f57f]
I20251211 20:52:42.150151 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_6e9f3ca164384cc8ac4501deb3f0c6aa [id=342c2469d6e74e3e9217236e3aa863d6]
I20251211 20:52:42.152660 2273 leader_election.cc:290] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.154331 2375 raft_consensus.cc:493] T 75dd170c5828409fa69bd68fdf28a72b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.154495 2375 raft_consensus.cc:515] T 75dd170c5828409fa69bd68fdf28a72b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.155136 2375 leader_election.cc:290] T 75dd170c5828409fa69bd68fdf28a72b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:42.156347 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75dd170c5828409fa69bd68fdf28a72b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:42.156528 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75dd170c5828409fa69bd68fdf28a72b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:42.157073 983 leader_election.cc:343] T 75dd170c5828409fa69bd68fdf28a72b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:42.157402 982 leader_election.cc:343] T 75dd170c5828409fa69bd68fdf28a72b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.157495 982 leader_election.cc:304] T 75dd170c5828409fa69bd68fdf28a72b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.157950 2375 raft_consensus.cc:2749] T 75dd170c5828409fa69bd68fdf28a72b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.158185 1166 tablet_bootstrap.cc:492] T 04086eaa32a74edd84235d3b28f9f965 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:42.158735 1166 ts_tablet_manager.cc:1403] T 04086eaa32a74edd84235d3b28f9f965 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.063s user 0.004s sys 0.000s
I20251211 20:52:42.159417 1166 raft_consensus.cc:359] T 04086eaa32a74edd84235d3b28f9f965 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.159623 1166 raft_consensus.cc:385] T 04086eaa32a74edd84235d3b28f9f965 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.159698 1166 raft_consensus.cc:740] T 04086eaa32a74edd84235d3b28f9f965 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.159945 1166 consensus_queue.cc:260] T 04086eaa32a74edd84235d3b28f9f965 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.160166 1168 tablet_bootstrap.cc:492] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:42.160594 1166 ts_tablet_manager.cc:1434] T 04086eaa32a74edd84235d3b28f9f965 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:42.160962 1166 tablet_bootstrap.cc:492] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:42.162457 1166 tablet_bootstrap.cc:654] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.163756 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_5a933aeef8dc44b18bb0cf7ddfe412f2 [id=3496dd8ce5184ac79c516278d403cbcb]
I20251211 20:52:42.164458 1168 tablet_bootstrap.cc:654] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.164777 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_76916c86042b49f487d21bf7aabfed56 [id=365cb9cb979d45dbaaff790819c90412]
I20251211 20:52:42.165307 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_bdb9aa9507d94416bbc3c8fe9cb11fa7 [id=367894d4c46a4cc6bde351a099a635b3]
I20251211 20:52:42.168620 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_5da0e9ddf96549c88878f7136d0ba415 [id=3811f8235b2f44cca283401302d613d4]
I20251211 20:52:42.169193 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_3721b5725e8c4297a392d3986a081128 [id=383392d835da4c50848068d83261cecc]
I20251211 20:52:42.171226 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_fde09a02a1664bc5bad79b2e0036b42a [id=3c1a33a40f334749b4b79833bfc6063a]
I20251211 20:52:42.172016 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_7fe80ff5483a4b0292091119980f6639 [id=3c3462a0c09748cea0d2ffb544ae2f13]
I20251211 20:52:42.172946 1160 tablet_bootstrap.cc:492] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:42.173156 1160 ts_tablet_manager.cc:1403] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.036s user 0.000s sys 0.011s
I20251211 20:52:42.173197 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_72c2c33862a149418a3c31c07caf8a37 [id=3e233f1f255847a5830bb85f65370aaa]
I20251211 20:52:42.173666 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_e3a99b043e4f420a93ebfee6aa87f2ec [id=3e7d6cefcb7d412191c6540e9ad77c2d]
I20251211 20:52:42.173949 1160 raft_consensus.cc:359] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.174201 1160 raft_consensus.cc:385] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.174288 1160 raft_consensus.cc:740] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.174530 1160 consensus_queue.cc:260] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.176374 1165 ts_tablet_manager.cc:1929] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:42.176591 1165 log.cc:1199] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/b529b38d754f46ba8cb60b19e9705256
I20251211 20:52:42.177106 1165 ts_tablet_manager.cc:1950] T b529b38d754f46ba8cb60b19e9705256 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:42.179550 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_494ee7d3e0dc4a47b47082643a3134ce [id=42422ac9805e49cbab71e262e73098b3]
I20251211 20:52:42.181313 1168 tablet_bootstrap.cc:492] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:42.181540 1168 ts_tablet_manager.cc:1403] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.002s
I20251211 20:52:42.181759 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_6bfa9e94e0b24faaa8240f4280e3d535 [id=4358b12fe36941e18e574120e6d892c8]
I20251211 20:52:42.182459 1168 raft_consensus.cc:359] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.182745 1168 raft_consensus.cc:385] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.182901 1168 raft_consensus.cc:740] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.183193 1168 consensus_queue.cc:260] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.185292 1160 ts_tablet_manager.cc:1434] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.012s user 0.000s sys 0.001s
I20251211 20:52:42.189409 1160 tablet_bootstrap.cc:492] T f737a7bfb0f44196abdc831bb22d5285 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:42.189580 1168 ts_tablet_manager.cc:1434] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20251211 20:52:42.189759 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 908) took 2483 ms. Trace:
I20251211 20:52:42.189996 1165 rpcz_store.cc:276] 1211 20:52:39.704939 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.706021 (+ 1082us) service_pool.cc:225] Handling call
1211 20:52:42.050850 (+2344829us) ts_tablet_manager.cc:1151] Deleting tablet b529b38d754f46ba8cb60b19e9705256
1211 20:52:42.051376 (+ 526us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:42.161559 (+110183us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:42.174952 (+ 13393us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:42.188665 (+ 13713us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5793,"tablet-delete.queue_time_us":2344652}
I20251211 20:52:42.190558 1165 tablet_replica.cc:333] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:42.191056 1160 tablet_bootstrap.cc:654] T f737a7bfb0f44196abdc831bb22d5285 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.198379 1168 tablet_bootstrap.cc:492] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:42.200273 1166 tablet_bootstrap.cc:492] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:42.200457 1166 ts_tablet_manager.cc:1403] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.040s user 0.003s sys 0.000s
I20251211 20:52:42.201009 1168 tablet_bootstrap.cc:654] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.201191 1166 raft_consensus.cc:359] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.201375 1166 raft_consensus.cc:385] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.201442 1166 raft_consensus.cc:740] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.201674 1166 consensus_queue.cc:260] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.202088 2385 raft_consensus.cc:493] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.202221 1166 ts_tablet_manager.cc:1434] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:42.202247 2385 raft_consensus.cc:515] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.202548 1166 tablet_bootstrap.cc:492] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:42.202947 2385 leader_election.cc:290] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:42.204193 1166 tablet_bootstrap.cc:654] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.204169 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f2d6c9781b4d4c37bc08703347963d5d" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:42.204385 777 raft_consensus.cc:2468] T f2d6c9781b4d4c37bc08703347963d5d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 0.
I20251211 20:52:42.205289 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f2d6c9781b4d4c37bc08703347963d5d" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:42.205394 982 leader_election.cc:304] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:42.205778 2375 raft_consensus.cc:2804] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:42.205884 2375 raft_consensus.cc:493] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:42.205966 2375 raft_consensus.cc:3060] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:42.208216 983 leader_election.cc:343] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.211290 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_b36aed5743884aa7bca0cb01b2af1bac [id=440072d0cbe248c381c8086eb62f1fcd]
I20251211 20:52:42.211795 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_80e3542d21b64eca991b2e81ac80ac7c [id=44f2f2bf53af4b038ced294f20db9e48]
I20251211 20:52:42.212579 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_b3b56f469dc54741924e5b55045fa626 [id=471a1295a9534492aff0ae89a5362a1a]
I20251211 20:52:42.212755 1160 tablet_bootstrap.cc:492] T f737a7bfb0f44196abdc831bb22d5285 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:42.212961 1160 ts_tablet_manager.cc:1403] T f737a7bfb0f44196abdc831bb22d5285 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.002s
I20251211 20:52:42.213074 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_8ead90e9bdef45bf8bf6c5d4c26c347a [id=475e0a1ec0344d10addbcfa42fe05b68]
I20251211 20:52:42.213557 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_2c7bb884f8b5488e9a80e5d20e75cc95 [id=4942c719263148deb2c5cb57a605672a]
I20251211 20:52:42.213778 1160 raft_consensus.cc:359] T f737a7bfb0f44196abdc831bb22d5285 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.213963 1160 raft_consensus.cc:385] T f737a7bfb0f44196abdc831bb22d5285 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.214007 1160 raft_consensus.cc:740] T f737a7bfb0f44196abdc831bb22d5285 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.214118 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_34e6ed4f13da437ea50d349b92e83bd6 [id=4a74b9a60cbb4198b55cee630b2f4bef]
I20251211 20:52:42.214238 1160 consensus_queue.cc:260] T f737a7bfb0f44196abdc831bb22d5285 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.214924 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_647731023b044ebb8954f7fdcc3faf97 [id=4b91abd6e9e94a8a941b33e62e23c083]
I20251211 20:52:42.215201 1168 tablet_bootstrap.cc:492] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:42.215384 1168 ts_tablet_manager.cc:1403] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.003s
I20251211 20:52:42.215512 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_0910ec0ec3e64c139587947350a39ae5 [id=4ce959831e384a14854ce5570dededaa]
I20251211 20:52:42.216203 2322 raft_consensus.cc:493] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.216322 2322 raft_consensus.cc:515] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.216909 2322 leader_election.cc:290] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:42.218437 847 leader_election.cc:343] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: ccc3ddd4b7f74f11a3835997cb7ae8d0
I20251211 20:52:42.218957 1168 raft_consensus.cc:359] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.219194 1168 raft_consensus.cc:385] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.219674 1160 ts_tablet_manager.cc:1434] T f737a7bfb0f44196abdc831bb22d5285 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20251211 20:52:42.219986 1160 tablet_bootstrap.cc:492] T 72e23f3033e14c9db6de8539830bac3b P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:42.221606 1160 tablet_bootstrap.cc:654] T 72e23f3033e14c9db6de8539830bac3b P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.219291 1168 raft_consensus.cc:740] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.222440 1168 consensus_queue.cc:260] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.222812 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_fffa8ba66cb04202b2116b8fd4a741de [id=4d2c08402d5845e4ae592cc28ce8d9b1]
I20251211 20:52:42.223145 1166 tablet_bootstrap.cc:492] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:42.223294 1166 ts_tablet_manager.cc:1403] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.003s
I20251211 20:52:42.223321 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_db4c0f706e0643b39d39ffa84352826d [id=4d459c42bf584068ac3851227d80331a]
W20251211 20:52:42.224624 847 leader_election.cc:343] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: ccc3ddd4b7f74f11a3835997cb7ae8d0
I20251211 20:52:42.224745 847 leader_election.cc:304] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:42.225126 2322 raft_consensus.cc:2749] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.226147 1166 raft_consensus.cc:359] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.226341 1166 raft_consensus.cc:385] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.226410 1166 raft_consensus.cc:740] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.226624 1166 consensus_queue.cc:260] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.229161 1166 ts_tablet_manager.cc:1434] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20251211 20:52:42.229418 2375 raft_consensus.cc:515] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:42.230145 711 consensus_peers.cc:597] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8308ad46551447439a248d20ed1e5922. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:42.230226 2375 leader_election.cc:290] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:42.230341 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_d0cd234de1cb4bb485da3d10b527eb65 [id=4ea51139b7824638a5ad1eaf2c0f11f8]
I20251211 20:52:42.230649 1168 ts_tablet_manager.cc:1434] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.015s user 0.000s sys 0.002s
I20251211 20:52:42.231339 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f2d6c9781b4d4c37bc08703347963d5d" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:42.231660 777 raft_consensus.cc:3060] T f2d6c9781b4d4c37bc08703347963d5d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:42.232317 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_842a98db481041b98480204da25a6419 [id=4fa6a5b9f21e41f4ae0fb953da11a51b]
I20251211 20:52:42.232725 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_6b8536dae8ce48be97d778489d1da99c [id=50493a9594aa423cb79ed4f4235a5e4a]
I20251211 20:52:42.233511 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_b7bf8c5ad5304175a1f2f6a03887f725 [id=50d88b663a4d4891b113aee9c5906699]
I20251211 20:52:42.234535 777 raft_consensus.cc:2468] T f2d6c9781b4d4c37bc08703347963d5d P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 1.
I20251211 20:52:42.234637 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_e870d2f987ff44e6b91ebe0205f6fe9e [id=53b3365f07864ac1a3cd47482bb6377a]
I20251211 20:52:42.235256 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_f0a70914acc94231a4e84155c3e4d13f [id=56c070ad80624c698ebe2e01b4929b6e]
I20251211 20:52:42.235231 982 leader_election.cc:304] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:42.235924 1166 tablet_bootstrap.cc:492] T 6b7dd8232d974a43b99ec9b28075dc8d P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:42.237182 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f2d6c9781b4d4c37bc08703347963d5d" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:42.238000 983 leader_election.cc:343] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.244004 1168 tablet_bootstrap.cc:492] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:42.245846 1168 tablet_bootstrap.cc:654] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.246744 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_7215dcc06d384c7d94b29ff2ea8ee8ee [id=593f8b9b01134fb09a1c7232509635bb]
I20251211 20:52:42.248068 1166 tablet_bootstrap.cc:654] T 6b7dd8232d974a43b99ec9b28075dc8d P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.249805 2375 raft_consensus.cc:2804] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:42.249974 2375 raft_consensus.cc:697] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 LEADER]: Becoming Leader. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Running, Role: LEADER
I20251211 20:52:42.250157 1160 tablet_bootstrap.cc:492] T 72e23f3033e14c9db6de8539830bac3b P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:42.250289 1160 ts_tablet_manager.cc:1403] T 72e23f3033e14c9db6de8539830bac3b P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.000s
I20251211 20:52:42.250211 2375 consensus_queue.cc:237] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.250996 1160 raft_consensus.cc:359] T 72e23f3033e14c9db6de8539830bac3b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.251210 1160 raft_consensus.cc:385] T 72e23f3033e14c9db6de8539830bac3b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.251273 1160 raft_consensus.cc:740] T 72e23f3033e14c9db6de8539830bac3b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.251490 1160 consensus_queue.cc:260] T 72e23f3033e14c9db6de8539830bac3b P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:42.253403 711 consensus_peers.cc:597] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:42.256136 1160 ts_tablet_manager.cc:1434] T 72e23f3033e14c9db6de8539830bac3b P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20251211 20:52:42.256541 1160 tablet_bootstrap.cc:492] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:42.258239 1160 tablet_bootstrap.cc:654] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.262223 2338 raft_consensus.cc:493] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.262398 2338 raft_consensus.cc:515] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.263056 2338 leader_election.cc:290] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.265264 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb4004212441473483a209d23c0800ce" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:42.269912 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_4b87bd2fa22f4d59b01baeca62ce98f8 [id=5a1c2ebabb5c49d5afe8f668d2bfa3e8]
W20251211 20:52:42.270666 710 leader_election.cc:343] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: bb4004212441473483a209d23c0800ce
I20251211 20:52:42.270888 1160 tablet_bootstrap.cc:492] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:42.271023 1160 ts_tablet_manager.cc:1403] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.001s
I20251211 20:52:42.271107 1165 raft_consensus.cc:2243] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:42.271832 1165 raft_consensus.cc:2272] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:42.272472 1168 tablet_bootstrap.cc:492] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:42.272547 1165 ts_tablet_manager.cc:1916] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:42.275301 711 leader_election.cc:343] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.283283 711 leader_election.cc:304] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.283639 2429 raft_consensus.cc:2749] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.284292 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_3db64a0e2af24640883c6413a68001b0 [id=5ccf927de35144b9b9cc9324638fe9c4]
I20251211 20:52:42.284914 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5]
I20251211 20:52:42.285315 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_015ac77359ed47db98e074fa602a0141 [id=5db43d4090b947ffa3bdad62184f1db9]
I20251211 20:52:42.285732 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_b8e180cc27234500a1c4c7bdd2b93ac8 [id=5dfca04de96e404a802046007f6a2afb]
I20251211 20:52:42.286638 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_b7695e75618444e5a8e1ae1e2fd02d5b [id=5f737bc407a14f83a660f02af3be8e39]
I20251211 20:52:42.287510 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_18b21232dfa74378b27defe672bb2574 [id=602d7eee703948d99a0edcb7cac0651b]
I20251211 20:52:42.287520 1168 ts_tablet_manager.cc:1403] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.044s user 0.001s sys 0.004s
I20251211 20:52:42.288519 1168 raft_consensus.cc:359] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.288898 1168 raft_consensus.cc:385] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.289032 1168 raft_consensus.cc:740] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.289353 1168 consensus_queue.cc:260] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.290032 1168 ts_tablet_manager.cc:1434] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:42.290385 1168 tablet_bootstrap.cc:492] T b30d2054316b49fa8096345527a45d87 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:42.292268 1168 tablet_bootstrap.cc:654] T b30d2054316b49fa8096345527a45d87 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.296416 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_bae5e6fe430247c99eae40ae86192b39 [id=6212dd14e00341be8a2c71adcf7f4c61]
I20251211 20:52:42.296586 2322 raft_consensus.cc:493] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.296733 2322 raft_consensus.cc:515] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.297422 2322 leader_election.cc:290] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:42.298673 847 leader_election.cc:343] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 2d849c62f7434099a16f62c2a2fc5e85
W20251211 20:52:42.299084 847 leader_election.cc:343] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 2d849c62f7434099a16f62c2a2fc5e85
I20251211 20:52:42.299190 847 leader_election.cc:304] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:42.299628 2322 raft_consensus.cc:2749] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.301092 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_0e243018f79e4e358c4f844e6ba51c35 [id=655a160341774729972a39fb7220a685]
I20251211 20:52:42.301821 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_46118198e87e4c158f00055e6867664b [id=6583487fed754a28a5bb4f14cdbef5a1]
I20251211 20:52:42.302657 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_40d581fe03ef48d1b5639a95ffc00b7b [id=66810eaf071c4613a4d30dfa016788a8]
I20251211 20:52:42.303289 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_b1d96cf6ac344427a92df55d1dfcee90 [id=679d27d782fb480086443c78df21fa52]
I20251211 20:52:42.303890 1160 raft_consensus.cc:359] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.304064 1160 raft_consensus.cc:385] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.304113 1160 raft_consensus.cc:740] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.304301 1160 consensus_queue.cc:260] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.308656 2429 raft_consensus.cc:493] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.308841 2429 raft_consensus.cc:515] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.309448 2429 leader_election.cc:290] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.310608 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "260bdeb73439437cbf234d20f7264ab3" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:42.311007 710 leader_election.cc:343] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 260bdeb73439437cbf234d20f7264ab3
W20251211 20:52:42.311239 711 leader_election.cc:343] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.311362 711 leader_election.cc:304] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.311949 2429 raft_consensus.cc:2749] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.315955 1168 tablet_bootstrap.cc:492] T b30d2054316b49fa8096345527a45d87 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:42.316128 1168 ts_tablet_manager.cc:1403] T b30d2054316b49fa8096345527a45d87 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.026s user 0.007s sys 0.006s
I20251211 20:52:42.316900 1168 raft_consensus.cc:359] T b30d2054316b49fa8096345527a45d87 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.317124 1168 raft_consensus.cc:385] T b30d2054316b49fa8096345527a45d87 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.317189 1168 raft_consensus.cc:740] T b30d2054316b49fa8096345527a45d87 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.317211 1160 ts_tablet_manager.cc:1434] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.046s user 0.002s sys 0.000s
I20251211 20:52:42.317255 2273 raft_consensus.cc:493] T 4df68d1b6a91458e9f212ae81aa25438 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.317395 2273 raft_consensus.cc:515] T 4df68d1b6a91458e9f212ae81aa25438 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.318012 2273 leader_election.cc:290] T 4df68d1b6a91458e9f212ae81aa25438 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.317400 1168 consensus_queue.cc:260] T b30d2054316b49fa8096345527a45d87 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:42.319291 710 leader_election.cc:343] T 4df68d1b6a91458e9f212ae81aa25438 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 4df68d1b6a91458e9f212ae81aa25438
I20251211 20:52:42.320799 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4df68d1b6a91458e9f212ae81aa25438" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:42.322158 711 leader_election.cc:343] T 4df68d1b6a91458e9f212ae81aa25438 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.322299 711 leader_election.cc:304] T 4df68d1b6a91458e9f212ae81aa25438 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.322753 1168 ts_tablet_manager.cc:1434] T b30d2054316b49fa8096345527a45d87 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20251211 20:52:42.323089 1168 tablet_bootstrap.cc:492] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:42.323064 2325 raft_consensus.cc:2749] T 4df68d1b6a91458e9f212ae81aa25438 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.324831 1168 tablet_bootstrap.cc:654] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.326500 1160 tablet_bootstrap.cc:492] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:42.333797 1160 tablet_bootstrap.cc:654] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.334102 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_beb19ebcfc3b426e9aa9f70a68445f72 [id=67d5a2d2ef5a4e329c5ffd80756ef8f2]
I20251211 20:52:42.334765 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_6402a801d48e4a53827b1b62b6cdaba1 [id=6876565d3a9c48ce841b830cf75139ff]
I20251211 20:52:42.336155 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_f89cac674abe45038710fc4b675afbc8 [id=687d8fab1ee24655b68058860cdd4355]
I20251211 20:52:42.336730 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_cec07d512c8049e581be33a0214fad11 [id=697f23a617f64762bf406b386cde7473]
I20251211 20:52:42.337571 1168 tablet_bootstrap.cc:492] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:42.337726 1168 ts_tablet_manager.cc:1403] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20251211 20:52:42.338455 1168 raft_consensus.cc:359] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.338658 1168 raft_consensus.cc:385] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.338748 1168 raft_consensus.cc:740] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.338963 1168 consensus_queue.cc:260] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.339768 1166 tablet_bootstrap.cc:492] T 6b7dd8232d974a43b99ec9b28075dc8d P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:42.338567 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_6f8c133c7a364b93af2c83d8911a0ac3 [id=6a21c38ee7224a02b937df088494f5d8]
I20251211 20:52:42.339936 1166 ts_tablet_manager.cc:1403] T 6b7dd8232d974a43b99ec9b28075dc8d P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.104s user 0.003s sys 0.003s
I20251211 20:52:42.340355 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_99917cb4c70840e19004ad381746d77e [id=6abb187bfd214e91b2e11267065a2841]
I20251211 20:52:42.340732 1166 raft_consensus.cc:359] T 6b7dd8232d974a43b99ec9b28075dc8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.340912 1166 raft_consensus.cc:385] T 6b7dd8232d974a43b99ec9b28075dc8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.340974 1166 raft_consensus.cc:740] T 6b7dd8232d974a43b99ec9b28075dc8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.342068 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_4e2726de74fd437796c8e0e6dd8b3af0 [id=6bca2b1cbc244d34b21efa75f651d35f]
I20251211 20:52:42.342082 1168 ts_tablet_manager.cc:1434] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:42.342393 1168 tablet_bootstrap.cc:492] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:42.344225 1168 tablet_bootstrap.cc:654] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.349160 1166 consensus_queue.cc:260] T 6b7dd8232d974a43b99ec9b28075dc8d P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.349938 1160 tablet_bootstrap.cc:492] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:42.350220 1160 ts_tablet_manager.cc:1403] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.000s
I20251211 20:52:42.351159 1160 raft_consensus.cc:359] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.351413 1160 raft_consensus.cc:385] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.351506 1160 raft_consensus.cc:740] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.351894 1160 consensus_queue.cc:260] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.355037 1160 ts_tablet_manager.cc:1434] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251211 20:52:42.355419 1160 tablet_bootstrap.cc:492] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:42.356354 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_11314b6f726648c3bda96c9c1c7da29e [id=6ce96b52af404551a7e7e5186d54ee8a]
I20251211 20:52:42.358314 1160 tablet_bootstrap.cc:654] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.359145 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_e02933dd91ce4fe7a40d1c3b2f9b2d4b [id=6d42f260246e420d8818b3841926302b]
I20251211 20:52:42.359745 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_d71f88120b9d4408a1b04988573d137d [id=6dae16bd96534a0c89619742c262e081]
I20251211 20:52:42.361078 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_53c0c6163e2447959bc8eb3303c1685f [id=6e0536d7748d456a9de9df3ba7e914be]
I20251211 20:52:42.361243 2322 raft_consensus.cc:493] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.361372 2322 raft_consensus.cc:515] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.361639 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_92281c5fcfee4355a79215ffedeee588 [id=6e5fe0a78e4b48f98dac028802f1ddf4]
I20251211 20:52:42.361994 2322 leader_election.cc:290] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.362057 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_92efd469dd324f7d886e2cbc90ece61c [id=6e7fb3271e0c48c0bb9d56fe4fccee3a]
I20251211 20:52:42.362443 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_339c5a45d0294fe7adf34fdbfa8e3d98 [id=6ef058fada9941629e0ea4d1906abdac]
I20251211 20:52:42.362900 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_78f3b1065a0a43d891bb775169c0eaca [id=6ef691749df24251a7a7c5919c219754]
I20251211 20:52:42.363389 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_9e7258165cbf458c83d7cd4874a8755f [id=6f1e5458bbef44fdb8ae111900bf5159]
W20251211 20:52:42.363385 847 leader_election.cc:343] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: bb112b6431af42a6a7b328dfdefbc2a2
I20251211 20:52:42.364393 1166 ts_tablet_manager.cc:1434] T 6b7dd8232d974a43b99ec9b28075dc8d P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.024s user 0.002s sys 0.000s
W20251211 20:52:42.364500 847 leader_election.cc:343] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: bb112b6431af42a6a7b328dfdefbc2a2
I20251211 20:52:42.364630 847 leader_election.cc:304] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:42.364832 1166 tablet_bootstrap.cc:492] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:42.365067 2322 raft_consensus.cc:2749] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.365351 2273 consensus_queue.cc:1048] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20251211 20:52:42.366545 1166 tablet_bootstrap.cc:654] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.380569 2322 raft_consensus.cc:493] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.380756 2322 raft_consensus.cc:515] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.381438 2322 leader_election.cc:290] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:42.382817 847 leader_election.cc:343] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 4201da4241aa48dfb670a713e4453ca5
W20251211 20:52:42.384370 847 leader_election.cc:343] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 4201da4241aa48dfb670a713e4453ca5
I20251211 20:52:42.384500 847 leader_election.cc:304] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:42.384596 1168 tablet_bootstrap.cc:492] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:42.384730 1168 ts_tablet_manager.cc:1403] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.042s user 0.019s sys 0.011s
I20251211 20:52:42.385015 2322 raft_consensus.cc:2749] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.385398 1160 tablet_bootstrap.cc:492] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:42.385581 1160 ts_tablet_manager.cc:1403] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.000s
I20251211 20:52:42.385499 1168 raft_consensus.cc:359] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.385676 1168 raft_consensus.cc:385] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.385740 1168 raft_consensus.cc:740] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.385934 1168 consensus_queue.cc:260] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.386641 1160 raft_consensus.cc:359] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.386868 1160 raft_consensus.cc:385] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.386958 1160 raft_consensus.cc:740] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.387197 1160 consensus_queue.cc:260] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.392530 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_1f11bf68fb18474c9a3cce63119e7e5f [id=713c1b5f22c24b918617f807b319b1c8]
I20251211 20:52:42.392613 1160 ts_tablet_manager.cc:1434] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20251211 20:52:42.392927 1350 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 2 FOLLOWER]: Refusing update from remote peer 11f5f94732f049d0be4c97cebfffcb05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1197. Preceding OpId from leader: term: 2 index: 1198. (index mismatch)
I20251211 20:52:42.393013 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_95bf875841aa45c8bf54c51b5839eb51 [id=71660a3234e74d639bca2b039ff420c3]
I20251211 20:52:42.393755 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_52e650ce0b824a41a39fcd4f1b8ba455 [id=748adb1caf074cf18655c4d7af403663]
I20251211 20:52:42.393800 2405 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1198, Last known committed idx: 1197, Time since last communication: 0.000s
I20251211 20:52:42.395720 1168 ts_tablet_manager.cc:1434] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20251211 20:52:42.395887 914 raft_consensus.cc:3060] T c7a2610f20df4c1ca81eae2fc7dba6bb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:42.398742 1168 tablet_bootstrap.cc:492] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:42.400194 1168 tablet_bootstrap.cc:654] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.402136 2273 consensus_queue.cc:1048] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:42.402240 1166 tablet_bootstrap.cc:492] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:42.402382 1166 ts_tablet_manager.cc:1403] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.001s
I20251211 20:52:42.406076 2375 raft_consensus.cc:493] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.406250 2375 raft_consensus.cc:515] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.406863 2375 leader_election.cc:290] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:42.407505 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4d2b7e2a0abe4592bc806d5ef3f894d8" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:42.407968 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4d2b7e2a0abe4592bc806d5ef3f894d8" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:42.408288 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_bd9633c55bd242c18aac3a7f2c18a329 [id=767f50bee5464316a18f73e64bab817a]
I20251211 20:52:42.407507 1160 tablet_bootstrap.cc:492] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:42.409092 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_6d9aa092c758405fa974ff1675c60a8a [id=78f84aeb9f40475f91cdf41fd040b400]
W20251211 20:52:42.409579 983 leader_election.cc:343] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.409914 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_c1fc29564f5e4ee5b8f8cf8b19c744f0 [id=7a100133ee30444794457344356629ce]
W20251211 20:52:42.410100 982 leader_election.cc:343] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.410213 982 leader_election.cc:304] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.410301 1160 tablet_bootstrap.cc:654] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.410595 2385 raft_consensus.cc:493] T 04086eaa32a74edd84235d3b28f9f965 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.410759 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_890c9027415a44ccaf95c0adede51992 [id=7bc8a2b91fe54b91b6594b371fea7bd3]
I20251211 20:52:42.410778 2385 raft_consensus.cc:515] T 04086eaa32a74edd84235d3b28f9f965 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.411365 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_c64075b948a646458740b55e6970fd94 [id=7c5b2e33fa1c4b12b3941577360908a8]
I20251211 20:52:42.411391 2385 leader_election.cc:290] T 04086eaa32a74edd84235d3b28f9f965 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:42.411520 2385 raft_consensus.cc:2749] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.412220 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "04086eaa32a74edd84235d3b28f9f965" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:42.412748 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "04086eaa32a74edd84235d3b28f9f965" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:42.413056 2322 raft_consensus.cc:493] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.413194 2322 raft_consensus.cc:515] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:42.413434 983 leader_election.cc:343] T 04086eaa32a74edd84235d3b28f9f965 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:42.413666 982 leader_election.cc:343] T 04086eaa32a74edd84235d3b28f9f965 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.413765 2322 leader_election.cc:290] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.413765 982 leader_election.cc:304] T 04086eaa32a74edd84235d3b28f9f965 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:42.414425 416 external_mini_cluster.cc:1715] Unable to check leaks on /tmp/dist-test-task5TWsQc/build/asan/bin/kudu pid 564: Timed out: CheckLeaks RPC to 127.0.104.60:42087 timed out after 1.000s (SENT)
I20251211 20:52:42.414608 416 external_mini_cluster.cc:1658] Killing /tmp/dist-test-task5TWsQc/build/asan/bin/kudu with pid 564
I20251211 20:52:42.414641 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cce44b80c97948e1af7ac68e77cccfd6" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:42.414803 777 raft_consensus.cc:2468] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 0.
I20251211 20:52:42.415239 847 leader_election.cc:304] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters:
I20251211 20:52:42.415504 2375 raft_consensus.cc:2749] T 04086eaa32a74edd84235d3b28f9f965 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.415521 2322 raft_consensus.cc:2804] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:42.415613 2322 raft_consensus.cc:493] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:42.415670 2322 raft_consensus.cc:3060] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:42.415988 847 leader_election.cc:343] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: cce44b80c97948e1af7ac68e77cccfd6
I20251211 20:52:42.416885 1166 raft_consensus.cc:359] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.417742 1166 raft_consensus.cc:385] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.417995 1166 raft_consensus.cc:740] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.418125 1160 tablet_bootstrap.cc:492] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:42.418458 1166 consensus_queue.cc:260] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.418637 2451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11f5f94732f049d0be4c97cebfffcb05. Latest consensus state: current_term: 2 leader_uuid: "11f5f94732f049d0be4c97cebfffcb05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } } }
I20251211 20:52:42.419015 2451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [sys.catalog]: This master's current role is: FOLLOWER
I20251211 20:52:42.421228 1166 ts_tablet_manager.cc:1434] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.019s user 0.001s sys 0.001s
I20251211 20:52:42.421556 1166 tablet_bootstrap.cc:492] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:42.424510 1166 tablet_bootstrap.cc:654] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:42.425196 1295 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.104.60:42087 timed out after 0.548s (SENT)
I20251211 20:52:42.427013 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_68c2d3a1dc614ca4bcf28180d6e4faf8 [id=7e7414fd3d1e4ec2b7372f5783297766]
I20251211 20:52:42.428557 1168 tablet_bootstrap.cc:492] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:42.428793 1168 ts_tablet_manager.cc:1403] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.000s
I20251211 20:52:42.418265 1160 ts_tablet_manager.cc:1403] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20251211 20:52:42.430447 1168 raft_consensus.cc:359] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.430961 1168 raft_consensus.cc:385] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.431129 1168 raft_consensus.cc:740] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.431452 1168 consensus_queue.cc:260] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.432142 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_c993b3973aa64eabb16fe02f9514b161 [id=7f24fb8df80943cb836588beda1de7cc]
I20251211 20:52:42.432325 1168 ts_tablet_manager.cc:1434] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251211 20:52:42.432404 1160 raft_consensus.cc:359] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.432596 1168 tablet_bootstrap.cc:492] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:42.432618 1160 raft_consensus.cc:385] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.432665 1160 raft_consensus.cc:740] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.433365 1160 consensus_queue.cc:260] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.433988 1168 tablet_bootstrap.cc:654] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.434536 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_cce138cd7e6f4a91aeb7632af1b61ade [id=7f521f77d3234e5a9b9612598e00fc94]
I20251211 20:52:42.438845 1165 ts_tablet_manager.cc:1929] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:42.438985 1165 log.cc:1199] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/14e2650f8ec1482ea9511b989fdc963c
I20251211 20:52:42.439312 1165 ts_tablet_manager.cc:1950] T 14e2650f8ec1482ea9511b989fdc963c P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:42.442553 2322 raft_consensus.cc:515] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.443295 2322 leader_election.cc:290] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Requested vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.444455 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_23402fab961249179b57df4d79b53a18 [id=7f966d5cdad3423bb770414802a3cf7b]
I20251211 20:52:42.444550 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cce44b80c97948e1af7ac68e77cccfd6" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:42.444725 777 raft_consensus.cc:3060] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:42.444922 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_fadb0061961d4706af19ddfb2f79d521 [id=8062ca5194e445a88f7dca2684519d16]
I20251211 20:52:42.445286 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 909) took 2740 ms. Trace:
I20251211 20:52:42.445505 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_31a95f7c09a34c4585eef1f7914a97a7 [id=8151557842dc4ca8ab0d1b3d663862ee]
I20251211 20:52:42.445379 1165 rpcz_store.cc:276] 1211 20:52:39.705067 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.707672 (+ 2605us) service_pool.cc:225] Handling call
1211 20:52:42.190460 (+2482788us) ts_tablet_manager.cc:1151] Deleting tablet 14e2650f8ec1482ea9511b989fdc963c
1211 20:52:42.190471 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:42.271442 (+ 80971us) spinlock_profiling.cc:243] Waited 76.5 ms on lock 0x6160000e1754. stack: 00007fc66457bee6 00007fc66457bca0 00007fc6635573ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fc6704461e0 00007fc6717940b3 00007fc674354757 00007fc674387744 00007fc674368240 00007fc674367fcd 0000000000a8970c 00007fc6645c220f 00007fc6645c552b 00007fc6645c53a8
1211 20:52:42.347692 (+ 76250us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:42.438686 (+ 90994us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:42.445262 (+ 6576us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":175842048,"tablet-delete.queue_time_us":2482599}
I20251211 20:52:42.445853 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 910) took 2728 ms. Trace:
I20251211 20:52:42.445942 1165 rpcz_store.cc:276] 1211 20:52:39.717294 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.717340 (+ 46us) service_pool.cc:225] Handling call
1211 20:52:42.445689 (+2728349us) ts_tablet_manager.cc:1151] Deleting tablet 66cdc6f297b24cfba3a3a0ae9f45d12d
1211 20:52:42.445701 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:42.445838 (+ 137us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2728183}
I20251211 20:52:42.446060 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_08f40dd56ad24729abbd776dabf07b8f [id=81c5998c4b8d4b7cb68ad8adb957cc0a]
I20251211 20:52:42.446215 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 911) took 2698 ms. Trace:
I20251211 20:52:42.446270 1165 rpcz_store.cc:276] 1211 20:52:39.747518 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.747573 (+ 55us) service_pool.cc:225] Handling call
1211 20:52:42.446129 (+2698556us) ts_tablet_manager.cc:1151] Deleting tablet 1944267def4e46949fd680ab0a369be9
1211 20:52:42.446136 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:42.446205 (+ 69us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2698333}
I20251211 20:52:42.446445 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_7e3bb905c8ec4e96809e6b7d4e44a3b7 [id=8212122882be451f9b5f5c6880bf3dce]
I20251211 20:52:42.446532 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 913) took 2668 ms. Trace:
I20251211 20:52:42.446589 1165 rpcz_store.cc:276] 1211 20:52:39.778066 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.778135 (+ 69us) service_pool.cc:225] Handling call
1211 20:52:42.446438 (+2668303us) ts_tablet_manager.cc:1151] Deleting tablet 7d4e63f54d8443ae8322d4659a165e76
1211 20:52:42.446446 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:42.446520 (+ 74us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2665617}
I20251211 20:52:42.446828 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_34bd5155aa3b4ef7818022ba9346c074 [id=83a61d23e80946439a14096ab3ab1bd9]
I20251211 20:52:42.446833 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 914) took 2665 ms. Trace:
I20251211 20:52:42.446887 1165 rpcz_store.cc:276] 1211 20:52:39.781881 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.781937 (+ 56us) service_pool.cc:225] Handling call
1211 20:52:42.446749 (+2664812us) ts_tablet_manager.cc:1151] Deleting tablet 0f8808bfd68841e4ae30818018040cf9
1211 20:52:42.446756 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:42.446822 (+ 66us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2655296}
I20251211 20:52:42.447103 1165 tablet_replica.cc:333] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:42.447182 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_b6ff995fa24746b8a8d1cfe5a656893f [id=83f8ca2368b9482ca769fd0da3d3606f]
I20251211 20:52:42.447276 1165 raft_consensus.cc:2243] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
W20251211 20:52:42.448138 847 leader_election.cc:343] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: cce44b80c97948e1af7ac68e77cccfd6
I20251211 20:52:42.449196 2375 raft_consensus.cc:493] T 4a39870c7eea4a379b284ea2e5ddf760 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.449350 2375 raft_consensus.cc:515] T 4a39870c7eea4a379b284ea2e5ddf760 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.450048 2375 leader_election.cc:290] T 4a39870c7eea4a379b284ea2e5ddf760 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:42.450403 1166 tablet_bootstrap.cc:492] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:42.450783 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4a39870c7eea4a379b284ea2e5ddf760" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:42.450909 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4a39870c7eea4a379b284ea2e5ddf760" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:42.450572 1166 ts_tablet_manager.cc:1403] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.029s user 0.000s sys 0.003s
W20251211 20:52:42.451416 983 leader_election.cc:343] T 4a39870c7eea4a379b284ea2e5ddf760 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:42.451898 982 leader_election.cc:343] T 4a39870c7eea4a379b284ea2e5ddf760 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.452023 982 leader_election.cc:304] T 4a39870c7eea4a379b284ea2e5ddf760 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.452394 2375 raft_consensus.cc:2749] T 4a39870c7eea4a379b284ea2e5ddf760 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.451900 1166 raft_consensus.cc:359] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.452867 1165 pending_rounds.cc:70] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919: Trying to abort 1 pending ops.
I20251211 20:52:42.452919 1166 raft_consensus.cc:385] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.452950 1165 pending_rounds.cc:77] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432130592329728 op_type: NO_OP noop_request { }
I20251211 20:52:42.453048 1165 raft_consensus.cc:2889] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:42.453085 1166 raft_consensus.cc:740] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.453141 1165 raft_consensus.cc:2272] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:42.453409 1166 consensus_queue.cc:260] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.453751 1165 ts_tablet_manager.cc:1916] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:42.454123 1166 ts_tablet_manager.cc:1434] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:42.454453 1166 tablet_bootstrap.cc:492] T 0aac6fb3972e4aba8950828c9e49d507 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:42.456194 1166 tablet_bootstrap.cc:654] T 0aac6fb3972e4aba8950828c9e49d507 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.447535 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_903bfd8112e444bcbe3e677773384da5 [id=849873b9731344feb51f5fbefd42576d]
I20251211 20:52:42.464259 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_6a85485dbf204937a1c5a8e0e151fc68 [id=856b47e2455f4727a81e6291ab2ccf38]
I20251211 20:52:42.464758 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_d17d0534f0544cef99924179cb1376b0 [id=85ab3a18cda143e5a7f72e49e6671318]
I20251211 20:52:42.465183 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_109eac21e7ee45e9a1ad808c49b10998 [id=8600a00e19a647188539792bdb9d6a86]
I20251211 20:52:42.465554 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_96bab30f95d5434ab659bb63869eb23d [id=86063f1c437a443c8d81b2f0cc4095de]
I20251211 20:52:42.466544 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_e70873b71ba24d7eae548e2acfd243b3 [id=87adcc552a704d56bd2fe812fb3b791f]
I20251211 20:52:42.466949 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f]
I20251211 20:52:42.467342 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_2198fd77ec454943a0c0a392964c1fd6 [id=88c89157d3404a558c1d0ca2ddbcf3bc]
W20251211 20:52:42.479784 1295 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.60:42087, user_credentials={real_user=slave}} blocked reactor thread for 54654.5us
I20251211 20:52:42.481638 2451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "11f5f94732f049d0be4c97cebfffcb05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } } }
I20251211 20:52:42.481822 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_cb6a4bac7b694f33a37de2ee78c91ca0 [id=8940f18527c249a9baefc170ed82f45a]
I20251211 20:52:42.481864 2451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [sys.catalog]: This master's current role is: FOLLOWER
I20251211 20:52:42.483085 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_b26cc06e1b0745e08181124e1cbce934 [id=8a12299efe714617a970fc05c3bb6232]
I20251211 20:52:42.483513 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_686a26915b42421380037144900dd97c [id=8a1b0c9d641b4a3c83d1ca48576d110d]
I20251211 20:52:42.483876 1166 tablet_bootstrap.cc:492] T 0aac6fb3972e4aba8950828c9e49d507 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:42.483928 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_083c478bedec45a5bf5203e4ed333860 [id=8baf0ea87d734d58896adbd0d6ef99c2]
I20251211 20:52:42.484076 1166 ts_tablet_manager.cc:1403] T 0aac6fb3972e4aba8950828c9e49d507 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.002s
I20251211 20:52:42.484524 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_1021d997007b41208355c31d22f4e26c [id=8c644777215b4617a8ce57691c34ce8c]
I20251211 20:52:42.484954 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_ee696fa128ac464da5a6f99bd3d7ed7a [id=8cb270b7f6e04c8c9307869d9dd6df4e]
I20251211 20:52:42.484961 1166 raft_consensus.cc:359] T 0aac6fb3972e4aba8950828c9e49d507 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.485132 1166 raft_consensus.cc:385] T 0aac6fb3972e4aba8950828c9e49d507 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.485203 1166 raft_consensus.cc:740] T 0aac6fb3972e4aba8950828c9e49d507 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.485339 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_dde47886ad4d4ea8a332c68e13085fe0 [id=8cc29ee550ed480c9e261d842fe08e3c]
I20251211 20:52:42.485420 1166 consensus_queue.cc:260] T 0aac6fb3972e4aba8950828c9e49d507 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.485961 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_d15e45922b4340a59fce3b9dc10faf71 [id=8e1b3fd5c43a446ab4cd6a8a4b4ee8e1]
I20251211 20:52:42.486514 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_b008891c4e894b02aa0db32d1740371a [id=8fe47ac33eb2461784b859e64fbd3e24]
I20251211 20:52:42.486874 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_48dfd045c6744e7db15278a2ebb7f434 [id=901b88b003d04e4783a9c09923910fdd]
I20251211 20:52:42.487412 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_ff22c4d4d5d34ac4bf51e3e4c97b2472 [id=925885faf3a6449faef52cc2033bfbf7]
I20251211 20:52:42.490989 2402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "11f5f94732f049d0be4c97cebfffcb05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } } }
I20251211 20:52:42.491246 2402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [sys.catalog]: This master's current role is: LEADER
I20251211 20:52:42.491894 2375 raft_consensus.cc:493] T 3e6f62c2deff4c11b137fc0cba5ee234 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.492048 2375 raft_consensus.cc:515] T 3e6f62c2deff4c11b137fc0cba5ee234 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.492647 2375 leader_election.cc:290] T 3e6f62c2deff4c11b137fc0cba5ee234 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:42.493446 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3e6f62c2deff4c11b137fc0cba5ee234" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:42.493639 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3e6f62c2deff4c11b137fc0cba5ee234" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:42.494165 982 leader_election.cc:343] T 3e6f62c2deff4c11b137fc0cba5ee234 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:42.494562 983 leader_election.cc:343] T 3e6f62c2deff4c11b137fc0cba5ee234 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.494673 983 leader_election.cc:304] T 3e6f62c2deff4c11b137fc0cba5ee234 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.495052 2375 raft_consensus.cc:2749] T 3e6f62c2deff4c11b137fc0cba5ee234 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.498234 1160 ts_tablet_manager.cc:1434] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.069s user 0.010s sys 0.015s
I20251211 20:52:42.498646 1160 tablet_bootstrap.cc:492] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:42.501791 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_bb1778853fbe4b7eaf0117deaf0877b9 [id=93b01cf646fd4f4dac01b7fecffa5f4e]
I20251211 20:52:42.502285 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_528b4823aba44a72a7af20911a8d2002 [id=940343c556c54c4fb2110f4f47ca91f3]
I20251211 20:52:42.502729 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_66eb1b021eed464f82f9cd223c88297b [id=94ff09564cd649c7a2d0e94b0f3c7b3a]
I20251211 20:52:42.503886 1166 ts_tablet_manager.cc:1434] T 0aac6fb3972e4aba8950828c9e49d507 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.020s user 0.001s sys 0.001s
W20251211 20:52:42.503919 983 consensus_peers.cc:597] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:42.504215 1166 tablet_bootstrap.cc:492] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:42.511190 1166 tablet_bootstrap.cc:654] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.513494 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_310ce452c2a54f069cf6c65f233c0603 [id=95c9ed1ec1834e2dbfd3198887b75369]
I20251211 20:52:42.514040 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_78bd2a084ad542c2968d1338f1ac71b5 [id=9649589ef6e94eebb27bb2b518105d2d]
I20251211 20:52:42.514655 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_c9d86e82aebb4ef0bc83789720352e38 [id=97377a8ef56c4cdebe95d15955207404]
I20251211 20:52:42.515090 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_e54af82590454fdf941b0462c30738ad [id=97e5719ea39a4ebe85ff613750c6863a]
W20251211 20:52:42.515494 711 consensus_peers.cc:597] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:42.515516 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_d44ac0044b77421f88b8a7b00811c8d3 [id=9800f52437ab4fb6b5a46e4861154750]
I20251211 20:52:42.516351 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_fd043ec094f94011986fc8f5a9f32f1a [id=994990d2a18f405c988452e93cca7bef]
I20251211 20:52:42.516986 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_59072235d93b41c0b5cd5e3debcb8ddf [id=9a15a8a075af4969ae2c8b3be1bfed78]
I20251211 20:52:42.517375 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_24737d22fc454a3da12979ae8966c9b3 [id=9a943e65becc4d47828265f814118328]
I20251211 20:52:42.517792 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_c1cf2780cc3941688af6d55d57d9e6a8 [id=9aa1c4d2d8a34aa5a6170b5c8e502581]
I20251211 20:52:42.518214 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_68ccb2052d824ccf92f7162d2961fcb1 [id=9ac9591d4f04482d926710fdf6e797c6]
I20251211 20:52:42.518877 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_fa6592c2e2624d81ad21abb13ed36045 [id=9b48fc2b39db4fd5a4e5da19e34427dc]
I20251211 20:52:42.521723 1160 tablet_bootstrap.cc:654] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.525674 1168 tablet_bootstrap.cc:492] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:42.525861 1168 ts_tablet_manager.cc:1403] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.093s user 0.004s sys 0.004s
I20251211 20:52:42.530298 1168 raft_consensus.cc:359] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.530504 1168 raft_consensus.cc:385] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.530563 1168 raft_consensus.cc:740] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.531327 1168 consensus_queue.cc:260] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.532162 777 raft_consensus.cc:2468] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 1.
I20251211 20:52:42.533149 847 leader_election.cc:304] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters: e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:42.535734 2460 raft_consensus.cc:2804] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:42.535980 2460 raft_consensus.cc:697] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 LEADER]: Becoming Leader. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Running, Role: LEADER
I20251211 20:52:42.536810 2460 consensus_queue.cc:237] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.538985 2462 raft_consensus.cc:493] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.539137 2462 raft_consensus.cc:515] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.541127 2462 leader_election.cc:290] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.541898 1165 ts_tablet_manager.cc:1929] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:42.542016 1165 log.cc:1199] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/2197822e0f11484f9679c4858626ac2c
I20251211 20:52:42.542346 1165 ts_tablet_manager.cc:1950] T 2197822e0f11484f9679c4858626ac2c P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
W20251211 20:52:42.543673 847 leader_election.cc:343] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: b631e6b3048e428f901af1b5b0b1ab3c
I20251211 20:52:42.543895 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_6b98cb90147a43c4aee6b39240c5edc4 [id=9c24845e4efe475fa792a51ce8a66be6]
I20251211 20:52:42.544414 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_4e6562227887405aae36515729c90a28 [id=9c44b27aece14760b4c94bc8d9f82218]
I20251211 20:52:42.545049 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_e887667aa38d47e9814a79f7b0e6990d [id=9d71d585608f49f48128aa54ecd0b724]
I20251211 20:52:42.546458 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_b011142d3fc34ef2a829989a9bad1114 [id=9e2abaab9bf940a8af1809342d51cd9b]
I20251211 20:52:42.546792 1160 tablet_bootstrap.cc:492] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:42.546911 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_328b7dd76e9e407c83b47a0f51cfcce5 [id=9e7c08f833e14a90b17a9b9f60b702de]
I20251211 20:52:42.547076 1160 ts_tablet_manager.cc:1403] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.048s user 0.002s sys 0.002s
I20251211 20:52:42.547758 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_4d6139c7976d414fbdcbdb96787d186b [id=9ffcb594267a4b0eb8d792dbe46d585f]
I20251211 20:52:42.548022 1160 raft_consensus.cc:359] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.548192 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_7e2c82ae1dbe44078774e5229684130d [id=a046f539a58a4c279d461cf9510c97d5]
I20251211 20:52:42.548319 1160 raft_consensus.cc:385] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.548451 1160 raft_consensus.cc:740] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.548786 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_d18401707c88493abdf90d2c713432ae [id=a07662f8dc3f4178be5dfcc1cdc85837]
I20251211 20:52:42.548765 1160 consensus_queue.cc:260] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.549441 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_3de7aa51e2814433b4d98f3226b96c97 [id=a137f0ddf52145a99c62444602ab3fdb]
I20251211 20:52:42.549602 1160 ts_tablet_manager.cc:1434] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:42.549952 1160 tablet_bootstrap.cc:492] T 676faec7f3d1488f99b220f8e968e9a0 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:42.550823 2462 raft_consensus.cc:493] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.550972 2462 raft_consensus.cc:515] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:42.551836 1784 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.104.60:42087 timed out after 0.651s (SENT)
I20251211 20:52:42.552759 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "793f543d6b614ad7b3ab8e717db2ceaf" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:42.553586 1168 ts_tablet_manager.cc:1434] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.028s user 0.001s sys 0.001s
I20251211 20:52:42.553910 1168 tablet_bootstrap.cc:492] T 7ec261837cf04bf88a48ff8add1b0d66 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:42.554128 847 leader_election.cc:343] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 793f543d6b614ad7b3ab8e717db2ceaf
I20251211 20:52:42.554250 847 leader_election.cc:304] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:42.554498 2322 raft_consensus.cc:2749] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.554808 2462 leader_election.cc:290] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.564597 1160 tablet_bootstrap.cc:654] T 676faec7f3d1488f99b220f8e968e9a0 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:42.565407 847 leader_election.cc:343] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: b631e6b3048e428f901af1b5b0b1ab3c
I20251211 20:52:42.566365 847 leader_election.cc:304] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:42.566824 2322 raft_consensus.cc:2749] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.566985 2322 raft_consensus.cc:493] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.567085 2322 raft_consensus.cc:515] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.567687 2322 leader_election.cc:290] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:42.569036 847 leader_election.cc:343] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: fb030a62796248869d46b682656eaeb0
W20251211 20:52:42.569439 847 leader_election.cc:343] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: fb030a62796248869d46b682656eaeb0
I20251211 20:52:42.569557 847 leader_election.cc:304] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:42.569999 2322 raft_consensus.cc:2749] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.571969 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_690a1a1fd5d848818451819f38fc1470 [id=a24d8c58b9d84cdea2b278ae9c6e0ba5]
I20251211 20:52:42.572552 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_ab826a3eec8e4c4aaa8d4021d87dd382 [id=a2bbfccfe3974c378521d5ef2333df86]
I20251211 20:52:42.573203 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_18ec995eeab648dd8f0399b55e571ece [id=a362f3dbfa82447483dc50751ed0a9b8]
I20251211 20:52:42.577649 1160 tablet_bootstrap.cc:492] T 676faec7f3d1488f99b220f8e968e9a0 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:42.577730 1166 tablet_bootstrap.cc:492] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:42.577826 1160 ts_tablet_manager.cc:1403] T 676faec7f3d1488f99b220f8e968e9a0 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.028s user 0.001s sys 0.003s
I20251211 20:52:42.577905 1166 ts_tablet_manager.cc:1403] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.074s user 0.004s sys 0.000s
I20251211 20:52:42.555567 1168 tablet_bootstrap.cc:654] T 7ec261837cf04bf88a48ff8add1b0d66 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.578773 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_5e5286ea174048f8ba9abd14daf14bcf [id=a455f6ef64ae44f78972d14b2ea9654c]
I20251211 20:52:42.578691 1166 raft_consensus.cc:359] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.578884 1166 raft_consensus.cc:385] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.578960 1166 raft_consensus.cc:740] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.579237 1166 consensus_queue.cc:260] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.579375 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_79cef56b856548afb009a80ec288ad1b [id=a4bb425aab7d4a9183630f3db61a6310]
I20251211 20:52:42.579957 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_b6e6aea68c88426fb01353146c06b6ac [id=a4f984b4263a4bc38f6248628f643ae3]
I20251211 20:52:42.580391 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_a06aeab492cf4bd59dbc1274c6c6827d [id=a58ccb217e734813a81f290602e09bef]
I20251211 20:52:42.581151 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_5db1ac98e36d45b1b01889ee275ad83d [id=a6849726434d41d29dce12f47066a685]
I20251211 20:52:42.581840 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_6bc2fc890a7a485ba23c3214f772b758 [id=a703dcf9f6244f34878332575ff2324c]
I20251211 20:52:42.582150 1168 tablet_bootstrap.cc:492] T 7ec261837cf04bf88a48ff8add1b0d66 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:42.582286 1168 ts_tablet_manager.cc:1403] T 7ec261837cf04bf88a48ff8add1b0d66 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.000s
I20251211 20:52:42.582654 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_f12876c06fc04c63a2f23a6fe5b89f2b [id=a8664a1ea42f411bba24a8ac719155ce]
I20251211 20:52:42.582998 1168 raft_consensus.cc:359] T 7ec261837cf04bf88a48ff8add1b0d66 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.583180 1168 raft_consensus.cc:385] T 7ec261837cf04bf88a48ff8add1b0d66 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.583230 1168 raft_consensus.cc:740] T 7ec261837cf04bf88a48ff8add1b0d66 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.583443 1168 consensus_queue.cc:260] T 7ec261837cf04bf88a48ff8add1b0d66 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.583941 1168 ts_tablet_manager.cc:1434] T 7ec261837cf04bf88a48ff8add1b0d66 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:42.584229 1168 tablet_bootstrap.cc:492] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:42.584501 2462 raft_consensus.cc:493] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.584659 2462 raft_consensus.cc:515] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.584721 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_6814df0b3dd640f58f421a6ed9010686 [id=aa4a2053a7cf478884770f8e36a02fc4]
I20251211 20:52:42.585131 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_98f173a3b5454e16ac5680ec2465a09c [id=ab5b964850544c718f621b5ec87e3372]
I20251211 20:52:42.585244 2462 leader_election.cc:290] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.585505 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_f2eaa62388894072a390cc4c167f3385 [id=ac6774893a6a4f82a7c9630bbe502fe9]
W20251211 20:52:42.586642 847 leader_election.cc:343] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 52ef1707f6f2425aafd454f303af0333
W20251211 20:52:42.589949 847 leader_election.cc:343] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 52ef1707f6f2425aafd454f303af0333
I20251211 20:52:42.590094 847 leader_election.cc:304] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:42.590540 2462 raft_consensus.cc:2749] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.591908 1160 raft_consensus.cc:359] T 676faec7f3d1488f99b220f8e968e9a0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.592101 1160 raft_consensus.cc:385] T 676faec7f3d1488f99b220f8e968e9a0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.592175 1160 raft_consensus.cc:740] T 676faec7f3d1488f99b220f8e968e9a0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.592424 1160 consensus_queue.cc:260] T 676faec7f3d1488f99b220f8e968e9a0 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.592927 1160 ts_tablet_manager.cc:1434] T 676faec7f3d1488f99b220f8e968e9a0 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20251211 20:52:42.585898 1168 tablet_bootstrap.cc:654] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.599982 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_f2301327ddd04d278201a1cb0da3afe8 [id=ace2fe24e1504556a7b7bd455d7c6c8c]
I20251211 20:52:42.600890 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_97f780d6dd39409dbb5e5ff7e4fd61fd [id=adef6ee1143c43e495691dc186c25241]
I20251211 20:52:42.601652 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_6829fee154a74021a1410d7535c0c89f [id=aea8a10b336548f78e1c093ce01a22a9]
I20251211 20:52:42.602376 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_ef6d1803b4bf4d62860e0bbdf745446a [id=b11896be27d848958e5b3fab8f5504a1]
I20251211 20:52:42.602926 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_f762d4c1a3c44c63abb1c9cd599ac9ed [id=b1509c25961f4fb1b49866b4410d326d]
I20251211 20:52:42.603488 1166 ts_tablet_manager.cc:1434] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.025s user 0.002s sys 0.000s
I20251211 20:52:42.605949 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 915) took 2797 ms. Trace:
I20251211 20:52:42.608114 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_eaad557efa374b0bbd5653d90a967f8b [id=b187d831069447e39a00bc5e91f1fcf8]
I20251211 20:52:42.606065 1165 rpcz_store.cc:276] 1211 20:52:39.808243 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.808302 (+ 59us) service_pool.cc:225] Handling call
1211 20:52:42.447065 (+2638763us) ts_tablet_manager.cc:1151] Deleting tablet 2197822e0f11484f9679c4858626ac2c
1211 20:52:42.447072 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:42.532501 (+ 85429us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:42.541768 (+ 9267us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:42.605922 (+ 64154us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2619152}
I20251211 20:52:42.609807 1165 tablet_replica.cc:333] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:42.609992 1165 raft_consensus.cc:2243] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:42.610316 1165 pending_rounds.cc:70] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919: Trying to abort 1 pending ops.
I20251211 20:52:42.610376 1165 pending_rounds.cc:77] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432129514188800 op_type: NO_OP noop_request { }
I20251211 20:52:42.610471 1165 raft_consensus.cc:2889] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:42.610555 1165 raft_consensus.cc:2272] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:42.611218 1165 ts_tablet_manager.cc:1916] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:42.611689 1166 tablet_bootstrap.cc:492] T d37a7d0d662e45079b0cfc02765d2e01 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:42.612321 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_4ecbff2ab6c243f994ef4fae46276e92 [id=b3230934455d40598c6213630dd3108b]
I20251211 20:52:42.613324 1166 tablet_bootstrap.cc:654] T d37a7d0d662e45079b0cfc02765d2e01 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.613554 2325 raft_consensus.cc:493] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.613678 2325 raft_consensus.cc:515] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.614219 2325 leader_election.cc:290] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.615044 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f5916a4d9b24d7b8db3fcf028bbe3c9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:42.615238 1045 raft_consensus.cc:2468] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:42.615641 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f5916a4d9b24d7b8db3fcf028bbe3c9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:42.615916 710 leader_election.cc:304] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:42.616186 2286 raft_consensus.cc:2804] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:42.616264 2286 raft_consensus.cc:493] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:42.616323 2286 raft_consensus.cc:3060] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:42.616403 711 leader_election.cc:343] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.616508 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_47ff5fcc1088481592447a200ff8432f [id=b4cbc8ef524d4b72bc1c109aa2dd09e0]
I20251211 20:52:42.617210 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_2aa7ff9186a04249abf8cd971cc0cb17 [id=b50ab5b032ba4c98938324460512468f]
I20251211 20:52:42.618475 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_2cff6fab088c4d939b13b504c133cc12 [id=b6c4b72063b3428593d5650a160dc66b]
I20251211 20:52:42.618920 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_5453a73a163149baa88d85b027bb42ee [id=b6d9975565684e9784951ede7dbd7988]
I20251211 20:52:42.620026 1168 tablet_bootstrap.cc:492] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:42.620173 1168 ts_tablet_manager.cc:1403] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.036s user 0.004s sys 0.001s
I20251211 20:52:42.621177 1168 raft_consensus.cc:359] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.621515 1168 raft_consensus.cc:385] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.621606 1168 raft_consensus.cc:740] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.621863 1168 consensus_queue.cc:260] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.622498 1168 ts_tablet_manager.cc:1434] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:42.623641 1168 tablet_bootstrap.cc:492] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:42.626405 2375 raft_consensus.cc:493] T 41c30bded6504bbc9f0af8f34dc81e8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.626557 2375 raft_consensus.cc:515] T 41c30bded6504bbc9f0af8f34dc81e8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.627154 2375 leader_election.cc:290] T 41c30bded6504bbc9f0af8f34dc81e8d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:42.627799 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_03298b88200d42148accc9f0dffbf739 [id=b7429ea93123473ab5d795d5f3d5a5f8]
I20251211 20:52:42.628137 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "41c30bded6504bbc9f0af8f34dc81e8d" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:42.628592 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "41c30bded6504bbc9f0af8f34dc81e8d" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:42.629366 983 leader_election.cc:343] T 41c30bded6504bbc9f0af8f34dc81e8d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:42.629657 982 leader_election.cc:343] T 41c30bded6504bbc9f0af8f34dc81e8d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.629745 982 leader_election.cc:304] T 41c30bded6504bbc9f0af8f34dc81e8d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.629905 2460 raft_consensus.cc:493] T 676faec7f3d1488f99b220f8e968e9a0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.630077 2460 raft_consensus.cc:515] T 676faec7f3d1488f99b220f8e968e9a0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.630157 2375 raft_consensus.cc:2749] T 41c30bded6504bbc9f0af8f34dc81e8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.631950 2460 leader_election.cc:290] T 676faec7f3d1488f99b220f8e968e9a0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.632298 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "676faec7f3d1488f99b220f8e968e9a0" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:42.633317 2286 raft_consensus.cc:515] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.633997 2286 leader_election.cc:290] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.636375 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_ec0bf98cd1ce42ceacacde30208d5d04 [id=b8fdb6772ace427e8273983d1e791956]
I20251211 20:52:42.637563 1165 ts_tablet_manager.cc:1929] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:42.637694 1165 log.cc:1199] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/793f543d6b614ad7b3ab8e717db2ceaf
I20251211 20:52:42.638531 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f5916a4d9b24d7b8db3fcf028bbe3c9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:42.638890 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f5916a4d9b24d7b8db3fcf028bbe3c9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:42.639096 1047 raft_consensus.cc:3060] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:42.641520 1166 tablet_bootstrap.cc:492] T d37a7d0d662e45079b0cfc02765d2e01 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:42.638962 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "676faec7f3d1488f99b220f8e968e9a0" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:42.641672 1166 ts_tablet_manager.cc:1403] T d37a7d0d662e45079b0cfc02765d2e01 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.002s
I20251211 20:52:42.642195 847 leader_election.cc:304] T 676faec7f3d1488f99b220f8e968e9a0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:42.642271 1160 tablet_bootstrap.cc:492] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:42.642910 2462 raft_consensus.cc:2749] T 676faec7f3d1488f99b220f8e968e9a0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.643216 1166 raft_consensus.cc:359] T d37a7d0d662e45079b0cfc02765d2e01 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.643388 1166 raft_consensus.cc:385] T d37a7d0d662e45079b0cfc02765d2e01 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.643453 1166 raft_consensus.cc:740] T d37a7d0d662e45079b0cfc02765d2e01 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.638324 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_4cbc8841b276406cb4434629653a5ce3 [id=b958532650374d609199cab9c07bb64b]
I20251211 20:52:42.643950 1166 consensus_queue.cc:260] T d37a7d0d662e45079b0cfc02765d2e01 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.644207 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_71e9c158dc22435a8fbb02aecca8919d [id=b966d0c0b49d4023894950c5dc06f678]
I20251211 20:52:42.649605 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_e250eb3a69ef4e6da3893852edc00469 [id=ba41048fe47d4749b5e084582989eb38]
W20251211 20:52:42.639499 711 leader_election.cc:343] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.651364 1166 ts_tablet_manager.cc:1434] T d37a7d0d662e45079b0cfc02765d2e01 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20251211 20:52:42.639740 1168 tablet_bootstrap.cc:654] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.651400 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_717a9e3f73be421585b11e3cdad48b64 [id=bd5290958c7249f39a6cc51f631f42a9]
I20251211 20:52:42.652472 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_6a2c9aead0a640b79bb4c9d6f2258e67 [id=bde705b86af14b11be4a2e1fc236b37a]
I20251211 20:52:42.653004 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_031de0f4cc7744bc9e320261f8b46696 [id=be36dabaa6d44325b4855b9f13bda847]
I20251211 20:52:42.653462 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_655c466dc1b54183b7e91ddeca33ae5e [id=be92c660569a46e7981df62185f0cbfd]
I20251211 20:52:42.653576 2460 raft_consensus.cc:493] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.653775 2460 raft_consensus.cc:515] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.654325 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_c2f4d6a8642a44b1b5566b7631b598e6 [id=bee5829ed7d44761a9ae57d3134d38dc]
I20251211 20:52:42.655035 2460 leader_election.cc:290] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:42.655701 847 leader_election.cc:343] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 2dde7f28e61d41f48015ba6e4f3757f2
W20251211 20:52:42.656659 711 consensus_peers.cc:597] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:42.657768 847 leader_election.cc:343] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 2dde7f28e61d41f48015ba6e4f3757f2
I20251211 20:52:42.657924 847 leader_election.cc:304] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:42.660149 2460 raft_consensus.cc:2749] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.660387 1166 tablet_bootstrap.cc:492] T 1578887b2f5748cd83a6fbdec2f3fc99 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:42.665122 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_bd0dcfadc59f4ffa81d0004ae868ed3e [id=c07b39a3869a4f3781ebccf30c95d5e5]
I20251211 20:52:42.665768 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_86d6443f6a564414a7f024e4aa12d450 [id=c1cd2830fb0648fab33480850c1d2adb]
I20251211 20:52:42.638731 1165 ts_tablet_manager.cc:1950] T 793f543d6b614ad7b3ab8e717db2ceaf P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:42.647325 1160 tablet_bootstrap.cc:654] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.667352 1166 tablet_bootstrap.cc:654] T 1578887b2f5748cd83a6fbdec2f3fc99 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.673417 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_7287bc8f28a64682b352f9adae2ec224 [id=c2883f35f01248219793761e8f0984aa]
I20251211 20:52:42.676465 1168 tablet_bootstrap.cc:492] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:42.676648 1168 ts_tablet_manager.cc:1403] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.053s user 0.008s sys 0.002s
I20251211 20:52:42.677413 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_c2f02a2463774499a7858b0a53ee965d [id=c342962bfd9242c6811897d1c980dfe9]
I20251211 20:52:42.677418 1168 raft_consensus.cc:359] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.677596 1168 raft_consensus.cc:385] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.677650 1168 raft_consensus.cc:740] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.677973 1168 consensus_queue.cc:260] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.678094 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_8e6bffee74fd4ea4a95ec5e51b0154c1 [id=c345c786f9a949dea34cc93bb72ae4cf]
I20251211 20:52:42.678572 1168 ts_tablet_manager.cc:1434] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:42.678596 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_e3de355d0a69496c842b4a35daa99b90 [id=c34e32ee8d9f41b68288ced87e28069f]
I20251211 20:52:42.678853 1168 tablet_bootstrap.cc:492] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:42.679194 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_3a9e5e76c5324906b05c61141dfb7034 [id=c356cc5067cd4dbb877c660a70aa71e7]
I20251211 20:52:42.679970 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_54b89a8f30cf414590cfd7a5773bf326 [id=c3d36e0944a74f6eb98b9bb510d8352a]
I20251211 20:52:42.681308 1160 tablet_bootstrap.cc:492] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:42.681481 1160 ts_tablet_manager.cc:1403] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.039s user 0.003s sys 0.002s
I20251211 20:52:42.681921 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_eeb5507eb6a74588905b9e236e414324 [id=c4432f9a16b84796ae788c7a3cde370b]
I20251211 20:52:42.682260 1160 raft_consensus.cc:359] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.682456 1160 raft_consensus.cc:385] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.682525 1160 raft_consensus.cc:740] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.682806 1160 consensus_queue.cc:260] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.683272 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_7809ac85fdfe4c1d9c58afb54edecb27 [id=c4b0bfff99f64888beb70d9154d2f156]
I20251211 20:52:42.688089 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_1ccc66e4393445059e9653a319360528 [id=c57f12c04ee54729b215809907e7f20e]
W20251211 20:52:42.688439 713 connection.cc:537] server connection from 127.0.0.1:37886 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251211 20:52:42.690129 2286 consensus_queue.cc:1048] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251211 20:52:42.691507 1295 connection.cc:537] client connection to 127.0.104.60:42087 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251211 20:52:42.691777 1295 leader_election.cc:336] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9230f94a6a1a49cead51aeea9f069087 (127.0.104.60:42087): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251211 20:52:42.691955 1295 leader_election.cc:304] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1830b0c58cfe495989f3d911dda0409a; no voters: 11f5f94732f049d0be4c97cebfffcb05, 9230f94a6a1a49cead51aeea9f069087
I20251211 20:52:42.692963 2451 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20251211 20:52:42.693629 1103 connection.cc:537] client connection to 127.0.104.60:42087 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251211 20:52:42.695205 1103 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.104.60:42087)
W20251211 20:52:42.695984 1103 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.104.60:42087)
W20251211 20:52:42.696624 1103 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.104.60:42087)
I20251211 20:52:42.697063 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_6d94053661f142b79292e19752df02df [id=c5be59a2c3934b3789641ceeb2d4d6e2]
W20251211 20:52:42.697389 1103 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.104.60:42087)
I20251211 20:52:42.697690 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_3418cf6b51394b03afec9b03cde8861c [id=c7c19676e1484bbcb73b3bf14f147ae5]
I20251211 20:52:42.698060 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_4e8d74eff9674d67b5b0f54370b06598 [id=c842792b01cb4c689fcfd3d796151508]
I20251211 20:52:42.698407 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_ec893136f41e41f6a6db70f193b8065e [id=c9f87468f622412a8fcd50d311f99117]
I20251211 20:52:42.699043 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_233d5484bd4e4456aed925bc271edfa5 [id=ca0c3c6978874d93a04b57cc0c425c83]
W20251211 20:52:42.700681 984 connection.cc:537] client connection to 127.0.104.60:42087 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251211 20:52:42.702522 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 916) took 2893 ms. Trace:
I20251211 20:52:42.703061 1047 raft_consensus.cc:2468] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:42.703758 710 leader_election.cc:304] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.704284 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_d465fa4868dd4c4db1036653f8b6f463 [id=cb97ccf262fd4c41a180267fd0678cec]
I20251211 20:52:42.704345 2325 raft_consensus.cc:2804] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:42.704478 2325 raft_consensus.cc:697] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:42.704753 2325 consensus_queue.cc:237] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.702740 1165 rpcz_store.cc:276] 1211 20:52:39.808930 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.808982 (+ 52us) service_pool.cc:225] Handling call
1211 20:52:42.609751 (+2800769us) ts_tablet_manager.cc:1151] Deleting tablet 793f543d6b614ad7b3ab8e717db2ceaf
1211 20:52:42.609764 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:42.634272 (+ 24508us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:42.637453 (+ 3181us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:42.702489 (+ 65036us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2781744}
W20251211 20:52:42.706193 713 connection.cc:441] server connection from 127.0.0.1:37886 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 916) could send its response
I20251211 20:52:42.706606 1165 tablet_replica.cc:333] T bb0fc179a337469eae056c5bb3815e6b P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:42.707000 1165 raft_consensus.cc:2243] T bb0fc179a337469eae056c5bb3815e6b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:42.707512 1165 raft_consensus.cc:2272] T bb0fc179a337469eae056c5bb3815e6b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:42.708423 2273 consensus_queue.cc:1048] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:42.709057 2460 raft_consensus.cc:493] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.709219 2460 raft_consensus.cc:515] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.710167 2460 leader_election.cc:290] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:42.711002 847 leader_election.cc:343] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: c74427cb9d6f4a679e43eae796ff8c3f
I20251211 20:52:42.711781 2375 raft_consensus.cc:493] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.711939 2375 raft_consensus.cc:515] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.712108 1165 ts_tablet_manager.cc:1916] T bb0fc179a337469eae056c5bb3815e6b P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:42.712522 2375 leader_election.cc:290] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:42.713128 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5453f68789fd4e89b69e98860050865b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:42.713465 847 leader_election.cc:343] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: c74427cb9d6f4a679e43eae796ff8c3f
I20251211 20:52:42.713515 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_909bbe6da1a648d68199fb27424e4342 [id=cc99459a5039424dbc192ff748b6708b]
I20251211 20:52:42.713593 847 leader_election.cc:304] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:42.713665 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5453f68789fd4e89b69e98860050865b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:42.713726 849 connection.cc:537] client connection to 127.0.104.60:42087 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251211 20:52:42.714051 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_86e88e19fbd645d3bd5a919328a0969c [id=cccf9f896f034dda92ea20dfbd569ae1]
W20251211 20:52:42.714351 983 leader_election.cc:343] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.714695 2460 raft_consensus.cc:2749] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:42.715524 712 connection.cc:537] client connection to 127.0.104.60:42087 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251211 20:52:42.716053 982 leader_election.cc:343] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.716192 982 leader_election.cc:304] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.716883 2375 raft_consensus.cc:2749] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.717929 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_ebf689834b574c408bb00777fac24c84 [id=d20c23f61583488f8c8104a18ad2461c]
I20251211 20:52:42.720141 2273 consensus_queue.cc:1048] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:42.722772 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_11a7f002cdb8441eabe140368c795751 [id=d25542cf6b174800a1c0d46f07397bfe]
I20251211 20:52:42.723620 1166 tablet_bootstrap.cc:492] T 1578887b2f5748cd83a6fbdec2f3fc99 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:42.724514 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_5d7c1dcaff1a420c95fb2907b7cce4b8 [id=d3274b7f43b6447f887db4c3c885fe2e]
I20251211 20:52:42.724689 2375 raft_consensus.cc:493] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.724776 1166 ts_tablet_manager.cc:1403] T 1578887b2f5748cd83a6fbdec2f3fc99 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.064s user 0.001s sys 0.004s
I20251211 20:52:42.724812 2375 raft_consensus.cc:515] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.725418 2375 leader_election.cc:290] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:42.725713 1166 raft_consensus.cc:359] T 1578887b2f5748cd83a6fbdec2f3fc99 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.725871 1166 raft_consensus.cc:385] T 1578887b2f5748cd83a6fbdec2f3fc99 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.725920 1166 raft_consensus.cc:740] T 1578887b2f5748cd83a6fbdec2f3fc99 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.726243 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b30d2054316b49fa8096345527a45d87" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:42.726734 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b30d2054316b49fa8096345527a45d87" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:42.726781 983 leader_election.cc:343] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.726936 777 raft_consensus.cc:2468] T b30d2054316b49fa8096345527a45d87 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 0.
I20251211 20:52:42.727020 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_862fff1ce3564d939f033f3f2fe049ac [id=d35a5d6eb11049239d1898d175b356b8]
I20251211 20:52:42.727489 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_4e2d2e9566764ee9bc5edf65d0e693a2 [id=d3c56e51e2464bafb3819af545f21a3f]
I20251211 20:52:42.727507 982 leader_election.cc:304] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.727689 1166 consensus_queue.cc:260] T 1578887b2f5748cd83a6fbdec2f3fc99 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.727875 2375 raft_consensus.cc:2804] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:42.727941 2375 raft_consensus.cc:493] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:42.727994 2375 raft_consensus.cc:3060] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:42.729488 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_b022d39850dc4b8f92b79b8a747f0409 [id=d49d5216afba41d68497f42fc72b1bf0]
I20251211 20:52:42.729931 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_44f971096d1e41f480e594985ffd5c17 [id=d4e1349374a346a5b8be180f9bdc329a]
I20251211 20:52:42.730290 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_c817dec9a7674f699b3366fc302ead57 [id=d555d1fc6aef4a21b4d3f2804264d030]
I20251211 20:52:42.730947 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_464acf89dba243ae925fa3b824ceb333 [id=d69ab7378b1d4272b1c2f8df48b96e81]
I20251211 20:52:42.731269 1166 ts_tablet_manager.cc:1434] T 1578887b2f5748cd83a6fbdec2f3fc99 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20251211 20:52:42.731724 1166 tablet_bootstrap.cc:492] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:42.733229 2485 negotiation.cc:337] Failed RPC negotiation. Trace:
1211 20:52:42.698430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.104.60:42087 (local address 127.0.0.1:39448)
1211 20:52:42.724114 (+ 25684us) negotiation.cc:107] Waiting for socket to connect
1211 20:52:42.725960 (+ 1846us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.104.60:42087: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":25505,"thread_start_us":25893,"threads_started":1}
I20251211 20:52:42.733601 1166 tablet_bootstrap.cc:654] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.736054 909 raft_consensus.cc:3060] T 3b2c6064c5eb4c0f8979c485f5d85a75 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:42.739768 2429 raft_consensus.cc:493] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.739931 2429 raft_consensus.cc:515] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.741096 2429 leader_election.cc:290] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.742003 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3de5b34b96404cdc9c191fed7f055638" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:42.742455 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3de5b34b96404cdc9c191fed7f055638" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:42.742655 1045 raft_consensus.cc:2468] T 3de5b34b96404cdc9c191fed7f055638 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
W20251211 20:52:42.742759 711 leader_election.cc:343] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.744884 2286 raft_consensus.cc:493] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.745112 2286 raft_consensus.cc:515] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.745872 2286 leader_election.cc:290] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.746986 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "861d2d70a24946699dccd65cab05195e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:42.747622 710 leader_election.cc:304] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.748194 2429 raft_consensus.cc:2804] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:42.748286 2429 raft_consensus.cc:493] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:42.748358 2429 raft_consensus.cc:3060] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:42.748418 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_cb402f83e7b24e308e46a798daa7db02 [id=d7bc4f722b6943438cffc3f0a7e5015a]
I20251211 20:52:42.749439 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "861d2d70a24946699dccd65cab05195e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:42.749786 1042 raft_consensus.cc:2468] T 861d2d70a24946699dccd65cab05195e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
W20251211 20:52:42.747648 711 leader_election.cc:343] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.751355 710 leader_election.cc:304] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.752156 2494 raft_consensus.cc:2804] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:42.752249 2494 raft_consensus.cc:493] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:42.752315 2494 raft_consensus.cc:3060] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:42.756217 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_24b29712550340f2bea51b04f527c4f3 [id=d8a0839df7524fb7b95bfc05fd4ada25]
I20251211 20:52:42.757390 1160 ts_tablet_manager.cc:1434] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.076s user 0.001s sys 0.001s
I20251211 20:52:42.757625 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_8db9acb9d35149fdabcb9a1d24061b46 [id=d919c6132b3c466e8a1404f981b003b8]
I20251211 20:52:42.758404 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_4ec0d87202fc419380c2542d827087de [id=d9829c79b55844ec92b1b0cf594ab96b]
I20251211 20:52:42.759167 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_91c76a3c00864c3fa3d653b3f6e0f64f [id=d991ee6b2aa54096a3b1676618b6cb22]
I20251211 20:52:42.759711 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_18262e22c26d4d66ba4c6134fbde0368 [id=da8cf2b99b42409091484a9513623d6f]
I20251211 20:52:42.760596 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_f5a4d53f860a4ab48ac66679cd67c7eb [id=db7d2c4d61b246f2984075c1787b24ec]
I20251211 20:52:42.760664 2286 raft_consensus.cc:493] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.760991 2286 raft_consensus.cc:515] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.761166 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_af66deaa476a4ea986b403ef120654ff [id=dc94606dc32f4fd0ae8d0885323ab561]
I20251211 20:52:42.761777 1166 tablet_bootstrap.cc:492] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:42.761801 2286 leader_election.cc:290] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.761947 1166 ts_tablet_manager.cc:1403] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.000s
I20251211 20:52:42.762151 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_cbecf49ce5e044049547838a5d6ce009 [id=dd75e76866b84667ae39862a9330a45d]
I20251211 20:52:42.762634 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_e2af0e2fbd3d46d595136c7a7bf24e4c [id=ddc12ae6d8ed4587b5ef4da3ef81a055]
I20251211 20:52:42.762822 1166 raft_consensus.cc:359] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.763032 1166 raft_consensus.cc:385] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.763089 1166 raft_consensus.cc:740] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.763289 1166 consensus_queue.cc:260] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.763936 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_68f5e21962ce48b89834785bf8dd332f [id=df1df827d2194032b668c7579bc1dcda]
I20251211 20:52:42.763957 1166 ts_tablet_manager.cc:1434] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:42.764132 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7147571483e41ad970e573a08c368ca" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:42.764246 1166 tablet_bootstrap.cc:492] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:42.765501 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7147571483e41ad970e573a08c368ca" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
W20251211 20:52:42.765643 711 leader_election.cc:343] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.765897 1166 tablet_bootstrap.cc:654] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.765865 1042 raft_consensus.cc:2468] T c7147571483e41ad970e573a08c368ca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:42.766398 710 leader_election.cc:304] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.766808 2286 raft_consensus.cc:2804] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:42.766943 2286 raft_consensus.cc:493] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:42.766970 1160 tablet_bootstrap.cc:492] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:42.767074 2286 raft_consensus.cc:3060] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:42.768157 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_37849576518b4320a3bac34baa096242 [id=df6255d66f334ef2b4406f4b845cf980]
I20251211 20:52:42.768896 1160 tablet_bootstrap.cc:654] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.769742 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_b43dda4aa0674e3f8ef0d55e0e2431e3 [id=e0cd4de0a942462d9e3b50079f6b757d]
I20251211 20:52:42.771042 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_c1f789cbcb1d4ddcbe038bc2ef5e2b92 [id=e154f2c3021f400c80ba23bcd269883f]
I20251211 20:52:42.771849 1168 tablet_bootstrap.cc:654] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.772282 2385 raft_consensus.cc:493] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.772441 2385 raft_consensus.cc:515] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.773103 2385 leader_election.cc:290] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:42.773834 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3562b5effa124378beac7ccf0d27ee7e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:42.773923 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_ed03ad475bed4804a65abe20efef0625 [id=e23f4305669a474aafa18928f130de3a]
I20251211 20:52:42.774398 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3562b5effa124378beac7ccf0d27ee7e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:42.774783 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_7e3cf73eaea24e34b5dd291f8a738947 [id=e3ff2e8147aa41a99206b3961c29d751]
W20251211 20:52:42.775223 983 leader_election.cc:343] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:42.776466 982 leader_election.cc:343] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.776645 982 leader_election.cc:304] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.776976 2385 raft_consensus.cc:2749] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:42.777770 2486 negotiation.cc:337] Failed RPC negotiation. Trace:
1211 20:52:42.702365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.104.60:42087 (local address 127.0.104.3:41625)
1211 20:52:42.777379 (+ 75014us) negotiation.cc:107] Waiting for socket to connect
1211 20:52:42.777488 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.104.60:42087: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":74902,"thread_start_us":74036,"threads_started":1}
I20251211 20:52:42.778589 2385 consensus_queue.cc:1048] T 22273287aa9245a2a66131fed2c8861c P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:42.788476 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_89da60cc38f54ac48f12f3b5988d48ad [id=e40a35fac2d7424ca101a4f6b4af3a2c]
I20251211 20:52:42.789804 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_60f0bdc6a1c849fd851ca0d31eeeaf79 [id=e6d99593c9aa45c7bca5a782bad9c904]
I20251211 20:52:42.794624 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_9f6d83209f134db4a00fbddf8f323113 [id=e7d1c65b881746eeb12169e26fe0db70]
W20251211 20:52:42.780740 1784 connection.cc:537] client connection to 127.0.104.60:42087 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251211 20:52:42.795853 1784 leader_election.cc:336] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9230f94a6a1a49cead51aeea9f069087 (127.0.104.60:42087): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251211 20:52:42.796290 1784 leader_election.cc:336] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9230f94a6a1a49cead51aeea9f069087 (127.0.104.60:42087): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251211 20:52:42.798607 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_b929f7410fd44316b3a3571d02066b45 [id=e9dcc3d777524e70b9653ec6a2df7da0]
I20251211 20:52:42.799913 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_ca4526eee78f4811a98c7a64f76431d8 [id=eb61893aa74349e79064946f7d8334a6]
W20251211 20:52:42.800185 711 consensus_peers.cc:597] T fed300d31c414f4a9b27c66e78097688 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:42.800769 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_13a8009dbcc342d98426f71c053a964a [id=eb64e2c8b06a44afbb4aa2c4df0cac69]
I20251211 20:52:42.801939 2375 raft_consensus.cc:515] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.804175 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_be91bd80b0374d5eae01b605de36b203 [id=eb7478f4f95f45479e8c830ba6575a40]
I20251211 20:52:42.804479 2429 raft_consensus.cc:515] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.804852 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_823505200e3142dbbe8cd56ac78daf55 [id=ec0bb10933fa4529bef3580bee6409e7]
I20251211 20:52:42.805159 2429 leader_election.cc:290] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.805807 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_c7ed8c0c309c40a3a371ffa130b47cd2 [id=ed1d0670426945aba888b818ad657225]
I20251211 20:52:42.806191 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_bcb5c15cfcc2448a9f814e5ff5665f93 [id=ed371e073250439fa44330c7101e9387]
I20251211 20:52:42.806711 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_142734af8412498fa19dd25ad1ef2c2e [id=edf08c4121454149890dcef509afd417]
I20251211 20:52:42.806588 2494 raft_consensus.cc:515] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.807230 2494 leader_election.cc:290] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:42.809625 1784 consensus_peers.cc:597] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 -> Peer 9230f94a6a1a49cead51aeea9f069087 (127.0.104.60:42087): Couldn't send request to peer 9230f94a6a1a49cead51aeea9f069087. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.60:42087: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:42.810493 2375 leader_election.cc:290] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:42.817323 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_07f1881614084afd936866d115aecbe7 [id=eed7dfa66ab545dca02bd10c0a6a0e71]
I20251211 20:52:42.818380 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "861d2d70a24946699dccd65cab05195e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:42.818565 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b30d2054316b49fa8096345527a45d87" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:42.819087 983 leader_election.cc:343] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:42.819566 711 leader_election.cc:343] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.821574 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_b7a123d140e64879b3fb2606998af217 [id=eee1b058701c499fac48dfab3dc873d0]
I20251211 20:52:42.822139 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_ddb3c1574c924ada922b623f332a3613 [id=ef058448a4ff49f59d0e943e11ed64b2]
I20251211 20:52:42.822650 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_b44b03dcd95a41c19061d5e52f5687c9 [id=ef458da415914c138373fbd5e0b8dec4]
I20251211 20:52:42.823155 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_e345037da95a4c16b6e8b650f0ba3911 [id=efb341043b8748dd883699a699425bbd]
I20251211 20:52:42.823637 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_a8ebfd7d7a7f4f6cae8b6b02b99b80b0 [id=f1d32002c612403a82bdff5baf209b9e]
I20251211 20:52:42.824220 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_f84cd3de863b446894133bb40aaf1a39 [id=f4d608cc41d0408486d3f6b0a09383a4]
I20251211 20:52:42.824707 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_3f8e5876c38a4c5aac82efa6e26de84d [id=f51019a5e161417b8dbca90195fdc761]
I20251211 20:52:42.829634 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "861d2d70a24946699dccd65cab05195e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:42.829877 1045 raft_consensus.cc:3060] T 861d2d70a24946699dccd65cab05195e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:42.830288 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3de5b34b96404cdc9c191fed7f055638" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:42.830430 1042 raft_consensus.cc:3060] T 3de5b34b96404cdc9c191fed7f055638 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:42.833664 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b30d2054316b49fa8096345527a45d87" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:42.833878 777 raft_consensus.cc:3060] T b30d2054316b49fa8096345527a45d87 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:42.834743 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3de5b34b96404cdc9c191fed7f055638" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:42.835668 711 leader_election.cc:343] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.836186 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_31d573e7af144e9997dfe480f8466902 [id=f689abc175c94366bfda8217545156aa]
I20251211 20:52:42.836728 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_43e9cb840d48444b9360e2fa8671dfb8 [id=f6cab1da32324fe6afc1df2007f0bdbf]
I20251211 20:52:42.837347 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_f6a10556ff104db5bd5ecc9fc380e228 [id=f75813add54e46189ff1bbed82d83ba7]
I20251211 20:52:42.838876 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_2fddb279b6164c2ba9e81fc55dfeb59a [id=f8c4f76e6a2d48fb93603b9d308a71ec]
I20251211 20:52:42.839423 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_20456517a58b40fabd534461d2ab6cc0 [id=f93e5d8158d74f7cb3b77ddefe065c8f]
I20251211 20:52:42.839927 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_7684c7e006504ea0a1ef81dc2eecaa8d [id=f943bf305b574c81b836e91e267d5d81]
I20251211 20:52:42.842993 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_a32f150ce5604ec999ac5f3edc6324b4 [id=f98e1455f27a4c98a9c484931ed502b3]
I20251211 20:52:42.844007 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_84d90fc27f874a01bf3187d6d02b2053 [id=fa858cefc5bc458ca620f77a25b697a8]
I20251211 20:52:42.844517 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_6ea777b3e1c3474f926a3e1ab782c69b [id=fc9d41c089a443fab5b61579f4c788f3]
I20251211 20:52:42.850297 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_b9de49d730424e2b863b64535f2ba732 [id=fd5a2aedcd31426394ad1ffdd0936893]
I20251211 20:52:42.852092 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_25d3985a897841b8b55ad7c44bbff154 [id=fde40e5a01ee40c48ca653928036c9a1]
I20251211 20:52:42.855893 2499 consensus_queue.cc:1048] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:42.872108 2509 raft_consensus.cc:493] T 6b7dd8232d974a43b99ec9b28075dc8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.872496 2509 raft_consensus.cc:515] T 6b7dd8232d974a43b99ec9b28075dc8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.875260 1168 tablet_bootstrap.cc:492] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:42.875437 1168 ts_tablet_manager.cc:1403] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.197s user 0.002s sys 0.002s
I20251211 20:52:42.876506 1168 raft_consensus.cc:359] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.876837 1168 raft_consensus.cc:385] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.876940 1168 raft_consensus.cc:740] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.877823 1168 consensus_queue.cc:260] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.879511 1168 ts_tablet_manager.cc:1434] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251211 20:52:42.882081 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_51db250005ed4b3d947f32acd1d62563 [id=fdfaa0b24d2f4736aee695ed228212cf]
I20251211 20:52:42.884334 2508 raft_consensus.cc:493] T bbe3529c301a4a65b58b634a443e1fb6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.884692 2508 raft_consensus.cc:515] T bbe3529c301a4a65b58b634a443e1fb6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.885526 2462 raft_consensus.cc:493] T 72e23f3033e14c9db6de8539830bac3b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.885697 2462 raft_consensus.cc:515] T 72e23f3033e14c9db6de8539830bac3b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.885924 2508 leader_election.cc:290] T bbe3529c301a4a65b58b634a443e1fb6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:42.886456 2462 leader_election.cc:290] T 72e23f3033e14c9db6de8539830bac3b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.888042 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "72e23f3033e14c9db6de8539830bac3b" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:42.888600 2460 raft_consensus.cc:493] T f737a7bfb0f44196abdc831bb22d5285 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.888774 2460 raft_consensus.cc:515] T f737a7bfb0f44196abdc831bb22d5285 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.888942 2415 catalog_manager.cc:679] Loaded metadata for table default.Table_760e3e4714ea4128bd1de3da5a7ccccc [id=fe9b692163f943559ba64021faa648f1]
I20251211 20:52:42.889385 2460 leader_election.cc:290] T f737a7bfb0f44196abdc831bb22d5285 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.872215 2510 raft_consensus.cc:493] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.890569 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbe3529c301a4a65b58b634a443e1fb6" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:42.890609 2510 raft_consensus.cc:515] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:42.891170 983 leader_election.cc:343] T bbe3529c301a4a65b58b634a443e1fb6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.891338 2509 leader_election.cc:290] T 6b7dd8232d974a43b99ec9b28075dc8d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:42.892160 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b7dd8232d974a43b99ec9b28075dc8d" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:42.892288 2510 leader_election.cc:290] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
W20251211 20:52:42.893628 983 leader_election.cc:343] T 6b7dd8232d974a43b99ec9b28075dc8d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.894002 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "215f8dc74a71482daa2ca80e15010f51" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:42.894604 983 leader_election.cc:343] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:42.896704 2489 negotiation.cc:337] Failed RPC negotiation. Trace:
1211 20:52:42.715195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.104.60:42087 (local address 127.0.104.2:46627)
1211 20:52:42.848984 (+133789us) negotiation.cc:107] Waiting for socket to connect
1211 20:52:42.849117 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.104.60:42087: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":133625,"threads_started":1}
I20251211 20:52:42.902415 2375 raft_consensus.cc:493] T d37a7d0d662e45079b0cfc02765d2e01 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.902591 2375 raft_consensus.cc:515] T d37a7d0d662e45079b0cfc02765d2e01 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.903203 2375 leader_election.cc:290] T d37a7d0d662e45079b0cfc02765d2e01 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:42.904696 1045 raft_consensus.cc:2468] T 861d2d70a24946699dccd65cab05195e P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:42.908084 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d37a7d0d662e45079b0cfc02765d2e01" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:42.908975 983 leader_election.cc:343] T d37a7d0d662e45079b0cfc02765d2e01 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.909863 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f737a7bfb0f44196abdc831bb22d5285" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:42.911015 1160 tablet_bootstrap.cc:492] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:42.911268 1160 ts_tablet_manager.cc:1403] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.144s user 0.000s sys 0.004s
I20251211 20:52:42.912168 1160 raft_consensus.cc:359] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.917163 1160 raft_consensus.cc:385] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.917383 1160 raft_consensus.cc:740] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.917718 1160 consensus_queue.cc:260] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.887065 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbe3529c301a4a65b58b634a443e1fb6" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:42.887473 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "72e23f3033e14c9db6de8539830bac3b" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:42.891430 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b7dd8232d974a43b99ec9b28075dc8d" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:42.892690 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "215f8dc74a71482daa2ca80e15010f51" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:42.892896 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f737a7bfb0f44196abdc831bb22d5285" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:42.919011 982 leader_election.cc:343] T bbe3529c301a4a65b58b634a443e1fb6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.919124 982 leader_election.cc:304] T bbe3529c301a4a65b58b634a443e1fb6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:42.895424 711 consensus_peers.cc:597] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:42.919684 982 leader_election.cc:343] T 6b7dd8232d974a43b99ec9b28075dc8d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.919787 982 leader_election.cc:304] T 6b7dd8232d974a43b99ec9b28075dc8d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:42.920141 982 leader_election.cc:343] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.900166 2325 raft_consensus.cc:493] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.920230 982 leader_election.cc:304] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.920331 2325 raft_consensus.cc:515] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.920528 2507 raft_consensus.cc:2749] T bbe3529c301a4a65b58b634a443e1fb6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.920689 2507 raft_consensus.cc:2749] T 6b7dd8232d974a43b99ec9b28075dc8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.920778 2507 raft_consensus.cc:2749] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.920881 2325 leader_election.cc:290] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.904151 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d37a7d0d662e45079b0cfc02765d2e01" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:42.904330 2286 raft_consensus.cc:515] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.921545 2286 leader_election.cc:290] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.905161 710 leader_election.cc:304] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.922720 2286 raft_consensus.cc:2804] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:42.922837 2286 raft_consensus.cc:697] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:42.923012 2325 raft_consensus.cc:493] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.923105 2325 raft_consensus.cc:515] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.923724 2325 leader_election.cc:290] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:42.925320 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7147571483e41ad970e573a08c368ca" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:42.925395 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b1ce6d2a12a4c7ea044b349412b4bf5" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:42.925513 1048 raft_consensus.cc:3060] T c7147571483e41ad970e573a08c368ca P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:42.925570 1045 raft_consensus.cc:2468] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:42.925346 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2e33606930774fe4aafc6e3819618af8" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:42.923054 2286 consensus_queue.cc:237] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.926054 1044 raft_consensus.cc:2468] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
W20251211 20:52:42.927886 982 leader_election.cc:343] T d37a7d0d662e45079b0cfc02765d2e01 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.928017 982 leader_election.cc:304] T d37a7d0d662e45079b0cfc02765d2e01 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.928308 2503 raft_consensus.cc:2749] T d37a7d0d662e45079b0cfc02765d2e01 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.928507 710 leader_election.cc:304] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:42.928818 710 leader_election.cc:304] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:42.929102 2499 raft_consensus.cc:2804] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:42.929172 2499 raft_consensus.cc:493] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:42.929226 2499 raft_consensus.cc:3060] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:42.929674 2325 raft_consensus.cc:2804] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:42.929749 2325 raft_consensus.cc:493] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:42.929802 2325 raft_consensus.cc:3060] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:42.912802 1165 ts_tablet_manager.cc:1929] T bb0fc179a337469eae056c5bb3815e6b P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:42.935451 1165 log.cc:1199] T bb0fc179a337469eae056c5bb3815e6b P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/bb0fc179a337469eae056c5bb3815e6b
I20251211 20:52:42.936228 1165 ts_tablet_manager.cc:1950] T bb0fc179a337469eae056c5bb3815e6b P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:42.920122 847 leader_election.cc:304] T 72e23f3033e14c9db6de8539830bac3b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:42.921841 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b1ce6d2a12a4c7ea044b349412b4bf5" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:42.924546 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7147571483e41ad970e573a08c368ca" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:42.924677 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2e33606930774fe4aafc6e3819618af8" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:42.939977 711 leader_election.cc:343] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:42.940351 711 leader_election.cc:343] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:42.940524 711 leader_election.cc:343] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.942173 1168 tablet_bootstrap.cc:492] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:42.942974 847 leader_election.cc:304] T f737a7bfb0f44196abdc831bb22d5285 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:42.943706 2462 raft_consensus.cc:2749] T 72e23f3033e14c9db6de8539830bac3b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.943887 2462 raft_consensus.cc:2749] T f737a7bfb0f44196abdc831bb22d5285 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.945576 2415 tablet_loader.cc:96] loaded metadata for tablet 01ada50100bc48e39d1bc67082da750e (table default.Table_690a1a1fd5d848818451819f38fc1470 [id=a24d8c58b9d84cdea2b278ae9c6e0ba5])
I20251211 20:52:42.954825 2503 raft_consensus.cc:493] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:42.955018 2503 raft_consensus.cc:515] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.955781 2503 leader_election.cc:290] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:42.957150 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eb9da081cb9d43fd9a35c4bc51ea133e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:42.957794 982 leader_election.cc:343] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.958973 1168 tablet_bootstrap.cc:654] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.954610 1160 ts_tablet_manager.cc:1434] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.043s user 0.002s sys 0.000s
I20251211 20:52:42.956562 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eb9da081cb9d43fd9a35c4bc51ea133e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:42.960800 983 leader_election.cc:343] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:42.960867 1166 tablet_bootstrap.cc:492] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:42.960923 983 leader_election.cc:304] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:42.961000 1166 ts_tablet_manager.cc:1403] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.197s user 0.042s sys 0.062s
I20251211 20:52:42.961418 2503 raft_consensus.cc:2749] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:42.961752 1166 raft_consensus.cc:359] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.962845 1166 raft_consensus.cc:385] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.962911 1166 raft_consensus.cc:740] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.963104 1166 consensus_queue.cc:260] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.963599 1166 ts_tablet_manager.cc:1434] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20251211 20:52:42.963887 1166 tablet_bootstrap.cc:492] T e484d499af1b4947adb36c32f8ea5704 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:42.968940 1166 tablet_bootstrap.cc:654] T e484d499af1b4947adb36c32f8ea5704 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.969364 1160 tablet_bootstrap.cc:492] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:42.971673 2415 tablet_loader.cc:96] loaded metadata for tablet 0302cc8613934c43a0650f081cf98797 (table default.Table_bcb5c15cfcc2448a9f814e5ff5665f93 [id=ed371e073250439fa44330c7101e9387])
I20251211 20:52:42.972280 2415 tablet_loader.cc:96] loaded metadata for tablet 03184dcc56dd4e92adc7d0ad22aa8d8b (table default.Table_cec07d512c8049e581be33a0214fad11 [id=697f23a617f64762bf406b386cde7473])
I20251211 20:52:42.973754 2415 tablet_loader.cc:96] loaded metadata for tablet 0350405f1d5844d2a987697fd0d8850c (table default.Table_c1cf2780cc3941688af6d55d57d9e6a8 [id=9aa1c4d2d8a34aa5a6170b5c8e502581])
I20251211 20:52:42.974576 2415 tablet_loader.cc:96] loaded metadata for tablet 04086eaa32a74edd84235d3b28f9f965 (table default.Table_2c7bb884f8b5488e9a80e5d20e75cc95 [id=4942c719263148deb2c5cb57a605672a])
I20251211 20:52:42.975273 2415 tablet_loader.cc:96] loaded metadata for tablet 04491faa6f594e128e51d8c266b33759 (table default.Table_3f8e5876c38a4c5aac82efa6e26de84d [id=f51019a5e161417b8dbca90195fdc761])
I20251211 20:52:42.975746 2415 tablet_loader.cc:96] loaded metadata for tablet 04679dfddf564d6f8c16d1c257e29b41 (table default.Table_2cd4051c3df74bbc9b5d715b770ee926 [id=188ac848ba6b41e88df38554246be155])
I20251211 20:52:42.975997 1166 tablet_bootstrap.cc:492] T e484d499af1b4947adb36c32f8ea5704 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:42.976142 1166 ts_tablet_manager.cc:1403] T e484d499af1b4947adb36c32f8ea5704 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.001s
I20251211 20:52:42.976958 1166 raft_consensus.cc:359] T e484d499af1b4947adb36c32f8ea5704 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.977139 1166 raft_consensus.cc:385] T e484d499af1b4947adb36c32f8ea5704 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:42.977200 1166 raft_consensus.cc:740] T e484d499af1b4947adb36c32f8ea5704 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:42.979290 1166 consensus_queue.cc:260] T e484d499af1b4947adb36c32f8ea5704 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:42.980165 2415 tablet_loader.cc:96] loaded metadata for tablet 06ba98a726424dd89450053c5c5793f4 (table default.Table_bdb9aa9507d94416bbc3c8fe9cb11fa7 [id=367894d4c46a4cc6bde351a099a635b3])
I20251211 20:52:42.980584 2415 tablet_loader.cc:96] loaded metadata for tablet 06d49ac8182143c69f494da50c050ee0 (table default.Table_3db64a0e2af24640883c6413a68001b0 [id=5ccf927de35144b9b9cc9324638fe9c4])
I20251211 20:52:42.981112 1166 ts_tablet_manager.cc:1434] T e484d499af1b4947adb36c32f8ea5704 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251211 20:52:42.981161 2415 tablet_loader.cc:96] loaded metadata for tablet 0763a475527546bb83cee8fb823893ac (table default.Table_227fbf051d5643528315ca00194583bc [id=1515561ab78d46f6b5aa37a20d90d3f5])
I20251211 20:52:42.981442 1166 tablet_bootstrap.cc:492] T fa79b67b9eee45e2b41c4a90967606cc P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:42.982160 2415 tablet_loader.cc:96] loaded metadata for tablet 09cd11f8b23c4f8a98a2b2eb77bb7b8c (table default.Table_440d3f9c240c448bb5bf5bfbdaf35d00 [id=276a75404b564adcb63880b9bbed2359])
I20251211 20:52:42.983183 1166 tablet_bootstrap.cc:654] T fa79b67b9eee45e2b41c4a90967606cc P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:42.985597 2415 tablet_loader.cc:96] loaded metadata for tablet 0b38de97bc4a42af91b2f2cd70ece669 (table default.Table_b6ff995fa24746b8a8d1cfe5a656893f [id=83f8ca2368b9482ca769fd0da3d3606f])
I20251211 20:52:42.986124 2415 tablet_loader.cc:96] loaded metadata for tablet 0bbb0b010aae44a3a43abdf910e08b77 (table default.Table_ef6d1803b4bf4d62860e0bbdf745446a [id=b11896be27d848958e5b3fab8f5504a1])
I20251211 20:52:42.986593 2415 tablet_loader.cc:96] loaded metadata for tablet 0beff1cfdb1e49908ee4e6756b8b7e65 (table default.Table_97f780d6dd39409dbb5e5ff7e4fd61fd [id=adef6ee1143c43e495691dc186c25241])
I20251211 20:52:42.987028 2415 tablet_loader.cc:96] loaded metadata for tablet 0c6266d18f674b41bc0e1689b6b8cbf6 (table default.Table_3a9e5e76c5324906b05c61141dfb7034 [id=c356cc5067cd4dbb877c660a70aa71e7])
I20251211 20:52:42.990325 2415 tablet_loader.cc:96] loaded metadata for tablet 11af6cfd71a94e768c66b36862b048bb (table default.Table_bae5e6fe430247c99eae40ae86192b39 [id=6212dd14e00341be8a2c71adcf7f4c61])
I20251211 20:52:42.990664 1042 raft_consensus.cc:2468] T 3de5b34b96404cdc9c191fed7f055638 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:42.991811 2415 tablet_loader.cc:96] loaded metadata for tablet 12d77bf2d30843daab1beb01c396faa5 (table default.Table_af66deaa476a4ea986b403ef120654ff [id=dc94606dc32f4fd0ae8d0885323ab561])
I20251211 20:52:42.996973 2415 tablet_loader.cc:96] loaded metadata for tablet 13269a20421240d1a98215129443248b (table default.Table_d887c953026046dc883ede774a58fd8e [id=1d8a79f716a0447cb0393f4759178598])
I20251211 20:52:42.998407 2415 tablet_loader.cc:96] loaded metadata for tablet 16d2ff8925e342729a71c93db378d031 (table default.Table_34e6ed4f13da437ea50d349b92e83bd6 [id=4a74b9a60cbb4198b55cee630b2f4bef])
I20251211 20:52:42.998925 2415 tablet_loader.cc:96] loaded metadata for tablet 16fbe528e228447a83e61852f002df60 (table default.Table_9fa0ba597863437e94957c10e3a68b4b [id=24012f9fe7eb4ba4ac585e3dea3b1425])
I20251211 20:52:42.999332 2415 tablet_loader.cc:96] loaded metadata for tablet 171d40a8a9124f9bb1b38797fef8e5d5 (table default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5])
I20251211 20:52:43.000020 2415 tablet_loader.cc:96] loaded metadata for tablet 1722730c38b344ffb1ff3347e314f197 (table default.Table_e870d2f987ff44e6b91ebe0205f6fe9e [id=53b3365f07864ac1a3cd47482bb6377a])
I20251211 20:52:43.000492 2415 tablet_loader.cc:96] loaded metadata for tablet 174902fb98a5492dbc29423797cb5a2b (table default.Table_f5a4d53f860a4ab48ac66679cd67c7eb [id=db7d2c4d61b246f2984075c1787b24ec])
I20251211 20:52:43.000974 2415 tablet_loader.cc:96] loaded metadata for tablet 1843dc4548134f29969d46fbc760f5d7 (table default.Table_99917cb4c70840e19004ad381746d77e [id=6abb187bfd214e91b2e11267065a2841])
I20251211 20:52:43.005784 2460 raft_consensus.cc:493] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.006009 2460 raft_consensus.cc:515] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.006758 2460 leader_election.cc:290] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:43.008176 847 leader_election.cc:343] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 92a71dff4ea8445d8484fc2747471148
W20251211 20:52:43.008761 847 leader_election.cc:343] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 92a71dff4ea8445d8484fc2747471148
I20251211 20:52:43.008910 847 leader_election.cc:304] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:43.009442 2460 raft_consensus.cc:2749] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:43.006801 2415 tablet_loader.cc:96] loaded metadata for tablet 18f125dd9efc401181f0644dad7aeffe (table default.Table_fa6592c2e2624d81ad21abb13ed36045 [id=9b48fc2b39db4fd5a4e5da19e34427dc])
I20251211 20:52:42.968623 2429 consensus_queue.cc:1048] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:42.985277 777 raft_consensus.cc:2468] T b30d2054316b49fa8096345527a45d87 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 1.
I20251211 20:52:42.991425 710 leader_election.cc:304] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.011356 2415 tablet_loader.cc:96] loaded metadata for tablet 199f0f51ba0d4dd887e65032b4ab9b7f (table default.Table_109eac21e7ee45e9a1ad808c49b10998 [id=8600a00e19a647188539792bdb9d6a86])
I20251211 20:52:43.011983 2415 tablet_loader.cc:96] loaded metadata for tablet 1a05ed40a4454e7e8721a556ac3de092 (table default.Table_9e7258165cbf458c83d7cd4874a8755f [id=6f1e5458bbef44fdb8ae111900bf5159])
I20251211 20:52:43.012108 2501 raft_consensus.cc:2804] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:43.012228 2501 raft_consensus.cc:697] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:43.012306 2415 tablet_loader.cc:96] loaded metadata for tablet 1a1016ef8bf94d2e95d4774df44f01f1 (table default.Table_b1d96cf6ac344427a92df55d1dfcee90 [id=679d27d782fb480086443c78df21fa52])
I20251211 20:52:43.012483 2501 consensus_queue.cc:237] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.012645 2415 tablet_loader.cc:96] loaded metadata for tablet 1a77e3e744ff42ccb68897704b4c2f1d (table default.Table_d17d0534f0544cef99924179cb1376b0 [id=85ab3a18cda143e5a7f72e49e6671318])
I20251211 20:52:43.012960 982 leader_election.cc:304] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.013096 2415 tablet_loader.cc:96] loaded metadata for tablet 1b8acfdf1a26427aa9863280dc39b818 (table default.Table_78bd2a084ad542c2968d1338f1ac71b5 [id=9649589ef6e94eebb27bb2b518105d2d])
I20251211 20:52:43.013414 2415 tablet_loader.cc:96] loaded metadata for tablet 1bcf36efcd204fea868fd04ee7a9cdc6 (table default.Table_b7a123d140e64879b3fb2606998af217 [id=eee1b058701c499fac48dfab3dc873d0])
I20251211 20:52:43.013540 2503 raft_consensus.cc:2804] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:43.013670 2503 raft_consensus.cc:697] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 LEADER]: Becoming Leader. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Running, Role: LEADER
I20251211 20:52:43.013900 2503 consensus_queue.cc:237] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:43.018443 983 consensus_peers.cc:597] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:43.018435 711 consensus_peers.cc:597] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:43.021648 2429 raft_consensus.cc:493] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.021833 2429 raft_consensus.cc:515] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.022536 2429 leader_election.cc:290] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.023239 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "940dfd8781de444a9caa78dd363c7451" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:43.023545 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "940dfd8781de444a9caa78dd363c7451" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:43.023788 1042 raft_consensus.cc:2468] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
W20251211 20:52:43.024317 711 leader_election.cc:343] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.024905 710 leader_election.cc:304] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.025033 911 raft_consensus.cc:3060] T 72e23f3033e14c9db6de8539830bac3b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.026559 2286 raft_consensus.cc:2804] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:43.026651 2286 raft_consensus.cc:493] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:43.026705 2286 raft_consensus.cc:3060] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.030584 2460 raft_consensus.cc:493] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.030826 2460 raft_consensus.cc:515] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.031507 2460 leader_election.cc:290] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.031690 2415 tablet_loader.cc:96] loaded metadata for tablet 1c35fd2abd054ad789fecdb29edbe55e (table default.Table_96bab30f95d5434ab659bb63869eb23d [id=86063f1c437a443c8d81b2f0cc4095de])
I20251211 20:52:43.032229 2415 tablet_loader.cc:96] loaded metadata for tablet 1c4b2c22bd4c456eb77af45576de466f (table default.Table_e3a99b043e4f420a93ebfee6aa87f2ec [id=3e7d6cefcb7d412191c6540e9ad77c2d])
I20251211 20:52:43.032907 2415 tablet_loader.cc:96] loaded metadata for tablet 1f4eb1a174c648c897ae2b7612e44585 (table default.Table_e70873b71ba24d7eae548e2acfd243b3 [id=87adcc552a704d56bd2fe812fb3b791f])
I20251211 20:52:43.033250 2415 tablet_loader.cc:96] loaded metadata for tablet 200b6db115964f3c8882b514c794698d (table default.Table_fffa8ba66cb04202b2116b8fd4a741de [id=4d2c08402d5845e4ae592cc28ce8d9b1])
W20251211 20:52:43.033567 847 leader_election.cc:343] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 2197822e0f11484f9679c4858626ac2c
I20251211 20:52:43.034268 2415 tablet_loader.cc:96] loaded metadata for tablet 21cb988566a247d0b9ac9b337b1188fb (table default.Table_11a7f002cdb8441eabe140368c795751 [id=d25542cf6b174800a1c0d46f07397bfe])
W20251211 20:52:43.034567 847 leader_election.cc:343] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 2197822e0f11484f9679c4858626ac2c
I20251211 20:52:43.034730 847 leader_election.cc:304] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:43.035190 2460 raft_consensus.cc:2749] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:43.036262 2415 tablet_loader.cc:96] loaded metadata for tablet 243d0d396aca438a997e26246ca1ea21 (table default.Table_1f11bf68fb18474c9a3cce63119e7e5f [id=713c1b5f22c24b918617f807b319b1c8])
I20251211 20:52:43.037418 2415 tablet_loader.cc:96] loaded metadata for tablet 26b2f69913b44a4281b8d61097e780e4 (table default.Table_11314b6f726648c3bda96c9c1c7da29e [id=6ce96b52af404551a7e7e5186d54ee8a])
I20251211 20:52:43.037904 2415 tablet_loader.cc:96] loaded metadata for tablet 2718aa08ba3b4a70a1caa67d0023ec4a (table default.Table_f762d4c1a3c44c63abb1c9cd599ac9ed [id=b1509c25961f4fb1b49866b4410d326d])
I20251211 20:52:43.038980 2415 tablet_loader.cc:96] loaded metadata for tablet 284e34970f634abcb2288c864f73e963 (table default.Table_e250eb3a69ef4e6da3893852edc00469 [id=ba41048fe47d4749b5e084582989eb38])
I20251211 20:52:43.054565 2338 raft_consensus.cc:493] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.054765 2338 raft_consensus.cc:515] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.055406 2338 leader_election.cc:290] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.056623 2385 raft_consensus.cc:493] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.056690 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d9cf0d92ccc7419cb6e7694e8e5c7bfb" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:43.056792 2385 raft_consensus.cc:515] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.057376 2385 leader_election.cc:290] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:43.058167 1043 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d9cf0d92ccc7419cb6e7694e8e5c7bfb" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:43.058354 1043 raft_consensus.cc:2468] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:43.058745 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f22e4d34cde24b37a025c1b5a1046b1b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:43.058992 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f22e4d34cde24b37a025c1b5a1046b1b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:43.059768 983 leader_election.cc:343] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:43.060102 711 leader_election.cc:343] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.060384 2338 raft_consensus.cc:493] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.060500 2338 raft_consensus.cc:515] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.061118 2338 leader_election.cc:290] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.061865 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3acfef6f082641a199b80eefcfc772ac" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:43.062318 982 leader_election.cc:343] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.062341 1043 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3acfef6f082641a199b80eefcfc772ac" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:43.062507 982 leader_election.cc:304] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.062566 1043 raft_consensus.cc:2468] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:43.062714 2385 raft_consensus.cc:2749] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:43.063422 711 leader_election.cc:343] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.064563 710 leader_election.cc:304] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.065017 2415 tablet_loader.cc:96] loaded metadata for tablet 2a276bc5e8a3458fb63a2a6a652aa28d (table default.Table_47ff5fcc1088481592447a200ff8432f [id=b4cbc8ef524d4b72bc1c109aa2dd09e0])
I20251211 20:52:43.065122 710 leader_election.cc:304] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.065397 2273 raft_consensus.cc:2804] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:43.065470 2273 raft_consensus.cc:493] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:43.065526 2273 raft_consensus.cc:3060] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.065627 2500 raft_consensus.cc:2804] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:43.065683 2500 raft_consensus.cc:493] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:43.065734 2500 raft_consensus.cc:3060] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.066197 2415 tablet_loader.cc:96] loaded metadata for tablet 2c65926700954af7ab539f3b78805d8d (table default.Table_beb19ebcfc3b426e9aa9f70a68445f72 [id=67d5a2d2ef5a4e329c5ffd80756ef8f2])
I20251211 20:52:43.067020 2415 tablet_loader.cc:96] loaded metadata for tablet 2cf2f91730fa480b92063e50489bd37e (table default.Table_b94ece9030c84d37a4ed8d28c2a12a91 [id=2026c3e019fb4c85bb9911107c1cd648])
I20251211 20:52:43.067512 2415 tablet_loader.cc:96] loaded metadata for tablet 2d06c480594a44948fe004f73857f3bd (table default.Table_6cce646a13f6405c96fd61f193add5fa [id=1ea52ab9ffc74cb18c6e24e4f4d1cf88])
I20251211 20:52:43.067966 2415 tablet_loader.cc:96] loaded metadata for tablet 2d0fa31361b74760ae7394a11f6fcafa (table default.Table_1ccc66e4393445059e9653a319360528 [id=c57f12c04ee54729b215809907e7f20e])
W20251211 20:52:43.069080 711 consensus_peers.cc:597] T a9bb22dce13f4c9da76ca446715cd99a P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:43.071530 2415 tablet_loader.cc:96] loaded metadata for tablet 2e4ff869b0104de5b05035ac880f90d5 (table default.Table_5d7c1dcaff1a420c95fb2907b7cce4b8 [id=d3274b7f43b6447f887db4c3c885fe2e])
I20251211 20:52:43.072042 2415 tablet_loader.cc:96] loaded metadata for tablet 2e7e158809b9471fbd6e6ee9fb9ac569 (table default.Table_6ea777b3e1c3474f926a3e1ab782c69b [id=fc9d41c089a443fab5b61579f4c788f3])
I20251211 20:52:43.072561 2415 tablet_loader.cc:96] loaded metadata for tablet 2e800d8cc5ea4535a065d7d9243326b4 (table default.Table_083c478bedec45a5bf5203e4ed333860 [id=8baf0ea87d734d58896adbd0d6ef99c2])
I20251211 20:52:43.073264 2415 tablet_loader.cc:96] loaded metadata for tablet 2f5916a4d9b24d7b8db3fcf028bbe3c9 (table default.Table_53c0c6163e2447959bc8eb3303c1685f [id=6e0536d7748d456a9de9df3ba7e914be])
I20251211 20:52:43.074005 2415 tablet_loader.cc:96] loaded metadata for tablet 3003ef71a3554c71a3b24cdfb9c293bc (table default.Table_92efd469dd324f7d886e2cbc90ece61c [id=6e7fb3271e0c48c0bb9d56fe4fccee3a])
I20251211 20:52:43.074404 2415 tablet_loader.cc:96] loaded metadata for tablet 31e860227d874abf8d97901fa432948a (table default.Table_ebf689834b574c408bb00777fac24c84 [id=d20c23f61583488f8c8104a18ad2461c])
I20251211 20:52:43.074687 913 raft_consensus.cc:3060] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.079457 1168 tablet_bootstrap.cc:492] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:43.080138 1168 ts_tablet_manager.cc:1403] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.142s user 0.004s sys 0.000s
I20251211 20:52:43.083685 1048 raft_consensus.cc:2468] T c7147571483e41ad970e573a08c368ca P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:43.083137 1168 raft_consensus.cc:359] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.083945 1168 raft_consensus.cc:385] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.084241 1168 raft_consensus.cc:740] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.084372 2499 raft_consensus.cc:515] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.084996 2499 leader_election.cc:290] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.086126 1168 consensus_queue.cc:260] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.088342 1168 ts_tablet_manager.cc:1434] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20251211 20:52:43.088726 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b1ce6d2a12a4c7ea044b349412b4bf5" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:43.089663 1168 tablet_bootstrap.cc:492] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:43.089979 1160 tablet_bootstrap.cc:654] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:43.090655 711 leader_election.cc:343] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.091508 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b1ce6d2a12a4c7ea044b349412b4bf5" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:43.091747 1048 raft_consensus.cc:3060] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.092141 2415 tablet_loader.cc:96] loaded metadata for tablet 33fd09ea06ee47908b85911e4ee79708 (table default.Table_3c3cd48c8fe5411e96cdf811d492925d [id=30620a01c27f456bac05fa7c02a2cb4d])
I20251211 20:52:43.092751 2415 tablet_loader.cc:96] loaded metadata for tablet 34083c49cc514fbabfb8457980629185 (table default.Table_c15e1d9cc43c4d0b951d51344fb6ad09 [id=09a33d8a7c7c46c2b8b737fc9a8543b8])
I20251211 20:52:43.093120 2415 tablet_loader.cc:96] loaded metadata for tablet 3416459feffc4f7f98e2d0e2b8109201 (table default.Table_d0cd234de1cb4bb485da3d10b527eb65 [id=4ea51139b7824638a5ad1eaf2c0f11f8])
I20251211 20:52:43.094998 710 leader_election.cc:304] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.095386 2499 raft_consensus.cc:2804] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:43.095486 2499 raft_consensus.cc:697] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:43.095840 2499 consensus_queue.cc:237] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.100025 2415 tablet_loader.cc:96] loaded metadata for tablet 36d91772201d4c5b928d9dee3f583eb4 (table default.Table_b3b56f469dc54741924e5b55045fa626 [id=471a1295a9534492aff0ae89a5362a1a])
I20251211 20:52:43.101382 2415 tablet_loader.cc:96] loaded metadata for tablet 373d3f03d7e0408cbe06955f8f60b16c (table default.Table_43e9cb840d48444b9360e2fa8671dfb8 [id=f6cab1da32324fe6afc1df2007f0bdbf])
I20251211 20:52:43.101737 2460 raft_consensus.cc:493] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.101897 2460 raft_consensus.cc:515] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.102553 2460 leader_election.cc:290] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.103492 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb0fc179a337469eae056c5bb3815e6b" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:43.104178 847 leader_election.cc:343] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: Tablet not RUNNING: STOPPED
W20251211 20:52:43.105254 847 leader_election.cc:343] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: bb0fc179a337469eae056c5bb3815e6b
I20251211 20:52:43.105365 847 leader_election.cc:304] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:43.105836 2460 raft_consensus.cc:2749] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:43.107493 1168 tablet_bootstrap.cc:654] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.107860 2415 tablet_loader.cc:96] loaded metadata for tablet 3809a89dfb31442d99455e06733b132e (table default.Table_f2301327ddd04d278201a1cb0da3afe8 [id=ace2fe24e1504556a7b7bd455d7c6c8c])
I20251211 20:52:43.108284 2415 tablet_loader.cc:96] loaded metadata for tablet 39e054bb3dc54c3d89972e6844b580e1 (table default.Table_6b8536dae8ce48be97d778489d1da99c [id=50493a9594aa423cb79ed4f4235a5e4a])
I20251211 20:52:43.108935 2415 tablet_loader.cc:96] loaded metadata for tablet 3b2c6064c5eb4c0f8979c485f5d85a75 (table default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f])
I20251211 20:52:43.114171 2415 tablet_loader.cc:96] loaded metadata for tablet 3c865c99e9314f5f91658b9126da4d39 (table default.Table_23402fab961249179b57df4d79b53a18 [id=7f966d5cdad3423bb770414802a3cf7b])
I20251211 20:52:43.115146 2415 tablet_loader.cc:96] loaded metadata for tablet 3de5b34b96404cdc9c191fed7f055638 (table default.Table_7215dcc06d384c7d94b29ff2ea8ee8ee [id=593f8b9b01134fb09a1c7232509635bb])
I20251211 20:52:43.116367 1160 tablet_bootstrap.cc:492] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:43.116603 1160 ts_tablet_manager.cc:1403] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.147s user 0.004s sys 0.000s
I20251211 20:52:43.117419 1160 raft_consensus.cc:359] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.117674 1160 raft_consensus.cc:385] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.117774 1160 raft_consensus.cc:740] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.118053 1160 consensus_queue.cc:260] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.118713 1160 ts_tablet_manager.cc:1434] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:43.119055 1160 tablet_bootstrap.cc:492] T e4647105ab57454d883ebfcde7bde891 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:43.120626 1160 tablet_bootstrap.cc:654] T e4647105ab57454d883ebfcde7bde891 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.121223 2415 tablet_loader.cc:96] loaded metadata for tablet 3dfea28c44d84c68850db65a30d52ee6 (table default.Table_c504f8f3796f43b697488bd604268543 [id=2af76b9d32094a4ca656060a4478e08a])
I20251211 20:52:43.122835 2415 tablet_loader.cc:96] loaded metadata for tablet 3e6f62c2deff4c11b137fc0cba5ee234 (table default.Table_51db250005ed4b3d947f32acd1d62563 [id=fdfaa0b24d2f4736aee695ed228212cf])
I20251211 20:52:43.123736 1168 tablet_bootstrap.cc:492] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:43.124042 1168 ts_tablet_manager.cc:1403] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.035s user 0.009s sys 0.003s
I20251211 20:52:43.125450 1168 raft_consensus.cc:359] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.125839 1168 raft_consensus.cc:385] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.125982 1168 raft_consensus.cc:740] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.126488 1168 consensus_queue.cc:260] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.127494 1168 ts_tablet_manager.cc:1434] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:43.128068 1168 tablet_bootstrap.cc:492] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:43.131407 1168 tablet_bootstrap.cc:654] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.132725 2415 tablet_loader.cc:96] loaded metadata for tablet 3fc173c396ff4af5a8589cff18165e85 (table default.Table_5e5286ea174048f8ba9abd14daf14bcf [id=a455f6ef64ae44f78972d14b2ea9654c])
I20251211 20:52:43.140019 2415 tablet_loader.cc:96] loaded metadata for tablet 40475640009142439755cba2f870822c (table default.Table_37fd7b6c68b64deaa3fb0ac9a1965eff [id=05826ee49d07424aa08e105c165c497c])
I20251211 20:52:43.140581 2415 tablet_loader.cc:96] loaded metadata for tablet 40722aef4e194ff094d271a079bda9d9 (table default.Table_12f41f708f1949598d6176d89777aa69 [id=3309205faabf4a42acadc43766e4f57f])
I20251211 20:52:43.141170 2415 tablet_loader.cc:96] loaded metadata for tablet 41627cb27be8443fa37e514f9b3825a6 (table default.Table_717a9e3f73be421585b11e3cdad48b64 [id=bd5290958c7249f39a6cc51f631f42a9])
I20251211 20:52:43.141601 2415 tablet_loader.cc:96] loaded metadata for tablet 41bf5b5111f64dfa91ed76ed4ab4f6f7 (table default.Table_6814df0b3dd640f58f421a6ed9010686 [id=aa4a2053a7cf478884770f8e36a02fc4])
I20251211 20:52:43.142262 2415 tablet_loader.cc:96] loaded metadata for tablet 41d17a49e22746c0904c8b96626262a6 (table default.Table_b8ccf690baa544a8a6f41b0c6eea6b38 [id=2ee7c64dbda8471ea684bd515d4f2f4a])
I20251211 20:52:43.143432 2415 tablet_loader.cc:96] loaded metadata for tablet 43044ba5660345eaa2b4e6562cc8d448 (table default.Table_54b89a8f30cf414590cfd7a5773bf326 [id=c3d36e0944a74f6eb98b9bb510d8352a])
I20251211 20:52:43.144084 2415 tablet_loader.cc:96] loaded metadata for tablet 4388893262b54ad8be1bdf82516031a5 (table default.Table_13a8009dbcc342d98426f71c053a964a [id=eb64e2c8b06a44afbb4aa2c4df0cac69])
I20251211 20:52:43.145110 2415 tablet_loader.cc:96] loaded metadata for tablet 44e2d2bdecf247baaac26e06bccc94de (table default.Table_6a2c9aead0a640b79bb4c9d6f2258e67 [id=bde705b86af14b11be4a2e1fc236b37a])
I20251211 20:52:43.145429 2525 tablet.cc:1722] T 72e23f3033e14c9db6de8539830bac3b P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:43.145754 2415 tablet_loader.cc:96] loaded metadata for tablet 46986e9537d54515b8de7a3c5dcaedf0 (table default.Table_c993b3973aa64eabb16fe02f9514b161 [id=7f24fb8df80943cb836588beda1de7cc])
I20251211 20:52:43.146230 2415 tablet_loader.cc:96] loaded metadata for tablet 46b225ef696a45088a4816d18fa2e500 (table default.Table_b36aed5743884aa7bca0cb01b2af1bac [id=440072d0cbe248c381c8086eb62f1fcd])
I20251211 20:52:43.146617 2415 tablet_loader.cc:96] loaded metadata for tablet 46cb1d16a0e94b3a833f32c635213fd2 (table default.Table_3482fbfc5b144a7198a7d01aa003cb6b [id=1bdaf5ce0f6b4b99a038f9f9efb828f6])
I20251211 20:52:43.149892 2415 tablet_loader.cc:96] loaded metadata for tablet 46ddf50b7e5c45e4b9a6483bb4346873 (table default.Table_18ec995eeab648dd8f0399b55e571ece [id=a362f3dbfa82447483dc50751ed0a9b8])
I20251211 20:52:43.150370 2415 tablet_loader.cc:96] loaded metadata for tablet 47428f41973642dbbd42ba051ea3d3b0 (table default.Table_68c2d3a1dc614ca4bcf28180d6e4faf8 [id=7e7414fd3d1e4ec2b7372f5783297766])
I20251211 20:52:43.150699 1166 tablet_bootstrap.cc:492] T fa79b67b9eee45e2b41c4a90967606cc P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:43.151041 1166 ts_tablet_manager.cc:1403] T fa79b67b9eee45e2b41c4a90967606cc P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.170s user 0.002s sys 0.027s
I20251211 20:52:43.152040 1166 raft_consensus.cc:359] T fa79b67b9eee45e2b41c4a90967606cc P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.152305 1166 raft_consensus.cc:385] T fa79b67b9eee45e2b41c4a90967606cc P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.152400 1166 raft_consensus.cc:740] T fa79b67b9eee45e2b41c4a90967606cc P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.152674 2503 raft_consensus.cc:493] T 1578887b2f5748cd83a6fbdec2f3fc99 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.152778 2503 raft_consensus.cc:515] T 1578887b2f5748cd83a6fbdec2f3fc99 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.153362 2503 leader_election.cc:290] T 1578887b2f5748cd83a6fbdec2f3fc99 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:43.154098 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1578887b2f5748cd83a6fbdec2f3fc99" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:43.154098 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1578887b2f5748cd83a6fbdec2f3fc99" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:43.154642 983 leader_election.cc:343] T 1578887b2f5748cd83a6fbdec2f3fc99 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:43.154803 982 leader_election.cc:343] T 1578887b2f5748cd83a6fbdec2f3fc99 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.154888 982 leader_election.cc:304] T 1578887b2f5748cd83a6fbdec2f3fc99 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.155241 2503 raft_consensus.cc:2749] T 1578887b2f5748cd83a6fbdec2f3fc99 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:43.155490 1166 consensus_queue.cc:260] T fa79b67b9eee45e2b41c4a90967606cc P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.156828 2415 tablet_loader.cc:96] loaded metadata for tablet 4870da0224e048ed82f227d7ba676a94 (table default.Table_595d8c100e69422ab5e6d44cce9e733c [id=28f3ebe9b7a64b5e89404b0407b0e5ae])
I20251211 20:52:43.157840 2415 tablet_loader.cc:96] loaded metadata for tablet 497c10742f01476db885bd70c4d2c491 (table default.Table_18b21232dfa74378b27defe672bb2574 [id=602d7eee703948d99a0edcb7cac0651b])
I20251211 20:52:43.158206 2415 tablet_loader.cc:96] loaded metadata for tablet 49a33084c0fc460eb9592224b6dfc279 (table default.Table_6b98cb90147a43c4aee6b39240c5edc4 [id=9c24845e4efe475fa792a51ce8a66be6])
I20251211 20:52:43.160753 1160 tablet_bootstrap.cc:492] T e4647105ab57454d883ebfcde7bde891 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:43.160907 1160 ts_tablet_manager.cc:1403] T e4647105ab57454d883ebfcde7bde891 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.042s user 0.004s sys 0.000s
I20251211 20:52:43.161571 1160 raft_consensus.cc:359] T e4647105ab57454d883ebfcde7bde891 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.161734 1160 raft_consensus.cc:385] T e4647105ab57454d883ebfcde7bde891 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.161777 1160 raft_consensus.cc:740] T e4647105ab57454d883ebfcde7bde891 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.161969 1160 consensus_queue.cc:260] T e4647105ab57454d883ebfcde7bde891 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.162470 1160 ts_tablet_manager.cc:1434] T e4647105ab57454d883ebfcde7bde891 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251211 20:52:43.162774 1160 tablet_bootstrap.cc:492] T a9bb22dce13f4c9da76ca446715cd99a P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:43.163777 2429 consensus_queue.cc:1048] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:43.165308 1160 tablet_bootstrap.cc:654] T a9bb22dce13f4c9da76ca446715cd99a P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.168358 2415 tablet_loader.cc:96] loaded metadata for tablet 4acbc679266b4efd815e00a855751ace (table default.Table_d44ac0044b77421f88b8a7b00811c8d3 [id=9800f52437ab4fb6b5a46e4861154750])
I20251211 20:52:43.169871 2415 tablet_loader.cc:96] loaded metadata for tablet 4b55b82d5fc94aabb50d46a8d3a66351 (table default.Table_434ed373bf494a40b55b1f275d648d28 [id=1224a8665eaf45c1a4ab549024bb4200])
I20251211 20:52:43.175395 1166 ts_tablet_manager.cc:1434] T fa79b67b9eee45e2b41c4a90967606cc P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.024s user 0.002s sys 0.000s
I20251211 20:52:43.176842 2415 tablet_loader.cc:96] loaded metadata for tablet 4c0455a5b4f44412803475d1c4852b48 (table default.Table_e00092673b0e4f09b8d7eec63e1ee034 [id=18497e06d7254441a4655ce1cdc5a9bf])
I20251211 20:52:43.177448 1166 tablet_bootstrap.cc:492] T ee7b93b0b87541d9af6eb9e68b4cea41 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:43.179625 1168 tablet_bootstrap.cc:492] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:43.179972 1168 ts_tablet_manager.cc:1403] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.052s user 0.003s sys 0.005s
I20251211 20:52:43.180883 1166 tablet_bootstrap.cc:654] T ee7b93b0b87541d9af6eb9e68b4cea41 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.181679 1168 raft_consensus.cc:359] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.182128 1168 raft_consensus.cc:385] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.182318 1168 raft_consensus.cc:740] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.182384 2415 tablet_loader.cc:96] loaded metadata for tablet 4c7b74f370b042ee8575f821c07dc085 (table default.Table_731826f26f9940b8afe425d694b642d9 [id=17dd9cea9ac844128ff9f82adb3f919f])
I20251211 20:52:43.182845 1168 consensus_queue.cc:260] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.183324 2415 tablet_loader.cc:96] loaded metadata for tablet 4cae3a6362be4834acd9b06123694a2b (table default.Table_d71f88120b9d4408a1b04988573d137d [id=6dae16bd96534a0c89619742c262e081])
I20251211 20:52:43.184428 2415 tablet_loader.cc:96] loaded metadata for tablet 4cf0cde801bf44449e539d2799b895a7 (table default.Table_907c596bdddc4edca25d14cbacebb23f [id=321235280d1d4fa0bae5cc0333860f85])
I20251211 20:52:43.185683 2415 tablet_loader.cc:96] loaded metadata for tablet 4d03f2f064c3420d817a46e4153054e8 (table default.Table_4ec0d87202fc419380c2542d827087de [id=d9829c79b55844ec92b1b0cf594ab96b])
I20251211 20:52:43.186272 2325 raft_consensus.cc:515] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.186584 1160 tablet_bootstrap.cc:492] T a9bb22dce13f4c9da76ca446715cd99a P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:43.186721 1160 ts_tablet_manager.cc:1403] T a9bb22dce13f4c9da76ca446715cd99a P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.001s
I20251211 20:52:43.186981 2325 leader_election.cc:290] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.187422 1160 raft_consensus.cc:359] T a9bb22dce13f4c9da76ca446715cd99a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.187600 1160 raft_consensus.cc:385] T a9bb22dce13f4c9da76ca446715cd99a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.187652 1160 raft_consensus.cc:740] T a9bb22dce13f4c9da76ca446715cd99a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.187851 1160 consensus_queue.cc:260] T a9bb22dce13f4c9da76ca446715cd99a P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.188241 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 917) took 3364 ms. Trace:
I20251211 20:52:43.188316 1165 rpcz_store.cc:276] 1211 20:52:39.824111 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.824171 (+ 60us) service_pool.cc:225] Handling call
1211 20:52:42.706543 (+2882372us) ts_tablet_manager.cc:1151] Deleting tablet bb0fc179a337469eae056c5bb3815e6b
1211 20:52:42.706557 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:42.839436 (+132879us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:42.912631 (+ 73195us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:43.188215 (+275584us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3,"tablet-delete.queue_time_us":2876037}
I20251211 20:52:43.188509 1160 ts_tablet_manager.cc:1434] T a9bb22dce13f4c9da76ca446715cd99a P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:43.188779 1160 tablet_bootstrap.cc:492] T 8b793241ee3b42b1a38e9f2f05e1173b P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:43.188782 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 918) took 3361 ms. Trace:
I20251211 20:52:43.188853 1165 rpcz_store.cc:276] 1211 20:52:39.827292 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.827344 (+ 52us) service_pool.cc:225] Handling call
1211 20:52:43.188635 (+3361291us) ts_tablet_manager.cc:1151] Deleting tablet fbc2d4aea084492c8270ad1d87096e58
1211 20:52:43.188648 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:43.188763 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3358045}
I20251211 20:52:43.189105 1165 tablet_replica.cc:333] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:43.189280 1165 raft_consensus.cc:2243] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:43.189530 1165 pending_rounds.cc:70] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919: Trying to abort 1 pending ops.
I20251211 20:52:43.189579 1165 pending_rounds.cc:77] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432131147407360 op_type: NO_OP noop_request { }
I20251211 20:52:43.189654 1165 raft_consensus.cc:2889] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:43.189715 1165 raft_consensus.cc:2272] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:43.189783 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2e33606930774fe4aafc6e3819618af8" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:43.190318 1160 tablet_bootstrap.cc:654] T 8b793241ee3b42b1a38e9f2f05e1173b P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:43.190280 711 leader_election.cc:343] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.192479 2415 tablet_loader.cc:96] loaded metadata for tablet 4e123b96b7534f07ae48a808f47e21a1 (table default.Table_31d573e7af144e9997dfe480f8466902 [id=f689abc175c94366bfda8217545156aa])
I20251211 20:52:43.193441 2415 tablet_loader.cc:96] loaded metadata for tablet 4fc56df5ccb94198b5b0e516eeae6941 (table default.Table_494ee7d3e0dc4a47b47082643a3134ce [id=42422ac9805e49cbab71e262e73098b3])
W20251211 20:52:43.194864 713 connection.cc:441] server connection from 127.0.0.1:37886 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 917) could send its response
W20251211 20:52:43.195008 713 connection.cc:441] server connection from 127.0.0.1:37886 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 918) could send its response
I20251211 20:52:43.197927 1168 ts_tablet_manager.cc:1434] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.018s user 0.002s sys 0.000s
I20251211 20:52:43.199278 1168 tablet_bootstrap.cc:492] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:43.190287 1165 ts_tablet_manager.cc:1916] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:43.204070 2415 tablet_loader.cc:96] loaded metadata for tablet 4fe5c534b572491c911aae01eaa79bae (table default.Table_c1f9dca5c2144d50b67681dcc6800e77 [id=14fa8debfe5443ce81587f02dfa15847])
I20251211 20:52:43.204829 2415 tablet_loader.cc:96] loaded metadata for tablet 516f7181aaac48cdadf9fbc08d0d3f57 (table default.Table_87f092378445464ab48c155fae469f29 [id=0f029363985443529615cdd1f8bbd78d])
I20251211 20:52:43.205341 2415 tablet_loader.cc:96] loaded metadata for tablet 51d12723f5194afb97d2c05fcbaf3776 (table default.Table_823505200e3142dbbe8cd56ac78daf55 [id=ec0bb10933fa4529bef3580bee6409e7])
I20251211 20:52:43.209550 1168 tablet_bootstrap.cc:654] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.187345 1048 raft_consensus.cc:2468] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:43.196704 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2e33606930774fe4aafc6e3819618af8" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:43.219463 1042 raft_consensus.cc:3060] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.215952 2503 raft_consensus.cc:493] T fa79b67b9eee45e2b41c4a90967606cc P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.222163 2503 raft_consensus.cc:515] T fa79b67b9eee45e2b41c4a90967606cc P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.222841 2503 leader_election.cc:290] T fa79b67b9eee45e2b41c4a90967606cc P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:43.223263 2415 tablet_loader.cc:96] loaded metadata for tablet 52b1bf7f19584d5297b56e5aacb98560 (table default.Table_b929f7410fd44316b3a3571d02066b45 [id=e9dcc3d777524e70b9653ec6a2df7da0])
I20251211 20:52:43.223387 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fa79b67b9eee45e2b41c4a90967606cc" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:43.223728 2501 raft_consensus.cc:493] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.223871 2501 raft_consensus.cc:515] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.224117 2415 tablet_loader.cc:96] loaded metadata for tablet 53b4116db0a44dfbb4c1ec5b2acebc67 (table default.Table_6829fee154a74021a1410d7535c0c89f [id=aea8a10b336548f78e1c093ce01a22a9])
I20251211 20:52:43.224273 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fa79b67b9eee45e2b41c4a90967606cc" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:43.224462 2415 tablet_loader.cc:96] loaded metadata for tablet 5453f68789fd4e89b69e98860050865b (table default.Table_b7bf8c5ad5304175a1f2f6a03887f725 [id=50d88b663a4d4891b113aee9c5906699])
I20251211 20:52:43.224556 2501 leader_election.cc:290] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.223245 911 raft_consensus.cc:3060] T f737a7bfb0f44196abdc831bb22d5285 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.224619 710 leader_election.cc:304] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.225023 2415 tablet_loader.cc:96] loaded metadata for tablet 56a51729b1824027baf7d6d1d5645846 (table default.Table_fde09a02a1664bc5bad79b2e0036b42a [id=3c1a33a40f334749b4b79833bfc6063a])
W20251211 20:52:43.225133 982 leader_election.cc:343] T fa79b67b9eee45e2b41c4a90967606cc P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.225450 2499 raft_consensus.cc:2804] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:43.225562 2499 raft_consensus.cc:697] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:43.225591 2415 tablet_loader.cc:96] loaded metadata for tablet 5773c13034714a008c8d7dbe777b4828 (table default.Table_66eb1b021eed464f82f9cd223c88297b [id=94ff09564cd649c7a2d0e94b0f3c7b3a])
I20251211 20:52:43.225582 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "41c30bded6504bbc9f0af8f34dc81e8d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:43.225736 1048 raft_consensus.cc:2468] T 41c30bded6504bbc9f0af8f34dc81e8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:43.225778 2499 consensus_queue.cc:237] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.225996 2415 tablet_loader.cc:96] loaded metadata for tablet 586f9d1d24ad40cea44a1704f85b2296 (table default.Table_11d5dfcc3a7d4db59119bf9c9666c50a [id=011d18ce4d764af8b3e399443dc217e9])
W20251211 20:52:43.226438 983 leader_election.cc:343] T fa79b67b9eee45e2b41c4a90967606cc P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.226538 2415 tablet_loader.cc:96] loaded metadata for tablet 59acb8474e114bf7ae7f1f043d611459 (table default.Table_c7ed8c0c309c40a3a371ffa130b47cd2 [id=ed1d0670426945aba888b818ad657225])
I20251211 20:52:43.226588 983 leader_election.cc:304] T fa79b67b9eee45e2b41c4a90967606cc P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.227108 710 leader_election.cc:304] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:43.227370 2503 raft_consensus.cc:2749] T fa79b67b9eee45e2b41c4a90967606cc P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:43.227391 2338 raft_consensus.cc:2804] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:43.227447 2338 raft_consensus.cc:493] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:43.227504 2338 raft_consensus.cc:3060] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:43.230736 711 consensus_peers.cc:597] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:43.226863 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "41c30bded6504bbc9f0af8f34dc81e8d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:43.232067 711 leader_election.cc:343] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.237747 1168 tablet_bootstrap.cc:492] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:43.238126 1168 ts_tablet_manager.cc:1403] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.039s user 0.006s sys 0.001s
I20251211 20:52:43.239464 1168 raft_consensus.cc:359] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.239989 1168 raft_consensus.cc:385] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.240190 1168 raft_consensus.cc:740] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.241153 2415 tablet_loader.cc:96] loaded metadata for tablet 59e955aa07c14fbbb347b88eca59a74e (table default.Table_f84cd3de863b446894133bb40aaf1a39 [id=f4d608cc41d0408486d3f6b0a09383a4])
I20251211 20:52:43.242823 2415 tablet_loader.cc:96] loaded metadata for tablet 5a42998dbedc465f9e39e0aa45285368 (table default.Table_868d96d23ca040158aaea84effd252cb [id=1aaf20327cd24cc682293c44cb5295f2])
I20251211 20:52:43.248514 1160 tablet_bootstrap.cc:492] T 8b793241ee3b42b1a38e9f2f05e1173b P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:43.248796 1160 ts_tablet_manager.cc:1403] T 8b793241ee3b42b1a38e9f2f05e1173b P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.060s user 0.004s sys 0.003s
I20251211 20:52:43.249682 1160 raft_consensus.cc:359] T 8b793241ee3b42b1a38e9f2f05e1173b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.250067 1160 raft_consensus.cc:385] T 8b793241ee3b42b1a38e9f2f05e1173b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.250162 1160 raft_consensus.cc:740] T 8b793241ee3b42b1a38e9f2f05e1173b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.250444 1160 consensus_queue.cc:260] T 8b793241ee3b42b1a38e9f2f05e1173b P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.240732 1168 consensus_queue.cc:260] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.253836 1160 ts_tablet_manager.cc:1434] T 8b793241ee3b42b1a38e9f2f05e1173b P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251211 20:52:43.254194 1160 tablet_bootstrap.cc:492] T f2d6c9781b4d4c37bc08703347963d5d P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:43.258582 1168 ts_tablet_manager.cc:1434] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.020s user 0.003s sys 0.000s
I20251211 20:52:43.259095 2415 tablet_loader.cc:96] loaded metadata for tablet 5b3e8ec3faf445ac9b746e65dc4dbfe0 (table default.Table_7fe80ff5483a4b0292091119980f6639 [id=3c3462a0c09748cea0d2ffb544ae2f13])
I20251211 20:52:43.259548 2415 tablet_loader.cc:96] loaded metadata for tablet 5b5fb5f22c824fd6af16db188cf32e45 (table default.Table_a32f150ce5604ec999ac5f3edc6324b4 [id=f98e1455f27a4c98a9c484931ed502b3])
I20251211 20:52:43.262097 2415 tablet_loader.cc:96] loaded metadata for tablet 5bd801124df84a6c96f4912993f1bdfd (table default.Table_80e3542d21b64eca991b2e81ac80ac7c [id=44f2f2bf53af4b038ced294f20db9e48])
I20251211 20:52:43.262385 1160 tablet_bootstrap.cc:654] T f2d6c9781b4d4c37bc08703347963d5d P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.263651 2415 tablet_loader.cc:96] loaded metadata for tablet 5d225553d0984b6e9685b5841616c9d8 (table default.Table_25d3985a897841b8b55ad7c44bbff154 [id=fde40e5a01ee40c48ca653928036c9a1])
I20251211 20:52:43.264159 1168 tablet_bootstrap.cc:492] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:43.267526 2460 raft_consensus.cc:493] T a9bb22dce13f4c9da76ca446715cd99a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.267724 2460 raft_consensus.cc:515] T a9bb22dce13f4c9da76ca446715cd99a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.268381 2460 leader_election.cc:290] T a9bb22dce13f4c9da76ca446715cd99a P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.268445 2415 tablet_loader.cc:96] loaded metadata for tablet 5df87ea7d28340da84ef141c92efc710 (table default.Table_6bfa9e94e0b24faaa8240f4280e3d535 [id=4358b12fe36941e18e574120e6d892c8])
I20251211 20:52:43.268827 2415 tablet_loader.cc:96] loaded metadata for tablet 5e47af616e1b4d2db3780c1d67cdcb47 (table default.Table_d87f970fdabb4d10871f55bb8193bd4d [id=052edcbde60c41449d468c8a449fc3f1])
I20251211 20:52:43.269166 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9bb22dce13f4c9da76ca446715cd99a" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:43.269423 2415 tablet_loader.cc:96] loaded metadata for tablet 5f2c14c7a972414e8328e0b388bde0a4 (table default.Table_7287bc8f28a64682b352f9adae2ec224 [id=c2883f35f01248219793761e8f0984aa])
I20251211 20:52:43.269606 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9bb22dce13f4c9da76ca446715cd99a" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:43.270280 847 leader_election.cc:304] T a9bb22dce13f4c9da76ca446715cd99a P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:43.270851 2460 raft_consensus.cc:2749] T a9bb22dce13f4c9da76ca446715cd99a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:43.270925 2415 tablet_loader.cc:96] loaded metadata for tablet 617db82d7747484a91e17219eedfe971 (table default.Table_e345037da95a4c16b6e8b650f0ba3911 [id=efb341043b8748dd883699a699425bbd])
I20251211 20:52:43.271296 2415 tablet_loader.cc:96] loaded metadata for tablet 6212b3b4425543168837946a54d60834 (table default.Table_8c30c4151d484db0bb9e2b9ca262248d [id=2d489cd7d73e403d84bf10960e6a2dfb])
I20251211 20:52:43.272295 1168 tablet_bootstrap.cc:654] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.272745 1166 tablet_bootstrap.cc:492] T ee7b93b0b87541d9af6eb9e68b4cea41 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:43.272919 1166 ts_tablet_manager.cc:1403] T ee7b93b0b87541d9af6eb9e68b4cea41 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.096s user 0.012s sys 0.007s
I20251211 20:52:43.273665 1166 raft_consensus.cc:359] T ee7b93b0b87541d9af6eb9e68b4cea41 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.273850 1166 raft_consensus.cc:385] T ee7b93b0b87541d9af6eb9e68b4cea41 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.273902 1166 raft_consensus.cc:740] T ee7b93b0b87541d9af6eb9e68b4cea41 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.278335 2503 consensus_queue.cc:1048] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:43.279345 2501 raft_consensus.cc:493] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.279932 2501 raft_consensus.cc:515] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.281050 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "50f9f29c064c4f938a0f20f302ab5e8a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:43.281544 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "50f9f29c064c4f938a0f20f302ab5e8a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
W20251211 20:52:43.281673 711 leader_election.cc:343] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.281755 1048 raft_consensus.cc:2468] T 50f9f29c064c4f938a0f20f302ab5e8a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:43.282238 710 leader_election.cc:304] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.282534 2429 raft_consensus.cc:2804] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:43.282624 2429 raft_consensus.cc:493] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:43.282722 2429 raft_consensus.cc:3060] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.282953 1160 tablet_bootstrap.cc:492] T f2d6c9781b4d4c37bc08703347963d5d P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:43.283133 1160 ts_tablet_manager.cc:1403] T f2d6c9781b4d4c37bc08703347963d5d P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.000s
I20251211 20:52:43.283113 2501 leader_election.cc:290] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.288935 2273 raft_consensus.cc:515] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.289638 2273 leader_election.cc:290] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.289800 1166 consensus_queue.cc:260] T ee7b93b0b87541d9af6eb9e68b4cea41 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.292006 1166 ts_tablet_manager.cc:1434] T ee7b93b0b87541d9af6eb9e68b4cea41 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.019s user 0.002s sys 0.000s
I20251211 20:52:43.292778 1166 tablet_bootstrap.cc:492] T 0763a475527546bb83cee8fb823893ac P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:43.293140 1042 raft_consensus.cc:2468] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:43.297019 1168 tablet_bootstrap.cc:492] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:43.299477 2460 raft_consensus.cc:493] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.299664 2460 raft_consensus.cc:515] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.300133 710 leader_election.cc:304] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.297351 2273 raft_consensus.cc:493] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.300271 2460 leader_election.cc:290] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.300343 2273 raft_consensus.cc:515] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.300915 2273 leader_election.cc:290] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.301010 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d9cf0d92ccc7419cb6e7694e8e5c7bfb" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:43.301079 2273 raft_consensus.cc:2804] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:43.301203 2273 raft_consensus.cc:697] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:43.301362 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d9cf0d92ccc7419cb6e7694e8e5c7bfb" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:43.301433 2273 consensus_queue.cc:237] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.301522 1042 raft_consensus.cc:3060] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.303016 1166 tablet_bootstrap.cc:654] T 0763a475527546bb83cee8fb823893ac P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.304225 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6a61cf8f5de14adaa4a1de5bddde9a2f" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
W20251211 20:52:43.304404 847 leader_election.cc:343] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: e4b5a7553110424d901788c7a2ecdce6
I20251211 20:52:43.304441 1047 raft_consensus.cc:2468] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:43.304898 710 leader_election.cc:304] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:43.305281 2501 raft_consensus.cc:2804] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:43.305387 2501 raft_consensus.cc:493] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:43.305462 2501 raft_consensus.cc:3060] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.303969 1168 ts_tablet_manager.cc:1403] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.040s user 0.004s sys 0.001s
W20251211 20:52:43.311213 711 leader_election.cc:343] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.312494 1168 raft_consensus.cc:359] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.312909 1168 raft_consensus.cc:385] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.313220 1168 raft_consensus.cc:740] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.313457 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6a61cf8f5de14adaa4a1de5bddde9a2f" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:43.313671 1168 consensus_queue.cc:260] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.314960 1168 ts_tablet_manager.cc:1434] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20251211 20:52:43.315297 2507 raft_consensus.cc:493] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.315456 2507 raft_consensus.cc:515] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.316152 2507 leader_election.cc:290] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:43.316874 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "efe4d6bb07374294b8ab50a9fe605f87" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:43.317303 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "efe4d6bb07374294b8ab50a9fe605f87" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:43.317449 982 leader_election.cc:343] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.320317 1168 tablet_bootstrap.cc:492] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:43.321467 847 leader_election.cc:343] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: e4b5a7553110424d901788c7a2ecdce6
I20251211 20:52:43.321619 847 leader_election.cc:304] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:43.322027 2460 raft_consensus.cc:2749] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:43.326771 983 leader_election.cc:343] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.326920 983 leader_election.cc:304] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.327452 2507 raft_consensus.cc:2749] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:43.321184 711 leader_election.cc:343] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.330273 1160 raft_consensus.cc:359] T f2d6c9781b4d4c37bc08703347963d5d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.331070 1160 raft_consensus.cc:385] T f2d6c9781b4d4c37bc08703347963d5d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.331190 1160 raft_consensus.cc:740] T f2d6c9781b4d4c37bc08703347963d5d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.331516 1160 consensus_queue.cc:260] T f2d6c9781b4d4c37bc08703347963d5d P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.332916 1160 ts_tablet_manager.cc:1434] T f2d6c9781b4d4c37bc08703347963d5d P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.050s user 0.003s sys 0.000s
I20251211 20:52:43.333428 1160 tablet_bootstrap.cc:492] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:43.337911 1160 tablet_bootstrap.cc:654] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.338516 1168 tablet_bootstrap.cc:654] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.342276 2415 tablet_loader.cc:96] loaded metadata for tablet 622e513e55d749f2a0105b5d6f9444c8 (table default.Table_ec0bf98cd1ce42ceacacde30208d5d04 [id=b8fdb6772ace427e8273983d1e791956])
I20251211 20:52:43.342948 2415 tablet_loader.cc:96] loaded metadata for tablet 62afbd04479f4dff87aa18bef705c6e9 (table default.Table_4b87bd2fa22f4d59b01baeca62ce98f8 [id=5a1c2ebabb5c49d5afe8f668d2bfa3e8])
I20251211 20:52:43.347787 913 raft_consensus.cc:3060] T 8b793241ee3b42b1a38e9f2f05e1173b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.353724 2415 tablet_loader.cc:96] loaded metadata for tablet 6365572dac1449428ed371cf4e3545de (table default.Table_98f173a3b5454e16ac5680ec2465a09c [id=ab5b964850544c718f621b5ec87e3372])
I20251211 20:52:43.354636 2415 tablet_loader.cc:96] loaded metadata for tablet 63f368b53f374bbe8e1ce9a4757d1639 (table default.Table_08f40dd56ad24729abbd776dabf07b8f [id=81c5998c4b8d4b7cb68ad8adb957cc0a])
I20251211 20:52:43.355890 2415 tablet_loader.cc:96] loaded metadata for tablet 645b08b33676420c96e880de3d460829 (table default.Table_e02933dd91ce4fe7a40d1c3b2f9b2d4b [id=6d42f260246e420d8818b3841926302b])
I20251211 20:52:43.359006 2415 tablet_loader.cc:96] loaded metadata for tablet 652828db3e31467fad8391e6cfe70846 (table default.Table_20456517a58b40fabd534461d2ab6cc0 [id=f93e5d8158d74f7cb3b77ddefe065c8f])
I20251211 20:52:43.359550 2415 tablet_loader.cc:96] loaded metadata for tablet 664c6a3afe5b4a4caf87dede87f199ba (table default.Table_c2f02a2463774499a7858b0a53ee965d [id=c342962bfd9242c6811897d1c980dfe9])
I20251211 20:52:43.360500 2415 tablet_loader.cc:96] loaded metadata for tablet 676faec7f3d1488f99b220f8e968e9a0 (table default.Table_ab826a3eec8e4c4aaa8d4021d87dd382 [id=a2bbfccfe3974c378521d5ef2333df86])
I20251211 20:52:43.360543 1168 tablet_bootstrap.cc:492] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:43.360738 1168 ts_tablet_manager.cc:1403] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.041s user 0.001s sys 0.003s
I20251211 20:52:43.361038 2415 tablet_loader.cc:96] loaded metadata for tablet 683d0f2068fb4a6ebc9c169ad7b455ce (table default.Table_7e2c82ae1dbe44078774e5229684130d [id=a046f539a58a4c279d461cf9510c97d5])
I20251211 20:52:43.361663 1168 raft_consensus.cc:359] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.361994 1168 raft_consensus.cc:385] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.362133 1168 raft_consensus.cc:740] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.362397 1168 consensus_queue.cc:260] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.363791 1168 ts_tablet_manager.cc:1434] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251211 20:52:43.364909 1168 tablet_bootstrap.cc:492] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:43.366703 1168 tablet_bootstrap.cc:654] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.369596 2415 tablet_loader.cc:96] loaded metadata for tablet 6888d2a2c6574c93ae0475faf07609f2 (table default.Table_bd71cd88b0354c88aa03799a4362f1b1 [id=2f4a7d71bf18442fa8f3ed7421512ff2])
I20251211 20:52:43.370370 2415 tablet_loader.cc:96] loaded metadata for tablet 69085cd41db846e7a6e03bcfee4377f6 (table default.Table_24737d22fc454a3da12979ae8966c9b3 [id=9a943e65becc4d47828265f814118328])
I20251211 20:52:43.370961 2415 tablet_loader.cc:96] loaded metadata for tablet 694518e6c1ae4032b06b99e9bde2e5dd (table default.Table_b022d39850dc4b8f92b79b8a747f0409 [id=d49d5216afba41d68497f42fc72b1bf0])
I20251211 20:52:43.371417 2415 tablet_loader.cc:96] loaded metadata for tablet 69bfeae3909b45959d591a8bbcb70ce5 (table default.Table_842a98db481041b98480204da25a6419 [id=4fa6a5b9f21e41f4ae0fb953da11a51b])
I20251211 20:52:43.376015 2415 tablet_loader.cc:96] loaded metadata for tablet 6ab29c688eb14134836ed661f9193f9e (table default.Table_6bc2fc890a7a485ba23c3214f772b758 [id=a703dcf9f6244f34878332575ff2324c])
I20251211 20:52:43.381165 1168 tablet_bootstrap.cc:492] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:43.381233 2415 tablet_loader.cc:96] loaded metadata for tablet 6b7dd8232d974a43b99ec9b28075dc8d (table default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55])
I20251211 20:52:43.381614 1168 ts_tablet_manager.cc:1403] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.002s
I20251211 20:52:43.382563 2415 tablet_loader.cc:96] loaded metadata for tablet 7023afec769c4ca3b2242a225004d482 (table default.Table_5da0e9ddf96549c88878f7136d0ba415 [id=3811f8235b2f44cca283401302d613d4])
I20251211 20:52:43.383302 1160 tablet_bootstrap.cc:492] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:43.383280 1168 raft_consensus.cc:359] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.383569 1160 ts_tablet_manager.cc:1403] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.050s user 0.005s sys 0.000s
I20251211 20:52:43.384356 1160 raft_consensus.cc:359] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.384547 1160 raft_consensus.cc:385] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.384639 1160 raft_consensus.cc:740] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.384598 1168 raft_consensus.cc:385] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.384893 1168 raft_consensus.cc:740] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.384910 1160 consensus_queue.cc:260] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.385470 1160 ts_tablet_manager.cc:1434] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:43.385396 1168 consensus_queue.cc:260] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.385872 416 external_mini_cluster.cc:1366] Running /tmp/dist-test-task5TWsQc/build/asan/bin/kudu
/tmp/dist-test-task5TWsQc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.60:42087
--webserver_interface=127.0.104.60
--webserver_port=40729
--builtin_ntp_servers=127.0.104.20:41333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.62:41123,127.0.104.61:40749,127.0.104.60:42087
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20251211 20:52:43.386201 1165 ts_tablet_manager.cc:1929] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:43.386330 1165 log.cc:1199] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/e6ba12f94f5f4d688f4222a4c2fb5436
I20251211 20:52:43.386660 1165 ts_tablet_manager.cc:1950] T e6ba12f94f5f4d688f4222a4c2fb5436 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:43.386682 1168 ts_tablet_manager.cc:1434] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20251211 20:52:43.387039 1042 raft_consensus.cc:2468] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:43.389534 1168 tablet_bootstrap.cc:492] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:43.390239 2415 tablet_loader.cc:96] loaded metadata for tablet 728b4d34376b4f00b5556adc25b46315 (table default.Table_84d90fc27f874a01bf3187d6d02b2053 [id=fa858cefc5bc458ca620f77a25b697a8])
I20251211 20:52:43.390663 2415 tablet_loader.cc:96] loaded metadata for tablet 72af55c8045b4f8f8e116a5782794d81 (table default.Table_2cff6fab088c4d939b13b504c133cc12 [id=b6c4b72063b3428593d5650a160dc66b])
I20251211 20:52:43.391228 710 leader_election.cc:304] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.391319 2415 tablet_loader.cc:96] loaded metadata for tablet 72e23f3033e14c9db6de8539830bac3b (table default.Table_3418cf6b51394b03afec9b03cde8861c [id=c7c19676e1484bbcb73b3bf14f147ae5])
I20251211 20:52:43.391744 2286 raft_consensus.cc:515] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.391918 2415 tablet_loader.cc:96] loaded metadata for tablet 73a4bf965f0b4ac0a63d2c48aeb9767a (table default.Table_0d3816dfd69844518591138a3a844a9a [id=2b6bb7b58170467abe6463a912f608c8])
I20251211 20:52:43.392308 2286 leader_election.cc:290] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.392490 2286 raft_consensus.cc:2804] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:43.392603 2286 raft_consensus.cc:697] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:43.392843 2286 consensus_queue.cc:237] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.394395 1160 tablet_bootstrap.cc:492] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:43.394410 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "940dfd8781de444a9caa78dd363c7451" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:43.395365 711 leader_election.cc:343] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.396407 2460 raft_consensus.cc:493] T e4647105ab57454d883ebfcde7bde891 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.396557 2460 raft_consensus.cc:515] T e4647105ab57454d883ebfcde7bde891 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.396893 1160 tablet_bootstrap.cc:654] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.397135 2460 leader_election.cc:290] T e4647105ab57454d883ebfcde7bde891 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.397598 2460 raft_consensus.cc:493] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.397722 2460 raft_consensus.cc:515] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.397799 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e4647105ab57454d883ebfcde7bde891" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:43.398193 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "940dfd8781de444a9caa78dd363c7451" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:43.398250 2460 leader_election.cc:290] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.398371 1042 raft_consensus.cc:3060] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.401868 1166 tablet_bootstrap.cc:492] T 0763a475527546bb83cee8fb823893ac P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:43.402020 1166 ts_tablet_manager.cc:1403] T 0763a475527546bb83cee8fb823893ac P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.109s user 0.028s sys 0.027s
I20251211 20:52:43.402321 2415 tablet_loader.cc:96] loaded metadata for tablet 77bd176108f24e79a632c798566e9e31 (table default.Table_1021d997007b41208355c31d22f4e26c [id=8c644777215b4617a8ce57691c34ce8c])
I20251211 20:52:43.402361 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e4647105ab57454d883ebfcde7bde891" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
W20251211 20:52:43.402827 847 leader_election.cc:343] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: a2058b67ddf1413db338bfa2f64e7a77
I20251211 20:52:43.402878 1166 raft_consensus.cc:359] T 0763a475527546bb83cee8fb823893ac P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.403017 1166 raft_consensus.cc:385] T 0763a475527546bb83cee8fb823893ac P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.403067 1166 raft_consensus.cc:740] T 0763a475527546bb83cee8fb823893ac P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.403203 2415 tablet_loader.cc:96] loaded metadata for tablet 78f77d8916d84edabbe95ce17d8df2c1 (table default.Table_e2af0e2fbd3d46d595136c7a7bf24e4c [id=ddc12ae6d8ed4587b5ef4da3ef81a055])
I20251211 20:52:43.403231 1166 consensus_queue.cc:260] T 0763a475527546bb83cee8fb823893ac P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:43.403337 847 leader_election.cc:343] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: a2058b67ddf1413db338bfa2f64e7a77
I20251211 20:52:43.403422 847 leader_election.cc:304] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:43.403894 847 leader_election.cc:304] T e4647105ab57454d883ebfcde7bde891 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:43.405406 2507 raft_consensus.cc:493] T 0763a475527546bb83cee8fb823893ac P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.405532 2507 raft_consensus.cc:515] T 0763a475527546bb83cee8fb823893ac P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.406100 2507 leader_election.cc:290] T 0763a475527546bb83cee8fb823893ac P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:43.406112 1166 ts_tablet_manager.cc:1434] T 0763a475527546bb83cee8fb823893ac P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:43.406610 1166 tablet_bootstrap.cc:492] T b85af79c2574443998e847bc19c53b69 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:43.408304 1166 tablet_bootstrap.cc:654] T b85af79c2574443998e847bc19c53b69 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.408846 2415 tablet_loader.cc:96] loaded metadata for tablet 7a0a20f95416425489af94408ac1e97c (table default.Table_903bfd8112e444bcbe3e677773384da5 [id=849873b9731344feb51f5fbefd42576d])
I20251211 20:52:43.409186 2415 tablet_loader.cc:96] loaded metadata for tablet 7a299ee03d0e48aba7c4f71e3e0f381a (table default.Table_6402a801d48e4a53827b1b62b6cdaba1 [id=6876565d3a9c48ce841b830cf75139ff])
I20251211 20:52:43.409168 2460 raft_consensus.cc:2749] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:43.409382 2460 raft_consensus.cc:2749] T e4647105ab57454d883ebfcde7bde891 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:43.409801 2415 tablet_loader.cc:96] loaded metadata for tablet 7b9f569554fb4836ac2b62542c3ecf7e (table default.Table_78f3b1065a0a43d891bb775169c0eaca [id=6ef691749df24251a7a7c5919c219754])
I20251211 20:52:43.410336 2415 tablet_loader.cc:96] loaded metadata for tablet 7be44df39a8f4d6da793479d425fa684 (table default.Table_f6a10556ff104db5bd5ecc9fc380e228 [id=f75813add54e46189ff1bbed82d83ba7])
I20251211 20:52:43.410646 2415 tablet_loader.cc:96] loaded metadata for tablet 7ca98b43863a47c091baae0a6ce65630 (table default.Table_f0a70914acc94231a4e84155c3e4d13f [id=56c070ad80624c698ebe2e01b4929b6e])
I20251211 20:52:43.413836 1166 tablet_bootstrap.cc:492] T b85af79c2574443998e847bc19c53b69 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:43.414075 1166 ts_tablet_manager.cc:1403] T b85af79c2574443998e847bc19c53b69 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20251211 20:52:43.414218 1160 tablet_bootstrap.cc:492] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:43.414345 1160 ts_tablet_manager.cc:1403] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.000s
I20251211 20:52:43.414870 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0763a475527546bb83cee8fb823893ac" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:43.415127 1160 raft_consensus.cc:359] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.415324 1160 raft_consensus.cc:385] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.415393 1160 raft_consensus.cc:740] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
W20251211 20:52:43.415385 983 leader_election.cc:343] T 0763a475527546bb83cee8fb823893ac P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.415628 1160 consensus_queue.cc:260] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.416139 1160 ts_tablet_manager.cc:1434] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:43.416229 1166 raft_consensus.cc:359] T b85af79c2574443998e847bc19c53b69 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.416448 1160 tablet_bootstrap.cc:492] T 22273287aa9245a2a66131fed2c8861c P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:43.416504 1166 raft_consensus.cc:385] T b85af79c2574443998e847bc19c53b69 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.416643 1166 raft_consensus.cc:740] T b85af79c2574443998e847bc19c53b69 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.417032 1166 consensus_queue.cc:260] T b85af79c2574443998e847bc19c53b69 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.418118 1160 tablet_bootstrap.cc:654] T 22273287aa9245a2a66131fed2c8861c P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.419299 1166 ts_tablet_manager.cc:1434] T b85af79c2574443998e847bc19c53b69 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251211 20:52:43.420993 1166 tablet_bootstrap.cc:492] T 46cb1d16a0e94b3a833f32c635213fd2 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:43.423599 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0763a475527546bb83cee8fb823893ac" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:43.424156 1166 tablet_bootstrap.cc:654] T 46cb1d16a0e94b3a833f32c635213fd2 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:43.425226 982 leader_election.cc:343] T 0763a475527546bb83cee8fb823893ac P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.425385 982 leader_election.cc:304] T 0763a475527546bb83cee8fb823893ac P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.425673 2507 raft_consensus.cc:2749] T 0763a475527546bb83cee8fb823893ac P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:43.426152 1168 tablet_bootstrap.cc:654] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.420504 911 raft_consensus.cc:3060] T 676faec7f3d1488f99b220f8e968e9a0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.411573 2415 tablet_loader.cc:96] loaded metadata for tablet 7e24d782602f4a639a60c797a7c92b5b (table default.Table_ca4526eee78f4811a98c7a64f76431d8 [id=eb61893aa74349e79064946f7d8334a6])
I20251211 20:52:43.429318 2415 tablet_loader.cc:96] loaded metadata for tablet 7e89288decd14ed4b59af6d17900ec6d (table default.Table_ff22c4d4d5d34ac4bf51e3e4c97b2472 [id=925885faf3a6449faef52cc2033bfbf7])
I20251211 20:52:43.430225 2415 tablet_loader.cc:96] loaded metadata for tablet 7ffa06c4a7ee48b0b337a27aff090592 (table default.Table_647731023b044ebb8954f7fdcc3faf97 [id=4b91abd6e9e94a8a941b33e62e23c083])
I20251211 20:52:43.432263 2415 tablet_loader.cc:96] loaded metadata for tablet 81fe26e81790448fb436929c03e3c284 (table default.Table_24b29712550340f2bea51b04f527c4f3 [id=d8a0839df7524fb7b95bfc05fd4ada25])
I20251211 20:52:43.432695 2415 tablet_loader.cc:96] loaded metadata for tablet 8229b704e7464b30b7d2902a938ce33b (table default.Table_b008891c4e894b02aa0db32d1740371a [id=8fe47ac33eb2461784b859e64fbd3e24])
I20251211 20:52:43.433759 2415 tablet_loader.cc:96] loaded metadata for tablet 834b84a557b04e78b5d2c8c84520a644 (table default.Table_909bbe6da1a648d68199fb27424e4342 [id=cc99459a5039424dbc192ff748b6708b])
I20251211 20:52:43.434149 2415 tablet_loader.cc:96] loaded metadata for tablet 845adb6038fc4c4dbc94418d8ebe3fec (table default.Table_4e2726de74fd437796c8e0e6dd8b3af0 [id=6bca2b1cbc244d34b21efa75f651d35f])
I20251211 20:52:43.434424 909 raft_consensus.cc:3060] T e4647105ab57454d883ebfcde7bde891 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.435155 2415 tablet_loader.cc:96] loaded metadata for tablet 84bb3583857848459e0e865283de623d (table default.Table_92281c5fcfee4355a79215ffedeee588 [id=6e5fe0a78e4b48f98dac028802f1ddf4])
I20251211 20:52:43.437158 1166 tablet_bootstrap.cc:492] T 46cb1d16a0e94b3a833f32c635213fd2 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:43.437482 1166 ts_tablet_manager.cc:1403] T 46cb1d16a0e94b3a833f32c635213fd2 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.002s
I20251211 20:52:43.438922 1166 raft_consensus.cc:359] T 46cb1d16a0e94b3a833f32c635213fd2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.439282 1166 raft_consensus.cc:385] T 46cb1d16a0e94b3a833f32c635213fd2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.439441 1166 raft_consensus.cc:740] T 46cb1d16a0e94b3a833f32c635213fd2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.439924 1166 consensus_queue.cc:260] T 46cb1d16a0e94b3a833f32c635213fd2 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.441457 1160 tablet_bootstrap.cc:492] T 22273287aa9245a2a66131fed2c8861c P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:43.441643 1160 ts_tablet_manager.cc:1403] T 22273287aa9245a2a66131fed2c8861c P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.000s
I20251211 20:52:43.442409 1160 raft_consensus.cc:359] T 22273287aa9245a2a66131fed2c8861c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.442524 2325 raft_consensus.cc:493] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.442603 1160 raft_consensus.cc:385] T 22273287aa9245a2a66131fed2c8861c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.442652 1160 raft_consensus.cc:740] T 22273287aa9245a2a66131fed2c8861c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.442723 1166 ts_tablet_manager.cc:1434] T 46cb1d16a0e94b3a833f32c635213fd2 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20251211 20:52:43.442868 1160 consensus_queue.cc:260] T 22273287aa9245a2a66131fed2c8861c P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.442965 2325 raft_consensus.cc:515] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.443454 1160 ts_tablet_manager.cc:1434] T 22273287aa9245a2a66131fed2c8861c P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:43.443781 1160 tablet_bootstrap.cc:492] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:43.444307 1166 tablet_bootstrap.cc:492] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:43.444521 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5262c2241e524395a73c0eb368917169" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:43.445575 1160 tablet_bootstrap.cc:654] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:43.446470 711 leader_election.cc:343] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.447255 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5262c2241e524395a73c0eb368917169" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:43.447469 1047 raft_consensus.cc:2468] T 5262c2241e524395a73c0eb368917169 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:43.447643 1166 tablet_bootstrap.cc:654] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.448086 2325 leader_election.cc:290] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.448285 710 leader_election.cc:304] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.449084 1168 tablet_bootstrap.cc:492] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:43.456079 1168 ts_tablet_manager.cc:1403] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.067s user 0.009s sys 0.002s
I20251211 20:52:43.451459 2273 raft_consensus.cc:493] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.456939 1168 raft_consensus.cc:359] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.457450 2415 tablet_loader.cc:96] loaded metadata for tablet 84cca3ee8fc446fcaf9971171235cf55 (table default.Table_68f5e21962ce48b89834785bf8dd332f [id=df1df827d2194032b668c7579bc1dcda])
I20251211 20:52:43.457576 1168 raft_consensus.cc:385] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.457643 1168 raft_consensus.cc:740] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.457876 1168 consensus_queue.cc:260] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.458060 2462 raft_consensus.cc:493] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.458187 2499 raft_consensus.cc:2804] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:43.458249 2415 tablet_loader.cc:96] loaded metadata for tablet 857512c55c644c36bf6cacf046b008eb (table default.Table_91f5eb149cce4e40a34b953dac528702 [id=1bb8d23d94f44ee88ef0aa71624480db])
I20251211 20:52:43.458199 2462 raft_consensus.cc:515] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.458287 2499 raft_consensus.cc:493] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:43.458390 1168 ts_tablet_manager.cc:1434] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:43.458397 2499 raft_consensus.cc:3060] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.458709 1168 tablet_bootstrap.cc:492] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:43.458850 2462 leader_election.cc:290] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.460197 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "260bdeb73439437cbf234d20f7264ab3" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:43.460422 777 raft_consensus.cc:2468] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 0.
I20251211 20:52:43.460853 847 leader_election.cc:304] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters:
I20251211 20:52:43.461243 2462 raft_consensus.cc:2804] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:43.461318 2462 raft_consensus.cc:493] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:43.461375 2462 raft_consensus.cc:3060] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.461515 1168 tablet_bootstrap.cc:654] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.457321 2273 raft_consensus.cc:515] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.462929 2273 leader_election.cc:290] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.463380 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "04086eaa32a74edd84235d3b28f9f965" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:43.463845 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "04086eaa32a74edd84235d3b28f9f965" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:43.464046 1048 raft_consensus.cc:2468] T 04086eaa32a74edd84235d3b28f9f965 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
W20251211 20:52:43.464040 711 leader_election.cc:343] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:43.465076 847 leader_election.cc:343] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 260bdeb73439437cbf234d20f7264ab3
I20251211 20:52:43.465502 710 leader_election.cc:304] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.465909 2273 raft_consensus.cc:2804] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:43.466049 2273 raft_consensus.cc:493] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:43.466136 2273 raft_consensus.cc:3060] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.475651 2415 tablet_loader.cc:96] loaded metadata for tablet 861d2d70a24946699dccd65cab05195e (table default.Table_819c4206e91e41358ac71e82185b4140 [id=28be836b71ed4107af7adc3fcb948b5e])
I20251211 20:52:43.476042 2325 raft_consensus.cc:493] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.476249 2325 raft_consensus.cc:515] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.476975 2325 leader_election.cc:290] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.477698 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f3855ffb51a74101b26fd28429e65bc2" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:43.478310 711 leader_election.cc:343] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.479305 2415 tablet_loader.cc:96] loaded metadata for tablet 8675e2a18f0f4a0e95a87005cf82d8b4 (table default.Table_bd9633c55bd242c18aac3a7f2c18a329 [id=767f50bee5464316a18f73e64bab817a])
I20251211 20:52:43.479848 2415 tablet_loader.cc:96] loaded metadata for tablet 86c5228210954839ae7bf5bf0b5aa027 (table default.Table_79cef56b856548afb009a80ec288ad1b [id=a4bb425aab7d4a9183630f3db61a6310])
I20251211 20:52:43.480337 2415 tablet_loader.cc:96] loaded metadata for tablet 871406797f6b4961be75eec0fc21e61d (table default.Table_f89cac674abe45038710fc4b675afbc8 [id=687d8fab1ee24655b68058860cdd4355])
I20251211 20:52:43.481207 2415 tablet_loader.cc:96] loaded metadata for tablet 87c4ec07e82d4bf79b4eeefe732674ff (table default.Table_a06aeab492cf4bd59dbc1274c6c6827d [id=a58ccb217e734813a81f290602e09bef])
I20251211 20:52:43.481173 2507 raft_consensus.cc:493] T 0aac6fb3972e4aba8950828c9e49d507 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.481309 2507 raft_consensus.cc:515] T 0aac6fb3972e4aba8950828c9e49d507 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.481961 2507 leader_election.cc:290] T 0aac6fb3972e4aba8950828c9e49d507 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:43.482187 2415 tablet_loader.cc:96] loaded metadata for tablet 88e4282d370e4df3a7dcab7819dbac45 (table default.Table_95bf875841aa45c8bf54c51b5839eb51 [id=71660a3234e74d639bca2b039ff420c3])
I20251211 20:52:43.483038 2415 tablet_loader.cc:96] loaded metadata for tablet 8b793241ee3b42b1a38e9f2f05e1173b (table default.Table_7506ffecc2b64648a9ba736e5298ddf4 [id=279f39d718ba45318f52f95e8aa13652])
I20251211 20:52:43.483975 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0aac6fb3972e4aba8950828c9e49d507" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:43.484423 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0aac6fb3972e4aba8950828c9e49d507" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:43.484995 982 leader_election.cc:343] T 0aac6fb3972e4aba8950828c9e49d507 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:43.485252 983 leader_election.cc:343] T 0aac6fb3972e4aba8950828c9e49d507 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.485340 983 leader_election.cc:304] T 0aac6fb3972e4aba8950828c9e49d507 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.485705 2507 raft_consensus.cc:2749] T 0aac6fb3972e4aba8950828c9e49d507 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:43.485965 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f3855ffb51a74101b26fd28429e65bc2" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:43.486124 1047 raft_consensus.cc:2468] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:43.486601 710 leader_election.cc:304] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.486852 2338 raft_consensus.cc:515] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.487469 2338 leader_election.cc:290] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.487624 2494 raft_consensus.cc:2804] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:43.487738 2494 raft_consensus.cc:493] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:43.487831 2494 raft_consensus.cc:3060] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.487872 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "41c30bded6504bbc9f0af8f34dc81e8d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:43.488368 711 leader_election.cc:343] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.491310 2415 tablet_loader.cc:96] loaded metadata for tablet 8dc4e730951144d78b77280f6cce6d0e (table default.Table_890c9027415a44ccaf95c0adede51992 [id=7bc8a2b91fe54b91b6594b371fea7bd3])
I20251211 20:52:43.497009 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "41c30bded6504bbc9f0af8f34dc81e8d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:43.497264 1047 raft_consensus.cc:3060] T 41c30bded6504bbc9f0af8f34dc81e8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.497934 1042 raft_consensus.cc:2468] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:43.501405 710 leader_election.cc:304] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.502202 2325 raft_consensus.cc:2804] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:43.502413 2325 raft_consensus.cc:697] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:43.502765 2325 consensus_queue.cc:237] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.506362 2415 tablet_loader.cc:96] loaded metadata for tablet 8e09b172980c4e40b1d3f695d8e54c37 (table default.Table_574610b44ec44cbfa5a7071d1956f2f6 [id=1a595087add94970af8aca08c75bd834])
I20251211 20:52:43.506676 2460 raft_consensus.cc:493] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.506820 2460 raft_consensus.cc:515] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.507426 2460 leader_election.cc:290] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.508424 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7d4bba0aa7ea403e8dfcb74cec2e1f9b" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:43.508636 777 raft_consensus.cc:2468] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 0.
I20251211 20:52:43.509265 2415 tablet_loader.cc:96] loaded metadata for tablet 8ed276a0fd3b4d7a8d26b8541e35fc60 (table default.Table_b7695e75618444e5a8e1ae1e2fd02d5b [id=5f737bc407a14f83a660f02af3be8e39])
I20251211 20:52:43.512122 847 leader_election.cc:304] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters:
I20251211 20:52:43.513427 2460 raft_consensus.cc:2804] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:43.513535 2460 raft_consensus.cc:493] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:43.513605 2460 raft_consensus.cc:3060] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.513813 2415 tablet_loader.cc:96] loaded metadata for tablet 8feee01fc0aa4b32be926ff2b617b894 (table default.Table_e6f3a819faad48aabfad3a235c3c1b95 [id=18f8d722faaa4293a112525ff0c96216])
W20251211 20:52:43.515091 847 leader_election.cc:343] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 7d4bba0aa7ea403e8dfcb74cec2e1f9b
I20251211 20:52:43.516242 2415 tablet_loader.cc:96] loaded metadata for tablet 903c7f4b312d4677bb35be1aaa46ad6b (table default.Table_6d9aa092c758405fa974ff1675c60a8a [id=78f84aeb9f40475f91cdf41fd040b400])
I20251211 20:52:43.526041 2338 raft_consensus.cc:493] T 7ec261837cf04bf88a48ff8add1b0d66 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.526310 2338 raft_consensus.cc:515] T 7ec261837cf04bf88a48ff8add1b0d66 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.527037 2338 leader_election.cc:290] T 7ec261837cf04bf88a48ff8add1b0d66 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.527664 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7ec261837cf04bf88a48ff8add1b0d66" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:43.528347 711 leader_election.cc:343] T 7ec261837cf04bf88a48ff8add1b0d66 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:43.528717 710 leader_election.cc:343] T 7ec261837cf04bf88a48ff8add1b0d66 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 7ec261837cf04bf88a48ff8add1b0d66
I20251211 20:52:43.528818 710 leader_election.cc:304] T 7ec261837cf04bf88a48ff8add1b0d66 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.529074 2338 raft_consensus.cc:2749] T 7ec261837cf04bf88a48ff8add1b0d66 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:43.530457 2338 raft_consensus.cc:493] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.530673 2415 tablet_loader.cc:96] loaded metadata for tablet 91254cc6ce5946b1bf3069b2202a6a11 (table default.Table_4ecbff2ab6c243f994ef4fae46276e92 [id=b3230934455d40598c6213630dd3108b])
I20251211 20:52:43.530642 2338 raft_consensus.cc:515] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.531829 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2e7e158809b9471fbd6e6ee9fb9ac569" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:43.532388 711 leader_election.cc:343] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.532351 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2e7e158809b9471fbd6e6ee9fb9ac569" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:43.532582 1042 raft_consensus.cc:2468] T 2e7e158809b9471fbd6e6ee9fb9ac569 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:43.532994 710 leader_election.cc:304] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.533119 2415 tablet_loader.cc:96] loaded metadata for tablet 92efda2bd0c84d07a0af6b537c0cd882 (table default.Table_ec893136f41e41f6a6db70f193b8065e [id=c9f87468f622412a8fcd50d311f99117])
I20251211 20:52:43.533313 2325 raft_consensus.cc:2804] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:43.533367 2338 leader_election.cc:290] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.533388 2325 raft_consensus.cc:493] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:43.533450 2325 raft_consensus.cc:3060] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.535265 2415 tablet_loader.cc:96] loaded metadata for tablet 938a0d8a090f490b816fdf2b8f1a51b8 (table default.Table_4d6139c7976d414fbdcbdb96787d186b [id=9ffcb594267a4b0eb8d792dbe46d585f])
I20251211 20:52:43.535964 2415 tablet_loader.cc:96] loaded metadata for tablet 94658fcf512a48d0a992a7e2e0f7c176 (table default.Table_c9d86e82aebb4ef0bc83789720352e38 [id=97377a8ef56c4cdebe95d15955207404])
I20251211 20:52:43.538147 2415 tablet_loader.cc:96] loaded metadata for tablet 9527db28329245e7b8343e69d3881123 (table default.Table_b9de49d730424e2b863b64535f2ba732 [id=fd5a2aedcd31426394ad1ffdd0936893])
I20251211 20:52:43.538643 2415 tablet_loader.cc:96] loaded metadata for tablet 9563684e423d41eda6ff7f9985a7b1c1 (table default.Table_91c76a3c00864c3fa3d653b3f6e0f64f [id=d991ee6b2aa54096a3b1676618b6cb22])
I20251211 20:52:43.539266 2415 tablet_loader.cc:96] loaded metadata for tablet 95af3eeefd084777a70736c59d487322 (table default.Table_b8e180cc27234500a1c4c7bdd2b93ac8 [id=5dfca04de96e404a802046007f6a2afb])
I20251211 20:52:43.539723 2415 tablet_loader.cc:96] loaded metadata for tablet 962b80ae747b4470abd23643e711fad9 (table default.Table_464acf89dba243ae925fa3b824ceb333 [id=d69ab7378b1d4272b1c2f8df48b96e81])
I20251211 20:52:43.548875 1160 tablet_bootstrap.cc:492] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:43.549046 1160 ts_tablet_manager.cc:1403] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.105s user 0.029s sys 0.028s
I20251211 20:52:43.549458 1166 tablet_bootstrap.cc:492] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:43.549844 1166 ts_tablet_manager.cc:1403] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.106s user 0.016s sys 0.022s
I20251211 20:52:43.549832 1160 raft_consensus.cc:359] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.550038 1160 raft_consensus.cc:385] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.550097 1160 raft_consensus.cc:740] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.550302 1160 consensus_queue.cc:260] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.551493 1166 raft_consensus.cc:359] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.552093 1166 raft_consensus.cc:385] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.552256 1166 raft_consensus.cc:740] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.552511 2415 tablet_loader.cc:96] loaded metadata for tablet 969a8587019c4202b6e705d126f83d44 (table default.Table_6460313002684bba93b30ad8631f0f49 [id=22347b15537d451d9927a17e8e5a202a])
I20251211 20:52:43.552750 1166 consensus_queue.cc:260] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.554402 2415 tablet_loader.cc:96] loaded metadata for tablet 9838930aafd64da2aafaaf416ae95344 (table default.Table_7684c7e006504ea0a1ef81dc2eecaa8d [id=f943bf305b574c81b836e91e267d5d81])
I20251211 20:52:43.556015 2415 tablet_loader.cc:96] loaded metadata for tablet 9898716a5fe04953b92a66dcbd10b2dc (table default.Table_cbecf49ce5e044049547838a5d6ce009 [id=dd75e76866b84667ae39862a9330a45d])
I20251211 20:52:43.560648 1168 tablet_bootstrap.cc:492] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:43.560894 1168 ts_tablet_manager.cc:1403] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.102s user 0.019s sys 0.007s
I20251211 20:52:43.561647 1168 raft_consensus.cc:359] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.561883 1168 raft_consensus.cc:385] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.561964 1168 raft_consensus.cc:740] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.562198 1168 consensus_queue.cc:260] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.563431 2415 tablet_loader.cc:96] loaded metadata for tablet 9b60966ae2ee4928a5e1b0e9c912ff77 (table default.Table_862fff1ce3564d939f033f3f2fe049ac [id=d35a5d6eb11049239d1898d175b356b8])
I20251211 20:52:43.565919 2507 raft_consensus.cc:493] T ee7b93b0b87541d9af6eb9e68b4cea41 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.566089 2507 raft_consensus.cc:515] T ee7b93b0b87541d9af6eb9e68b4cea41 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.566300 1168 ts_tablet_manager.cc:1434] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251211 20:52:43.566756 2507 leader_election.cc:290] T ee7b93b0b87541d9af6eb9e68b4cea41 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:43.568091 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee7b93b0b87541d9af6eb9e68b4cea41" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:43.568944 2563 raft_consensus.cc:493] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.570132 1166 ts_tablet_manager.cc:1434] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.020s user 0.001s sys 0.001s
W20251211 20:52:43.571615 982 leader_election.cc:343] T ee7b93b0b87541d9af6eb9e68b4cea41 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.572352 1168 tablet_bootstrap.cc:492] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:43.574268 1168 tablet_bootstrap.cc:654] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:43.571233 847 consensus_peers.cc:597] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cce44b80c97948e1af7ac68e77cccfd6. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:43.576399 2415 tablet_loader.cc:96] loaded metadata for tablet 9b99860ee6be45a0b358e686332d3137 (table default.Table_a5b944a25f4c45cf8c70765ea43d1ac2 [id=0ee0170e570a4d9aa25ea2858e9d9e2a])
I20251211 20:52:43.576880 2415 tablet_loader.cc:96] loaded metadata for tablet 9c5c937cc7db4ed18fe03f4a8a99a66a (table default.Table_68b9d0d7258a4b5caa1779a4769bdd7f [id=06f467911b6f45849e99f0b6fe615163])
I20251211 20:52:43.577255 2415 tablet_loader.cc:96] loaded metadata for tablet 9d00e07cfc23457eb09e570dfecdcb9e (table default.Table_eaad557efa374b0bbd5653d90a967f8b [id=b187d831069447e39a00bc5e91f1fcf8])
I20251211 20:52:43.577680 2415 tablet_loader.cc:96] loaded metadata for tablet 9db471247b674e34acf4239a8b55e1d6 (table default.Table_7e3bb905c8ec4e96809e6b7d4e44a3b7 [id=8212122882be451f9b5f5c6880bf3dce])
I20251211 20:52:43.578125 2415 tablet_loader.cc:96] loaded metadata for tablet 9f71d67cde1d4ac98ee195c3b81b6e75 (table default.Table_fd043ec094f94011986fc8f5a9f32f1a [id=994990d2a18f405c988452e93cca7bef])
I20251211 20:52:43.578680 2415 tablet_loader.cc:96] loaded metadata for tablet 9f8aa28c1c9b4fc58dfc128952c180ba (table default.Table_bf7d402a36be44cfb6b79041906d26a1 [id=0327e686e7ed4475a3e8cda8721a3a3a])
I20251211 20:52:43.579352 2415 tablet_loader.cc:96] loaded metadata for tablet a0618ed6fb164b9b9bcde43149b60c2d (table default.Table_9fb97b3359de4806bf0557739a5da4fd [id=17c25e0db2f9436688a7338067e09aac])
I20251211 20:52:43.575397 2563 raft_consensus.cc:515] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.579970 2415 tablet_loader.cc:96] loaded metadata for tablet a249906ca3204563a485aa4f5abe91f2 (table default.Table_8e6bffee74fd4ea4a95ec5e51b0154c1 [id=c345c786f9a949dea34cc93bb72ae4cf])
I20251211 20:52:43.580528 2415 tablet_loader.cc:96] loaded metadata for tablet a37c273c9a5742eaae616e5c5f87f9d7 (table default.Table_b0ac4ccf9e224d32af03dcaa1b97330a [id=10bf2a53ba594b5e8b8a37d2ba0235b9])
I20251211 20:52:43.580917 2415 tablet_loader.cc:96] loaded metadata for tablet a3a6e2d4b060449f957eb8d6cacee843 (table default.Table_d812d6f6c90842e4821d33226a54ec68 [id=276b9b5c545849dcb1ed38324b7ad540])
I20251211 20:52:43.581286 2415 tablet_loader.cc:96] loaded metadata for tablet a3b1aadf17ea4bf0a896d23cd543fca7 (table default.Table_d18401707c88493abdf90d2c713432ae [id=a07662f8dc3f4178be5dfcc1cdc85837])
I20251211 20:52:43.581624 2415 tablet_loader.cc:96] loaded metadata for tablet a3b263c205eb479b8210ac79c766523f (table default.Table_948151057bf94b75a251b99e3906f4a4 [id=1ab6db436e504bfabeb44e435a5453b8])
W20251211 20:52:43.582428 847 leader_election.cc:343] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: b529b38d754f46ba8cb60b19e9705256
I20251211 20:52:43.582772 1166 tablet_bootstrap.cc:492] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:43.582744 2429 raft_consensus.cc:515] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.583366 2429 leader_election.cc:290] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.583748 1047 raft_consensus.cc:2468] T 41c30bded6504bbc9f0af8f34dc81e8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:43.584153 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee7b93b0b87541d9af6eb9e68b4cea41" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:43.586544 1166 tablet_bootstrap.cc:654] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:43.587004 983 leader_election.cc:343] T ee7b93b0b87541d9af6eb9e68b4cea41 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.587149 983 leader_election.cc:304] T ee7b93b0b87541d9af6eb9e68b4cea41 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.587857 2507 raft_consensus.cc:2749] T ee7b93b0b87541d9af6eb9e68b4cea41 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:43.590689 710 leader_election.cc:304] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.592037 2338 raft_consensus.cc:2804] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:43.592187 2338 raft_consensus.cc:697] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:43.591980 2563 leader_election.cc:290] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.592445 2338 consensus_queue.cc:237] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.593070 2415 tablet_loader.cc:96] loaded metadata for tablet a3e05f60fd5540d38230ffab3bf49307 (table default.Table_5a933aeef8dc44b18bb0cf7ddfe412f2 [id=3496dd8ce5184ac79c516278d403cbcb])
I20251211 20:52:43.594293 2415 tablet_loader.cc:96] loaded metadata for tablet a5b2f4079315477bb5862629b7896f80 (table default.Table_e887667aa38d47e9814a79f7b0e6990d [id=9d71d585608f49f48128aa54ecd0b724])
I20251211 20:52:43.595132 1160 ts_tablet_manager.cc:1434] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.046s user 0.007s sys 0.012s
I20251211 20:52:43.595407 2415 tablet_loader.cc:96] loaded metadata for tablet a7afcce522494c9ca1d80e6eea913e6e (table default.Table_cb6a4bac7b694f33a37de2ee78c91ca0 [id=8940f18527c249a9baefc170ed82f45a])
I20251211 20:52:43.595438 1160 tablet_bootstrap.cc:492] T a3b1aadf17ea4bf0a896d23cd543fca7 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:43.595798 2415 tablet_loader.cc:96] loaded metadata for tablet a87c69ae71d24de09294c7c8c34d79da (table default.Table_ee696fa128ac464da5a6f99bd3d7ed7a [id=8cb270b7f6e04c8c9307869d9dd6df4e])
I20251211 20:52:43.597213 1160 tablet_bootstrap.cc:654] T a3b1aadf17ea4bf0a896d23cd543fca7 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.598430 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "50f9f29c064c4f938a0f20f302ab5e8a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:43.598673 1042 raft_consensus.cc:3060] T 50f9f29c064c4f938a0f20f302ab5e8a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:43.603761 847 leader_election.cc:343] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: b529b38d754f46ba8cb60b19e9705256
I20251211 20:52:43.603928 847 leader_election.cc:304] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:43.604225 2564 raft_consensus.cc:2749] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:43.605255 2415 tablet_loader.cc:96] loaded metadata for tablet a87fe058d1de4781b24f02424eaeeccd (table default.Table_d87c2c33f4ff4aee9ff82b0653d4cc43 [id=2876f1fc09634b03afe5664e82c017d6])
I20251211 20:52:43.605836 2415 tablet_loader.cc:96] loaded metadata for tablet a98a2b091d9b44a7b954ddb45c819da9 (table default.Table_2fddb279b6164c2ba9e81fc55dfeb59a [id=f8c4f76e6a2d48fb93603b9d308a71ec])
I20251211 20:52:43.606702 2415 tablet_loader.cc:96] loaded metadata for tablet a9bb22dce13f4c9da76ca446715cd99a (table default.Table_528b4823aba44a72a7af20911a8d2002 [id=940343c556c54c4fb2110f4f47ca91f3])
I20251211 20:52:43.606892 1160 tablet_bootstrap.cc:492] T a3b1aadf17ea4bf0a896d23cd543fca7 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:43.607082 1160 ts_tablet_manager.cc:1403] T a3b1aadf17ea4bf0a896d23cd543fca7 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.001s
I20251211 20:52:43.607502 1166 tablet_bootstrap.cc:492] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:43.607740 1166 ts_tablet_manager.cc:1403] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.025s user 0.000s sys 0.006s
I20251211 20:52:43.607924 1160 raft_consensus.cc:359] T a3b1aadf17ea4bf0a896d23cd543fca7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.608134 1160 raft_consensus.cc:385] T a3b1aadf17ea4bf0a896d23cd543fca7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.608230 1160 raft_consensus.cc:740] T a3b1aadf17ea4bf0a896d23cd543fca7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.608448 1160 consensus_queue.cc:260] T a3b1aadf17ea4bf0a896d23cd543fca7 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.608575 1166 raft_consensus.cc:359] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.608826 1166 raft_consensus.cc:385] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.608944 1166 raft_consensus.cc:740] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.608976 1160 ts_tablet_manager.cc:1434] T a3b1aadf17ea4bf0a896d23cd543fca7 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:43.609449 1166 consensus_queue.cc:260] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.610767 1160 tablet_bootstrap.cc:492] T 28ddb8a59a214576a25d613d42c2fef5 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:43.612617 1160 tablet_bootstrap.cc:654] T 28ddb8a59a214576a25d613d42c2fef5 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.617717 1166 ts_tablet_manager.cc:1434] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.010s user 0.000s sys 0.002s
I20251211 20:52:43.623960 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "50f9f29c064c4f938a0f20f302ab5e8a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:43.625135 1166 tablet_bootstrap.cc:492] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:43.627300 1166 tablet_bootstrap.cc:654] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:43.627791 711 leader_election.cc:343] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:43.628086 711 consensus_peers.cc:597] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:43.628587 2415 tablet_loader.cc:96] loaded metadata for tablet aa50ddb2789b4be3bfa9aedef9287109 (table default.Table_44f971096d1e41f480e594985ffd5c17 [id=d4e1349374a346a5b8be180f9bdc329a])
I20251211 20:52:43.629417 2415 tablet_loader.cc:96] loaded metadata for tablet aab26df748724fe5b2bb7fad9e3bfc0c (table default.Table_37849576518b4320a3bac34baa096242 [id=df6255d66f334ef2b4406f4b845cf980])
I20251211 20:52:43.629868 2415 tablet_loader.cc:96] loaded metadata for tablet ab4e7a9062134e489d6f9abe6ff26fb4 (table default.Table_4cbc8841b276406cb4434629653a5ce3 [id=b958532650374d609199cab9c07bb64b])
I20251211 20:52:43.630337 2415 tablet_loader.cc:96] loaded metadata for tablet ac8fb4234af54acc8b250a081e8aeecc (table default.Table_4e2d2e9566764ee9bc5edf65d0e693a2 [id=d3c56e51e2464bafb3819af545f21a3f])
I20251211 20:52:43.631274 2415 tablet_loader.cc:96] loaded metadata for tablet acc952c98f414a05aa863372de1ab397 (table default.Table_ee37a9ef24e548ebb2b6b55b54fa6f91 [id=162bfc57131c46c9bbf360299e331097])
I20251211 20:52:43.632217 2415 tablet_loader.cc:96] loaded metadata for tablet ae1abef61473443b9922eb13f2ae92fa (table default.Table_40d581fe03ef48d1b5639a95ffc00b7b [id=66810eaf071c4613a4d30dfa016788a8])
I20251211 20:52:43.636257 2415 tablet_loader.cc:96] loaded metadata for tablet ae45b0b63c554fd79ff243f14a3820c4 (table default.Table_d8d764377e524faf931940bfe23100be [id=198db6e18dcf4f65bde2405f4f0bad0e])
I20251211 20:52:43.637957 1166 tablet_bootstrap.cc:492] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:43.638216 1166 ts_tablet_manager.cc:1403] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.001s
I20251211 20:52:43.639110 1166 raft_consensus.cc:359] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.639374 1166 raft_consensus.cc:385] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.639540 1166 raft_consensus.cc:740] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.639997 1166 consensus_queue.cc:260] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.647483 1166 ts_tablet_manager.cc:1434] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20251211 20:52:43.647771 2415 tablet_loader.cc:96] loaded metadata for tablet affe191cba57401f9c58211f2d06b80d (table default.Table_7730b7f512954bcab4c83c9f272d9a59 [id=2783ef72c8944e8188a8646565a6df13])
I20251211 20:52:43.649912 2415 tablet_loader.cc:96] loaded metadata for tablet b03a2f87c2c4415b98f461a559d4e960 (table default.Table_51015ac894894382b4c0d1931b24f4d4 [id=2118ffd1623f491e95ecf3719fa44130])
I20251211 20:52:43.652383 2415 tablet_loader.cc:96] loaded metadata for tablet b072b759d1254859b0b8d401e3078792 (table default.Table_3de7aa51e2814433b4d98f3226b96c97 [id=a137f0ddf52145a99c62444602ab3fdb])
I20251211 20:52:43.654886 1166 tablet_bootstrap.cc:492] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:43.655923 2415 tablet_loader.cc:96] loaded metadata for tablet b115d737d7fc483d9f5a1f040b511bae (table default.Table_6d94053661f142b79292e19752df02df [id=c5be59a2c3934b3789641ceeb2d4d6e2])
I20251211 20:52:43.658461 2564 consensus_queue.cc:1048] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:43.658901 1166 tablet_bootstrap.cc:654] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.660422 1160 tablet_bootstrap.cc:492] T 28ddb8a59a214576a25d613d42c2fef5 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:43.660573 1160 ts_tablet_manager.cc:1403] T 28ddb8a59a214576a25d613d42c2fef5 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.050s user 0.009s sys 0.011s
I20251211 20:52:43.661338 1160 raft_consensus.cc:359] T 28ddb8a59a214576a25d613d42c2fef5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.661497 1160 raft_consensus.cc:385] T 28ddb8a59a214576a25d613d42c2fef5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.661545 1160 raft_consensus.cc:740] T 28ddb8a59a214576a25d613d42c2fef5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.661731 1160 consensus_queue.cc:260] T 28ddb8a59a214576a25d613d42c2fef5 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.661372 2415 tablet_loader.cc:96] loaded metadata for tablet b2974f16034c44cf85ff85b5095cf6a0 (table default.Table_031de0f4cc7744bc9e320261f8b46696 [id=be36dabaa6d44325b4855b9f13bda847])
I20251211 20:52:43.662227 1160 ts_tablet_manager.cc:1434] T 28ddb8a59a214576a25d613d42c2fef5 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:43.662477 1160 tablet_bootstrap.cc:492] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:43.664714 1160 tablet_bootstrap.cc:654] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.668077 2415 tablet_loader.cc:96] loaded metadata for tablet b3ccde2b054841769a4c85f19f80f20a (table default.Table_7e3cf73eaea24e34b5dd291f8a738947 [id=e3ff2e8147aa41a99206b3961c29d751])
I20251211 20:52:43.670717 2415 tablet_loader.cc:96] loaded metadata for tablet b52377c8ffd741a598e5328a00d876fe (table default.Table_686a26915b42421380037144900dd97c [id=8a1b0c9d641b4a3c83d1ca48576d110d])
I20251211 20:52:43.674492 2415 tablet_loader.cc:96] loaded metadata for tablet b5b68f08566e4c01b004ecdab970fffa (table default.Table_4b444339f63048179e9b53a3aec9173c [id=1f67fc9412d2431ab260b87363248dd7])
I20251211 20:52:43.676668 1166 tablet_bootstrap.cc:492] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:43.676872 1166 ts_tablet_manager.cc:1403] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.000s
I20251211 20:52:43.679368 1166 raft_consensus.cc:359] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.679652 1166 raft_consensus.cc:385] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.679737 1166 raft_consensus.cc:740] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.679999 1166 consensus_queue.cc:260] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.680226 2415 tablet_loader.cc:96] loaded metadata for tablet b65b142f801e4111ad153164dfe9b211 (table default.Table_0910ec0ec3e64c139587947350a39ae5 [id=4ce959831e384a14854ce5570dededaa])
I20251211 20:52:43.683072 2460 raft_consensus.cc:515] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.683648 2415 tablet_loader.cc:96] loaded metadata for tablet b6cc819332e1445c96ce3ffac4861454 (table default.Table_db4c0f706e0643b39d39ffa84352826d [id=4d459c42bf584068ac3851227d80331a])
I20251211 20:52:43.684608 2501 raft_consensus.cc:515] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.685253 2501 leader_election.cc:290] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.686312 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6a61cf8f5de14adaa4a1de5bddde9a2f" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:43.686527 1047 raft_consensus.cc:3060] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.687033 2460 leader_election.cc:290] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Requested vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.687815 2415 tablet_loader.cc:96] loaded metadata for tablet b81a4cef27164824b52695039744db15 (table default.Table_defdeab28749411590cf75b5251f604b [id=11352566d1744193a40a8bb605aa3912])
I20251211 20:52:43.688796 1042 raft_consensus.cc:2468] T 50f9f29c064c4f938a0f20f302ab5e8a P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
W20251211 20:52:43.689338 847 leader_election.cc:343] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 7d4bba0aa7ea403e8dfcb74cec2e1f9b
I20251211 20:52:43.689611 710 leader_election.cc:304] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.690251 2415 tablet_loader.cc:96] loaded metadata for tablet b85af79c2574443998e847bc19c53b69 (table default.Table_fa3f2f115cd043a9aa6be2b070d320cb [id=15fad5e1cf8d4ea08dae8275a29a11dd])
I20251211 20:52:43.690443 2429 raft_consensus.cc:2804] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:43.690564 2429 raft_consensus.cc:697] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:43.690795 2429 consensus_queue.cc:237] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.691118 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7d4bba0aa7ea403e8dfcb74cec2e1f9b" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:43.691278 777 raft_consensus.cc:3060] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.691890 1166 ts_tablet_manager.cc:1434] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.015s user 0.003s sys 0.000s
I20251211 20:52:43.695021 2415 tablet_loader.cc:96] loaded metadata for tablet ba0e3a6b2699493ba9b9627b7d944b74 (table default.Table_cce138cd7e6f4a91aeb7632af1b61ade [id=7f521f77d3234e5a9b9612598e00fc94])
I20251211 20:52:43.699080 2415 tablet_loader.cc:96] loaded metadata for tablet bb0eda1a984840679c5aa87a58523552 (table default.Table_46118198e87e4c158f00055e6867664b [id=6583487fed754a28a5bb4f14cdbef5a1])
I20251211 20:52:43.700245 1166 tablet_bootstrap.cc:492] T d391d4e6a101462d8d1220f1e17d0daa P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:43.700232 711 consensus_peers.cc:597] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:43.701972 1166 tablet_bootstrap.cc:654] T d391d4e6a101462d8d1220f1e17d0daa P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.704679 2415 tablet_loader.cc:96] loaded metadata for tablet bb358f9722234ed7959560c6dfb13dc6 (table default.Table_06dc3002fa354d78ab4d5a90fd0fb3ab [id=13db407cd0aa40889e554244cbcc3616])
I20251211 20:52:43.705003 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6a61cf8f5de14adaa4a1de5bddde9a2f" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:43.705649 711 leader_election.cc:343] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.709530 1168 tablet_bootstrap.cc:492] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:43.709681 1168 ts_tablet_manager.cc:1403] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.137s user 0.002s sys 0.002s
I20251211 20:52:43.710506 1168 raft_consensus.cc:359] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.710698 1168 raft_consensus.cc:385] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.710378 2415 tablet_loader.cc:96] loaded metadata for tablet bb9fccb6981e446a939a28d7fdd38e25 (table default.Table_760e3e4714ea4128bd1de3da5a7ccccc [id=fe9b692163f943559ba64021faa648f1])
I20251211 20:52:43.710764 1168 raft_consensus.cc:740] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.711796 1166 tablet_bootstrap.cc:492] T d391d4e6a101462d8d1220f1e17d0daa P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:43.711972 1166 ts_tablet_manager.cc:1403] T d391d4e6a101462d8d1220f1e17d0daa P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20251211 20:52:43.712884 2507 raft_consensus.cc:493] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.713081 2507 raft_consensus.cc:515] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.713686 2507 leader_election.cc:290] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:43.712878 1166 raft_consensus.cc:359] T d391d4e6a101462d8d1220f1e17d0daa P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.713973 1166 raft_consensus.cc:385] T d391d4e6a101462d8d1220f1e17d0daa P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.714058 1166 raft_consensus.cc:740] T d391d4e6a101462d8d1220f1e17d0daa P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.714305 1166 consensus_queue.cc:260] T d391d4e6a101462d8d1220f1e17d0daa P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.714442 2415 tablet_loader.cc:96] loaded metadata for tablet bbe6e6220eb14e59a821805c438230c9 (table default.Table_03298b88200d42148accc9f0dffbf739 [id=b7429ea93123473ab5d795d5f3d5a5f8])
I20251211 20:52:43.716282 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3ecf0ea8ce19486c81d1d1a91d57a7ce" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:43.717170 982 leader_election.cc:343] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.718077 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3ecf0ea8ce19486c81d1d1a91d57a7ce" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:43.718609 983 leader_election.cc:343] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.718745 983 leader_election.cc:304] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.719188 2507 raft_consensus.cc:2749] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:43.720992 2415 tablet_loader.cc:96] loaded metadata for tablet bc4e7362e6d045da81bf39f4135f500e (table default.Table_328b7dd76e9e407c83b47a0f51cfcce5 [id=9e7c08f833e14a90b17a9b9f60b702de])
I20251211 20:52:43.722023 1166 ts_tablet_manager.cc:1434] T d391d4e6a101462d8d1220f1e17d0daa P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20251211 20:52:43.727916 2415 tablet_loader.cc:96] loaded metadata for tablet bc6a1c84501d4e179cd2828b50fddc23 (table default.Table_339c5a45d0294fe7adf34fdbfa8e3d98 [id=6ef058fada9941629e0ea4d1906abdac])
I20251211 20:52:43.728469 1166 tablet_bootstrap.cc:492] T 47779034ad89466e9e1801987128b4dd P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:43.729876 2415 tablet_loader.cc:96] loaded metadata for tablet bfb52935296443c58190d4d06e2e6e9c (table default.Table_3721b5725e8c4297a392d3986a081128 [id=383392d835da4c50848068d83261cecc])
I20251211 20:52:43.729884 2507 raft_consensus.cc:493] T e484d499af1b4947adb36c32f8ea5704 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.730042 2507 raft_consensus.cc:515] T e484d499af1b4947adb36c32f8ea5704 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.730652 2507 leader_election.cc:290] T e484d499af1b4947adb36c32f8ea5704 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:43.731468 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e484d499af1b4947adb36c32f8ea5704" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:43.732069 982 leader_election.cc:343] T e484d499af1b4947adb36c32f8ea5704 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.732198 1160 tablet_bootstrap.cc:492] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:43.732373 1160 ts_tablet_manager.cc:1403] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.070s user 0.016s sys 0.004s
I20251211 20:52:43.732980 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e484d499af1b4947adb36c32f8ea5704" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:43.733166 1160 raft_consensus.cc:359] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.733327 1160 raft_consensus.cc:385] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.733404 1160 raft_consensus.cc:740] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
W20251211 20:52:43.733536 983 leader_election.cc:343] T e484d499af1b4947adb36c32f8ea5704 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.733618 1160 consensus_queue.cc:260] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.733685 983 leader_election.cc:304] T e484d499af1b4947adb36c32f8ea5704 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.734053 1160 ts_tablet_manager.cc:1434] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:43.734108 2507 raft_consensus.cc:2749] T e484d499af1b4947adb36c32f8ea5704 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:43.734337 1160 tablet_bootstrap.cc:492] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:43.734997 1168 consensus_queue.cc:260] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.736704 1168 ts_tablet_manager.cc:1434] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.027s user 0.001s sys 0.001s
I20251211 20:52:43.740207 1166 tablet_bootstrap.cc:654] T 47779034ad89466e9e1801987128b4dd P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.740712 1160 tablet_bootstrap.cc:654] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.739488 1168 tablet_bootstrap.cc:492] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:43.742658 1168 tablet_bootstrap.cc:654] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.742630 2415 tablet_loader.cc:96] loaded metadata for tablet bfb694657a0a44a0a3a3325cd171a78e (table default.Table_86e88e19fbd645d3bd5a919328a0969c [id=cccf9f896f034dda92ea20dfbd569ae1])
I20251211 20:52:43.745898 2415 tablet_loader.cc:96] loaded metadata for tablet c081396b492049b996ac4661e1318388 (table default.Table_310ce452c2a54f069cf6c65f233c0603 [id=95c9ed1ec1834e2dbfd3198887b75369])
I20251211 20:52:43.750146 1166 tablet_bootstrap.cc:492] T 47779034ad89466e9e1801987128b4dd P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:43.750331 1166 ts_tablet_manager.cc:1403] T 47779034ad89466e9e1801987128b4dd P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.000s
I20251211 20:52:43.751091 1160 tablet_bootstrap.cc:492] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:43.751402 1160 ts_tablet_manager.cc:1403] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.004s
I20251211 20:52:43.751101 1166 raft_consensus.cc:359] T 47779034ad89466e9e1801987128b4dd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.751551 1166 raft_consensus.cc:385] T 47779034ad89466e9e1801987128b4dd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.751654 1166 raft_consensus.cc:740] T 47779034ad89466e9e1801987128b4dd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.751906 1166 consensus_queue.cc:260] T 47779034ad89466e9e1801987128b4dd P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.756588 1160 raft_consensus.cc:359] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.756937 1160 raft_consensus.cc:385] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.757081 1160 raft_consensus.cc:740] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.757460 2415 tablet_loader.cc:96] loaded metadata for tablet c7a2610f20df4c1ca81eae2fc7dba6bb (table default.Table_59072235d93b41c0b5cd5e3debcb8ddf [id=9a15a8a075af4969ae2c8b3be1bfed78])
I20251211 20:52:43.757463 1160 consensus_queue.cc:260] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.757997 2415 tablet_loader.cc:96] loaded metadata for tablet c7bd218b3ff340b2a589c9517b49a649 (table default.Table_233d5484bd4e4456aed925bc271edfa5 [id=ca0c3c6978874d93a04b57cc0c425c83])
I20251211 20:52:43.758710 1160 ts_tablet_manager.cc:1434] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20251211 20:52:43.759361 1160 tablet_bootstrap.cc:492] T 5262c2241e524395a73c0eb368917169 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:43.763067 1168 tablet_bootstrap.cc:492] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:43.763216 1168 ts_tablet_manager.cc:1403] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.001s
I20251211 20:52:43.763857 1160 tablet_bootstrap.cc:654] T 5262c2241e524395a73c0eb368917169 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.764004 1168 raft_consensus.cc:359] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.764165 1168 raft_consensus.cc:385] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.764215 1168 raft_consensus.cc:740] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.765136 2415 tablet_loader.cc:96] loaded metadata for tablet c8bb92fc25124ba4be887c453abc7d9a (table default.Table_f2eaa62388894072a390cc4c167f3385 [id=ac6774893a6a4f82a7c9630bbe502fe9])
I20251211 20:52:43.764793 1166 ts_tablet_manager.cc:1434] T 47779034ad89466e9e1801987128b4dd P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.014s user 0.001s sys 0.001s
I20251211 20:52:43.765506 2460 raft_consensus.cc:493] T 22273287aa9245a2a66131fed2c8861c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.765559 1168 consensus_queue.cc:260] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.765669 2460 raft_consensus.cc:515] T 22273287aa9245a2a66131fed2c8861c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.766075 1168 ts_tablet_manager.cc:1434] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:43.766359 2460 leader_election.cc:290] T 22273287aa9245a2a66131fed2c8861c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.768397 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "22273287aa9245a2a66131fed2c8861c" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:43.768523 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "22273287aa9245a2a66131fed2c8861c" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:43.769106 2415 tablet_loader.cc:96] loaded metadata for tablet c9956749c83a4a468571fdabcaf0e4aa (table default.Table_b011142d3fc34ef2a829989a9bad1114 [id=9e2abaab9bf940a8af1809342d51cd9b])
I20251211 20:52:43.769464 847 leader_election.cc:304] T 22273287aa9245a2a66131fed2c8861c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:43.769531 2415 tablet_loader.cc:96] loaded metadata for tablet ca89f09cc0954389b679e8bb8b5e4dfb (table default.Table_52e650ce0b824a41a39fcd4f1b8ba455 [id=748adb1caf074cf18655c4d7af403663])
I20251211 20:52:43.769786 2460 raft_consensus.cc:2749] T 22273287aa9245a2a66131fed2c8861c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:43.769905 2415 tablet_loader.cc:96] loaded metadata for tablet cb3937bc6d6a49928af633d10bdd7074 (table default.Table_e2163380d2474e9a93f10e69de094195 [id=041f6b72446c42f68aa2fcaa8438ad0c])
I20251211 20:52:43.774473 2415 tablet_loader.cc:96] loaded metadata for tablet ce1799e6604f4ed7875ddedecab6f327 (table default.Table_5db1ac98e36d45b1b01889ee275ad83d [id=a6849726434d41d29dce12f47066a685])
I20251211 20:52:43.777648 1166 tablet_bootstrap.cc:492] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:43.779518 1166 tablet_bootstrap.cc:654] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.766376 1168 tablet_bootstrap.cc:492] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:43.782089 1168 tablet_bootstrap.cc:654] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.782377 2462 raft_consensus.cc:515] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.783124 2462 leader_election.cc:290] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Requested vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:43.783425 2500 consensus_meta.cc:287] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919: Time spent flushing consensus metadata: real 0.718s user 0.003s sys 0.000s
I20251211 20:52:43.783536 2500 raft_consensus.cc:515] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.784130 2500 leader_election.cc:290] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.784456 1047 raft_consensus.cc:2468] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:43.784570 2415 tablet_loader.cc:96] loaded metadata for tablet ced8a953d7744b3c806f0184a1921327 (table default.Table_25ddd02b2a6640c4a6898d1796553a8b [id=14985ae63d8740f4b692dea1578be3ce])
I20251211 20:52:43.784914 710 leader_election.cc:304] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.785017 2415 tablet_loader.cc:96] loaded metadata for tablet cf1201bba4b6421f8d7ce593f4bd892b (table default.Table_b44b03dcd95a41c19061d5e52f5687c9 [id=ef458da415914c138373fbd5e0b8dec4])
I20251211 20:52:43.785845 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 919) took 3946 ms. Trace:
I20251211 20:52:43.785938 1165 rpcz_store.cc:276] 1211 20:52:39.839446 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:39.839501 (+ 55us) service_pool.cc:225] Handling call
1211 20:52:43.189063 (+3349562us) ts_tablet_manager.cc:1151] Deleting tablet e6ba12f94f5f4d688f4222a4c2fb5436
1211 20:52:43.189072 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:43.290007 (+100935us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:43.386075 (+ 96068us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:43.785824 (+399749us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3349303}
I20251211 20:52:43.786351 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 926) took 3707 ms. Trace:
I20251211 20:52:43.786414 1165 rpcz_store.cc:276] 1211 20:52:40.078845 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:40.078902 (+ 57us) service_pool.cc:225] Handling call
1211 20:52:43.786232 (+3707330us) ts_tablet_manager.cc:1151] Deleting tablet 4810f8d8b53a45eb939bf3c8bf1b4e77
1211 20:52:43.786242 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:43.786335 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3643275}
I20251211 20:52:43.786700 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 927) took 3694 ms. Trace:
I20251211 20:52:43.786754 1165 rpcz_store.cc:276] 1211 20:52:40.091830 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:40.091904 (+ 74us) service_pool.cc:225] Handling call
1211 20:52:43.786611 (+3694707us) ts_tablet_manager.cc:1151] Deleting tablet 0add14ccd2af42aeb02fad7a28ed4520
1211 20:52:43.786619 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:43.786688 (+ 69us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3643606}
I20251211 20:52:43.787002 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 934) took 3408 ms. Trace:
I20251211 20:52:43.787065 1165 rpcz_store.cc:276] 1211 20:52:40.378516 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:40.435416 (+ 56900us) service_pool.cc:225] Handling call
1211 20:52:43.786922 (+3351506us) ts_tablet_manager.cc:1151] Deleting tablet 877aa237fe43452cbdbb4dac46f613ae
1211 20:52:43.786929 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:43.786990 (+ 61us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3351218}
I20251211 20:52:43.787308 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 942) took 3267 ms. Trace:
I20251211 20:52:43.787364 1165 rpcz_store.cc:276] 1211 20:52:40.520131 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:40.520191 (+ 60us) service_pool.cc:225] Handling call
1211 20:52:43.787223 (+3267032us) ts_tablet_manager.cc:1151] Deleting tablet 757ec913dff840f992a41a38b05f8d41
1211 20:52:43.787231 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:43.787289 (+ 58us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3266861}
I20251211 20:52:43.787783 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3acfef6f082641a199b80eefcfc772ac" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:43.788280 2415 tablet_loader.cc:96] loaded metadata for tablet d1391fcd70ab420bbfcb30738a871b6f (table default.Table_d15e45922b4340a59fce3b9dc10faf71 [id=8e1b3fd5c43a446ab4cd6a8a4b4ee8e1])
I20251211 20:52:43.788245 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3acfef6f082641a199b80eefcfc772ac" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:43.788410 1042 raft_consensus.cc:3060] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.789105 2415 tablet_loader.cc:96] loaded metadata for tablet d1ad0dc52d4548e0a9a7b92eae0e2c0e (table default.Table_76916c86042b49f487d21bf7aabfed56 [id=365cb9cb979d45dbaaff790819c90412])
I20251211 20:52:43.789436 2415 tablet_loader.cc:96] loaded metadata for tablet d1b3710a104e4480b4913e3df1bb31e0 (table default.Table_4721ef8d42714525ba28461e11e30dec [id=146fd60d3483482c8f46de7c44f3cc90])
I20251211 20:52:43.789988 2415 tablet_loader.cc:96] loaded metadata for tablet d3167c876d9d4c0d976ff21cbba653ab (table default.Table_dde47886ad4d4ea8a332c68e13085fe0 [id=8cc29ee550ed480c9e261d842fe08e3c])
I20251211 20:52:43.790308 2415 tablet_loader.cc:96] loaded metadata for tablet d35dd69f888747ec89c7a5f5f32cd409 (table default.Table_89da60cc38f54ac48f12f3b5988d48ad [id=e40a35fac2d7424ca101a4f6b4af3a2c])
I20251211 20:52:43.790670 2415 tablet_loader.cc:96] loaded metadata for tablet d37a7d0d662e45079b0cfc02765d2e01 (table default.Table_b26cc06e1b0745e08181124e1cbce934 [id=8a12299efe714617a970fc05c3bb6232])
I20251211 20:52:43.790975 2415 tablet_loader.cc:96] loaded metadata for tablet d391d4e6a101462d8d1220f1e17d0daa (table default.Table_c1f789cbcb1d4ddcbe038bc2ef5e2b92 [id=e154f2c3021f400c80ba23bcd269883f])
W20251211 20:52:43.791195 711 leader_election.cc:343] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.791679 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "260bdeb73439437cbf234d20f7264ab3" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:43.791839 771 raft_consensus.cc:3060] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.792032 2415 tablet_loader.cc:96] loaded metadata for tablet d43210ec61af423ebde624397dea9f2e (table default.Table_c64075b948a646458740b55e6970fd94 [id=7c5b2e33fa1c4b12b3941577360908a8])
I20251211 20:52:43.792560 1160 tablet_bootstrap.cc:492] T 5262c2241e524395a73c0eb368917169 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:43.792822 1160 ts_tablet_manager.cc:1403] T 5262c2241e524395a73c0eb368917169 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.008s
W20251211 20:52:43.793699 713 connection.cc:441] server connection from 127.0.0.1:37886 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 919) could send its response
W20251211 20:52:43.793821 713 connection.cc:441] server connection from 127.0.0.1:37886 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 926) could send its response
W20251211 20:52:43.793918 713 connection.cc:441] server connection from 127.0.0.1:37886 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 927) could send its response
W20251211 20:52:43.793998 713 connection.cc:441] server connection from 127.0.0.1:37886 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 934) could send its response
W20251211 20:52:43.794070 713 connection.cc:441] server connection from 127.0.0.1:37886 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 942) could send its response
I20251211 20:52:43.794127 1160 raft_consensus.cc:359] T 5262c2241e524395a73c0eb368917169 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.794373 2501 raft_consensus.cc:2804] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:43.794442 1160 raft_consensus.cc:385] T 5262c2241e524395a73c0eb368917169 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.794482 2501 raft_consensus.cc:697] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:43.794565 1160 raft_consensus.cc:740] T 5262c2241e524395a73c0eb368917169 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.794705 2501 consensus_queue.cc:237] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.794899 1160 consensus_queue.cc:260] T 5262c2241e524395a73c0eb368917169 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.793434 2415 tablet_loader.cc:96] loaded metadata for tablet d539d7e75d6d4e2b8486b134f971acfc (table default.Table_c1fc29564f5e4ee5b8f8cf8b19c744f0 [id=7a100133ee30444794457344356629ce])
I20251211 20:52:43.796353 2415 tablet_loader.cc:96] loaded metadata for tablet d6923cfd826d495291d9ac8e0b3b44c6 (table default.Table_68ccb2052d824ccf92f7162d2961fcb1 [id=9ac9591d4f04482d926710fdf6e797c6])
I20251211 20:52:43.797547 2415 tablet_loader.cc:96] loaded metadata for tablet d8943d496a254ccebc987ed3ea17fe3d (table default.Table_be91bd80b0374d5eae01b605de36b203 [id=eb7478f4f95f45479e8c830ba6575a40])
I20251211 20:52:43.798120 2415 tablet_loader.cc:96] loaded metadata for tablet da81bf8854a24954ba4548185812295d (table default.Table_18262e22c26d4d66ba4c6134fbde0368 [id=da8cf2b99b42409091484a9513623d6f])
I20251211 20:52:43.798441 2415 tablet_loader.cc:96] loaded metadata for tablet daf364774dea41bb8e94b49db55b8ea5 (table default.Table_0e243018f79e4e358c4f844e6ba51c35 [id=655a160341774729972a39fb7220a685])
I20251211 20:52:43.798498 2429 raft_consensus.cc:493] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.798621 2429 raft_consensus.cc:515] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.799218 2429 leader_election.cc:290] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:43.800477 847 leader_election.cc:343] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 260bdeb73439437cbf234d20f7264ab3
I20251211 20:52:43.800737 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3562b5effa124378beac7ccf0d27ee7e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:43.801009 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3562b5effa124378beac7ccf0d27ee7e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:43.801194 1047 raft_consensus.cc:2468] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:43.801283 1160 ts_tablet_manager.cc:1434] T 5262c2241e524395a73c0eb368917169 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20251211 20:52:43.802464 1160 tablet_bootstrap.cc:492] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:43.802431 711 leader_election.cc:343] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.802426 2507 consensus_queue.cc:1048] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:43.803154 710 leader_election.cc:304] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.803268 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 943) took 3250 ms. Trace:
I20251211 20:52:43.803341 1165 rpcz_store.cc:276] 1211 20:52:40.552538 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:40.552591 (+ 53us) service_pool.cc:225] Handling call
1211 20:52:43.787534 (+3234943us) ts_tablet_manager.cc:1151] Deleting tablet 79d92736bb1c46d2bfa5678b2d396ec2
1211 20:52:43.787542 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:43.803249 (+ 15707us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3234731}
I20251211 20:52:43.803684 2338 raft_consensus.cc:2804] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:43.803730 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 963) took 2750 ms. Trace:
I20251211 20:52:43.803758 2338 raft_consensus.cc:493] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:43.803821 2338 raft_consensus.cc:3060] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:43.803864 713 connection.cc:441] server connection from 127.0.0.1:37886 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 943) could send its response
W20251211 20:52:43.804764 711 consensus_peers.cc:597] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:43.803797 1165 rpcz_store.cc:276] 1211 20:52:41.052774 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:41.052846 (+ 72us) service_pool.cc:225] Handling call
1211 20:52:43.803610 (+2750764us) ts_tablet_manager.cc:1151] Deleting tablet a56f1a7bd5464d289bfd018c826ee574
1211 20:52:43.803622 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:43.803715 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2750429}
I20251211 20:52:43.805195 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 961) took 2752 ms. Trace:
I20251211 20:52:43.805260 1165 rpcz_store.cc:276] 1211 20:52:41.052601 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:41.053622 (+ 1021us) service_pool.cc:225] Handling call
1211 20:52:43.805094 (+2751472us) ts_tablet_manager.cc:1151] Deleting tablet 4776c456b4d74c90b1fc71d57b62a461
1211 20:52:43.805104 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:43.805180 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2751216}
I20251211 20:52:43.805508 1165 tablet_replica.cc:333] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:43.805696 1165 raft_consensus.cc:2243] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:43.806026 1165 raft_consensus.cc:2272] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:43.806672 713 connection.cc:441] server connection from 127.0.0.1:37886 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 963) could send its response
I20251211 20:52:43.806746 2415 tablet_loader.cc:96] loaded metadata for tablet dc5de12701714fcd81b9dba6973a28b5 (table default.Table_d465fa4868dd4c4db1036653f8b6f463 [id=cb97ccf262fd4c41a180267fd0678cec])
W20251211 20:52:43.806836 713 connection.cc:441] server connection from 127.0.0.1:37886 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 961) could send its response
I20251211 20:52:43.807734 2415 tablet_loader.cc:96] loaded metadata for tablet dd425601d89148eb95525f796a3abf1e (table default.Table_4e8d74eff9674d67b5b0f54370b06598 [id=c842792b01cb4c689fcfd3d796151508])
I20251211 20:52:43.807987 1165 ts_tablet_manager.cc:1916] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:43.808444 2415 tablet_loader.cc:96] loaded metadata for tablet de61a3b7ea634a9d9f329db19f74b630 (table default.Table_087c001a257049babda598744c761be8 [id=0e1a601c016d4b0e8e7d05969a4ea590])
I20251211 20:52:43.810041 2500 raft_consensus.cc:493] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.810178 2500 raft_consensus.cc:515] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.810792 2500 leader_election.cc:290] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.811460 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d37a7d0d662e45079b0cfc02765d2e01" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:43.812114 711 leader_election.cc:343] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.813956 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d37a7d0d662e45079b0cfc02765d2e01" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:43.814178 1047 raft_consensus.cc:2468] T d37a7d0d662e45079b0cfc02765d2e01 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:43.814683 710 leader_election.cc:304] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.814893 2415 tablet_loader.cc:96] loaded metadata for tablet df457783b2534b3ea23bb1fd529ce268 (table default.Table_6e9f3ca164384cc8ac4501deb3f0c6aa [id=342c2469d6e74e3e9217236e3aa863d6])
I20251211 20:52:43.815121 2501 raft_consensus.cc:2804] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:43.815248 2501 raft_consensus.cc:493] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:43.815280 2415 tablet_loader.cc:96] loaded metadata for tablet dfd194cf00f549c8937cb6ec40987240 (table default.Table_8db9acb9d35149fdabcb9a1d24061b46 [id=d919c6132b3c466e8a1404f981b003b8])
I20251211 20:52:43.815373 2501 raft_consensus.cc:3060] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.816197 1168 tablet_bootstrap.cc:492] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:43.816330 1168 ts_tablet_manager.cc:1403] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.050s user 0.003s sys 0.002s
I20251211 20:52:43.817454 1168 raft_consensus.cc:359] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.817612 1168 raft_consensus.cc:385] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.817659 1168 raft_consensus.cc:740] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.819041 1160 tablet_bootstrap.cc:654] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.820067 1168 consensus_queue.cc:260] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:43.820931 983 consensus_peers.cc:597] T b30d2054316b49fa8096345527a45d87 P e715a811a82446ce8d5bb4aafd5b81c7 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:43.820549 1168 ts_tablet_manager.cc:1434] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:43.821592 2415 tablet_loader.cc:96] loaded metadata for tablet dfeb47dfaa6445309e28715b1b8db2ae (table default.Table_bd0dcfadc59f4ffa81d0004ae868ed3e [id=c07b39a3869a4f3781ebccf30c95d5e5])
I20251211 20:52:43.822283 1168 tablet_bootstrap.cc:492] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:43.822688 2415 tablet_loader.cc:96] loaded metadata for tablet e218051cdc404af0b0335fa4c373222d (table default.Table_015ac77359ed47db98e074fa602a0141 [id=5db43d4090b947ffa3bdad62184f1db9])
I20251211 20:52:43.823757 2415 tablet_loader.cc:96] loaded metadata for tablet e3f711f02af843adb4f5c3c750924e33 (table default.Table_34bd5155aa3b4ef7818022ba9346c074 [id=83a61d23e80946439a14096ab3ab1bd9])
I20251211 20:52:43.823974 1168 tablet_bootstrap.cc:654] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.824457 2415 tablet_loader.cc:96] loaded metadata for tablet e43140b127cc462096008617ab8e6253 (table default.Table_60f0bdc6a1c849fd851ca0d31eeeaf79 [id=e6d99593c9aa45c7bca5a782bad9c904])
I20251211 20:52:43.824909 2415 tablet_loader.cc:96] loaded metadata for tablet e4647105ab57454d883ebfcde7bde891 (table default.Table_c817dec9a7674f699b3366fc302ead57 [id=d555d1fc6aef4a21b4d3f2804264d030])
I20251211 20:52:43.825302 2415 tablet_loader.cc:96] loaded metadata for tablet e484d499af1b4947adb36c32f8ea5704 (table default.Table_07f1881614084afd936866d115aecbe7 [id=eed7dfa66ab545dca02bd10c0a6a0e71])
I20251211 20:52:43.825351 1166 tablet_bootstrap.cc:492] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:43.825517 1166 ts_tablet_manager.cc:1403] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.048s user 0.005s sys 0.000s
I20251211 20:52:43.826166 2415 tablet_loader.cc:96] loaded metadata for tablet e53583fc6ae74e5a8293c611291a331b (table default.Table_48dfd045c6744e7db15278a2ebb7f434 [id=901b88b003d04e4783a9c09923910fdd])
I20251211 20:52:43.826407 1166 raft_consensus.cc:359] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.826612 1166 raft_consensus.cc:385] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.826792 1166 raft_consensus.cc:740] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.827050 1166 consensus_queue.cc:260] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.827713 2415 tablet_loader.cc:96] loaded metadata for tablet e59f8afe1834485d9fc64615e7bfe174 (table default.Table_bb1778853fbe4b7eaf0117deaf0877b9 [id=93b01cf646fd4f4dac01b7fecffa5f4e])
I20251211 20:52:43.828205 2415 tablet_loader.cc:96] loaded metadata for tablet e61d2400b5534a119106ad54d5859cbf (table default.Table_142734af8412498fa19dd25ad1ef2c2e [id=edf08c4121454149890dcef509afd417])
I20251211 20:52:43.829648 2415 tablet_loader.cc:96] loaded metadata for tablet e71c5eed203546fea69be5026da29fbb (table default.Table_2198fd77ec454943a0c0a392964c1fd6 [id=88c89157d3404a558c1d0ca2ddbcf3bc])
I20251211 20:52:43.833606 1166 ts_tablet_manager.cc:1434] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20251211 20:52:43.834854 2415 tablet_loader.cc:96] loaded metadata for tablet e89d4d76bf64453892a4d0b55fcc3454 (table default.Table_fba99384e9f14160b3e3504f8d1d0b07 [id=01bba75f854d4c338d6c09a446dd7499])
I20251211 20:52:43.836001 2500 raft_consensus.cc:493] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.837258 2415 tablet_loader.cc:96] loaded metadata for tablet e9b9f24dcdd54b34acb435091f093333 (table default.Table_e54af82590454fdf941b0462c30738ad [id=97e5719ea39a4ebe85ff613750c6863a])
I20251211 20:52:43.838020 1166 tablet_bootstrap.cc:492] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:43.839762 1166 tablet_bootstrap.cc:654] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.841732 1160 tablet_bootstrap.cc:492] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:43.841970 1160 ts_tablet_manager.cc:1403] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.040s user 0.000s sys 0.007s
I20251211 20:52:43.842815 1160 raft_consensus.cc:359] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.843091 1160 raft_consensus.cc:385] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.843209 1160 raft_consensus.cc:740] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.843528 1160 consensus_queue.cc:260] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.844931 1160 ts_tablet_manager.cc:1434] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:43.836153 2500 raft_consensus.cc:515] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.845928 1160 tablet_bootstrap.cc:492] T bfb52935296443c58190d4d06e2e6e9c P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:43.845906 2500 leader_election.cc:290] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.846264 2462 raft_consensus.cc:493] T 28ddb8a59a214576a25d613d42c2fef5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.846401 2462 raft_consensus.cc:515] T 28ddb8a59a214576a25d613d42c2fef5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.846597 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b7dd8232d974a43b99ec9b28075dc8d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:43.846814 1047 raft_consensus.cc:2468] T 6b7dd8232d974a43b99ec9b28075dc8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:43.846959 2462 leader_election.cc:290] T 28ddb8a59a214576a25d613d42c2fef5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.847632 2507 raft_consensus.cc:493] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:43.847772 2507 raft_consensus.cc:515] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.848362 2507 leader_election.cc:290] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:43.848690 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b7dd8232d974a43b99ec9b28075dc8d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:43.848881 710 leader_election.cc:304] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:43.849247 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28ddb8a59a214576a25d613d42c2fef5" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:43.849685 711 leader_election.cc:343] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.850003 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28ddb8a59a214576a25d613d42c2fef5" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:43.850373 2500 raft_consensus.cc:2804] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:43.850458 2500 raft_consensus.cc:493] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:43.850517 2500 raft_consensus.cc:3060] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.853526 1166 tablet_bootstrap.cc:492] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:43.854269 1166 ts_tablet_manager.cc:1403] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.002s
I20251211 20:52:43.855511 2415 tablet_loader.cc:96] loaded metadata for tablet ea03b0f5edb84116ae650e76092970fc (table default.Table_e3de355d0a69496c842b4a35daa99b90 [id=c34e32ee8d9f41b68288ced87e28069f])
I20251211 20:52:43.856438 1160 tablet_bootstrap.cc:654] T bfb52935296443c58190d4d06e2e6e9c P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.858325 2415 tablet_loader.cc:96] loaded metadata for tablet ea4483b0917045348eab2d05dd2a446e (table default.Table_31a95f7c09a34c4585eef1f7914a97a7 [id=8151557842dc4ca8ab0d1b3d663862ee])
I20251211 20:52:43.857235 914 raft_consensus.cc:3060] T f2d6c9781b4d4c37bc08703347963d5d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.860080 1166 raft_consensus.cc:359] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.860436 1166 raft_consensus.cc:385] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.860561 1166 raft_consensus.cc:740] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.857781 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "41bf5b5111f64dfa91ed76ed4ab4f6f7" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:43.861045 1166 consensus_queue.cc:260] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.861774 847 leader_election.cc:304] T 28ddb8a59a214576a25d613d42c2fef5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:43.862183 2462 raft_consensus.cc:2749] T 28ddb8a59a214576a25d613d42c2fef5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:43.862172 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "41bf5b5111f64dfa91ed76ed4ab4f6f7" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:43.862744 982 leader_election.cc:343] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:43.862974 983 leader_election.cc:343] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:43.863111 983 leader_election.cc:304] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.863379 2507 raft_consensus.cc:2749] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:43.865154 2415 tablet_loader.cc:96] loaded metadata for tablet eb6ae7c6ed0b40e19300d2922681205c (table default.Table_6a85485dbf204937a1c5a8e0e151fc68 [id=856b47e2455f4727a81e6291ab2ccf38])
I20251211 20:52:43.865697 2415 tablet_loader.cc:96] loaded metadata for tablet eb9da081cb9d43fd9a35c4bc51ea133e (table default.Table_b43dda4aa0674e3f8ef0d55e0e2431e3 [id=e0cd4de0a942462d9e3b50079f6b757d])
I20251211 20:52:43.866281 2415 tablet_loader.cc:96] loaded metadata for tablet ecc347e0b71a48e4ab803280232d139e (table default.Table_86d6443f6a564414a7f024e4aa12d450 [id=c1cd2830fb0648fab33480850c1d2adb])
I20251211 20:52:43.867097 2415 tablet_loader.cc:96] loaded metadata for tablet ee7b93b0b87541d9af6eb9e68b4cea41 (table default.Table_b6e6aea68c88426fb01353146c06b6ac [id=a4f984b4263a4bc38f6248628f643ae3])
I20251211 20:52:43.867720 2415 tablet_loader.cc:96] loaded metadata for tablet ef4fada332cf4b5c9da26ba5f908b4c6 (table default.Table_ddb3c1574c924ada922b623f332a3613 [id=ef058448a4ff49f59d0e943e11ed64b2])
I20251211 20:52:43.868765 2415 tablet_loader.cc:96] loaded metadata for tablet f00128ba84c245d9a0e35f26bc842b3e (table default.Table_6f8c133c7a364b93af2c83d8911a0ac3 [id=6a21c38ee7224a02b937df088494f5d8])
I20251211 20:52:43.870031 2415 tablet_loader.cc:96] loaded metadata for tablet f1630b5d35284a998934a30f86b6fcb7 (table default.Table_2eeb4b33f8424175a9e15a54e38099a2 [id=1b0b021139bc4dc4b08a90403f2d58d0])
I20251211 20:52:43.871459 2415 tablet_loader.cc:96] loaded metadata for tablet f1a765c46fc34ce2899b45ae3fa1d9a2 (table default.Table_e9b10d4bc5db49289ea753470f1585a6 [id=26d30d9b3c3d46fd9fbffa0f0124a63e])
I20251211 20:52:43.872740 2415 tablet_loader.cc:96] loaded metadata for tablet f1bca08aba654a05880eb3b45e669be4 (table default.Table_4e6562227887405aae36515729c90a28 [id=9c44b27aece14760b4c94bc8d9f82218])
I20251211 20:52:43.874034 2415 tablet_loader.cc:96] loaded metadata for tablet f262dfbc01824f6093785a21bcd27247 (table default.Table_fadb0061961d4706af19ddfb2f79d521 [id=8062ca5194e445a88f7dca2684519d16])
I20251211 20:52:43.875283 1160 tablet_bootstrap.cc:492] T bfb52935296443c58190d4d06e2e6e9c P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:43.875520 1160 ts_tablet_manager.cc:1403] T bfb52935296443c58190d4d06e2e6e9c P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.030s user 0.006s sys 0.000s
I20251211 20:52:43.876399 1160 raft_consensus.cc:359] T bfb52935296443c58190d4d06e2e6e9c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.876654 1160 raft_consensus.cc:385] T bfb52935296443c58190d4d06e2e6e9c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.876766 1160 raft_consensus.cc:740] T bfb52935296443c58190d4d06e2e6e9c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.877053 1160 consensus_queue.cc:260] T bfb52935296443c58190d4d06e2e6e9c P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.877665 1160 ts_tablet_manager.cc:1434] T bfb52935296443c58190d4d06e2e6e9c P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:43.880381 1160 tablet_bootstrap.cc:492] T 5710fc2f24764f818f7e5c86c6fd985e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:43.882377 1160 tablet_bootstrap.cc:654] T 5710fc2f24764f818f7e5c86c6fd985e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.883055 2415 tablet_loader.cc:96] loaded metadata for tablet f26ba8f4bd3a4a1f8fa2ba6e80d543f8 (table default.Table_71e9c158dc22435a8fbb02aecca8919d [id=b966d0c0b49d4023894950c5dc06f678])
I20251211 20:52:43.883912 2415 tablet_loader.cc:96] loaded metadata for tablet f2d6c9781b4d4c37bc08703347963d5d (table default.Table_2aa7ff9186a04249abf8cd971cc0cb17 [id=b50ab5b032ba4c98938324460512468f])
I20251211 20:52:43.884845 1042 raft_consensus.cc:2468] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:43.885324 2499 raft_consensus.cc:515] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.885877 2415 tablet_loader.cc:96] loaded metadata for tablet f4e9123ee4124d7dad8eb19b577d2664 (table default.Table_655c466dc1b54183b7e91ddeca33ae5e [id=be92c660569a46e7981df62185f0cbfd])
I20251211 20:52:43.886008 2499 leader_election.cc:290] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.886507 2415 tablet_loader.cc:96] loaded metadata for tablet f737a7bfb0f44196abdc831bb22d5285 (table default.Table_f12876c06fc04c63a2f23a6fe5b89f2b [id=a8664a1ea42f411bba24a8ac719155ce])
I20251211 20:52:43.886965 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5262c2241e524395a73c0eb368917169" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:43.887154 909 raft_consensus.cc:3060] T 5262c2241e524395a73c0eb368917169 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.887322 2415 tablet_loader.cc:96] loaded metadata for tablet f92d26bcde3b410aa56bc23a61b1ac35 (table default.Table_c2f4d6a8642a44b1b5566b7631b598e6 [id=bee5829ed7d44761a9ae57d3134d38dc])
I20251211 20:52:43.887959 2415 tablet_loader.cc:96] loaded metadata for tablet f94b656e3c1844509fdae36e33b263e8 (table default.Table_ed03ad475bed4804a65abe20efef0625 [id=e23f4305669a474aafa18928f130de3a])
I20251211 20:52:43.888960 2415 tablet_loader.cc:96] loaded metadata for tablet fb29f983c4524a49bdc5c0c1144ad708 (table default.Table_7809ac85fdfe4c1d9c58afb54edecb27 [id=c4b0bfff99f64888beb70d9154d2f156])
I20251211 20:52:43.889752 2415 tablet_loader.cc:96] loaded metadata for tablet fcf5afcfd7f142d29ff91272913e5fcb (table default.Table_5453a73a163149baa88d85b027bb42ee [id=b6d9975565684e9784951ede7dbd7988])
I20251211 20:52:43.891834 2415 tablet_loader.cc:96] loaded metadata for tablet fd43a27db69044b68826cda87e16a086 (table default.Table_a8ebfd7d7a7f4f6cae8b6b02b99b80b0 [id=f1d32002c612403a82bdff5baf209b9e])
I20251211 20:52:43.892246 710 leader_election.cc:304] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:43.892799 2415 tablet_loader.cc:96] loaded metadata for tablet fdb2b65b027f4abf9b55080c95ab27ae (table default.Table_cb402f83e7b24e308e46a798daa7db02 [id=d7bc4f722b6943438cffc3f0a7e5015a])
I20251211 20:52:43.894454 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5262c2241e524395a73c0eb368917169" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:43.894690 1042 raft_consensus.cc:3060] T 5262c2241e524395a73c0eb368917169 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.894857 2415 tablet_loader.cc:96] loaded metadata for tablet fdbb1787e1bd41a4a7f86beb1fe36b5a (table default.Table_eeb5507eb6a74588905b9e236e414324 [id=c4432f9a16b84796ae788c7a3cde370b])
I20251211 20:52:43.895377 2415 tablet_loader.cc:96] loaded metadata for tablet fed300d31c414f4a9b27c66e78097688 (table default.Table_9f6d83209f134db4a00fbddf8f323113 [id=e7d1c65b881746eeb12169e26fe0db70])
I20251211 20:52:43.896109 2415 tablet_loader.cc:96] loaded metadata for tablet ff4d5ec1f87247e896f9c099c532265a (table default.Table_8ead90e9bdef45bf8bf6c5d4c26c347a [id=475e0a1ec0344d10addbcfa42fe05b68])
I20251211 20:52:43.896517 2429 raft_consensus.cc:2804] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:43.896654 2429 raft_consensus.cc:697] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:43.897007 2415 tablet_loader.cc:96] loaded metadata for tablet ff9e65a8c1a245ef9b79b77848cafca0 (table default.Table_72c2c33862a149418a3c31c07caf8a37 [id=3e233f1f255847a5830bb85f65370aaa])
I20251211 20:52:43.896911 2429 consensus_queue.cc:237] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:43.897753 2415 catalog_manager.cc:1486] Time spent T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Loading table and tablet metadata into memory: real 1.889s user 0.365s sys 0.067s
I20251211 20:52:43.897868 2415 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251211 20:52:43.898119 1808 rpcz_store.cc:275] Call kudu.master.MasterService.ConnectToMaster from 127.0.0.1:46568 (request call id 0) took 1088 ms. Trace:
I20251211 20:52:43.898311 1808 rpcz_store.cc:276] 1211 20:52:42.810001 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:42.810060 (+ 59us) service_pool.cc:225] Handling call
1211 20:52:43.898068 (+1088008us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20251211 20:52:43.899808 1166 ts_tablet_manager.cc:1434] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.045s user 0.001s sys 0.001s
I20251211 20:52:43.900224 1166 tablet_bootstrap.cc:492] T 92efda2bd0c84d07a0af6b537c0cd882 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:43.901955 1166 tablet_bootstrap.cc:654] T 92efda2bd0c84d07a0af6b537c0cd882 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.906021 2499 consensus_queue.cc:1048] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:43.918128 1168 tablet_bootstrap.cc:492] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:43.918393 1168 ts_tablet_manager.cc:1403] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.097s user 0.002s sys 0.004s
I20251211 20:52:43.919988 1168 raft_consensus.cc:359] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.920297 1168 raft_consensus.cc:385] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.920392 1168 raft_consensus.cc:740] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.921402 1168 consensus_queue.cc:260] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.924995 1168 ts_tablet_manager.cc:1434] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20251211 20:52:43.925624 1168 tablet_bootstrap.cc:492] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:43.928704 914 raft_consensus.cc:3060] T 28ddb8a59a214576a25d613d42c2fef5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.927572 1166 tablet_bootstrap.cc:492] T 92efda2bd0c84d07a0af6b537c0cd882 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:43.930574 1166 ts_tablet_manager.cc:1403] T 92efda2bd0c84d07a0af6b537c0cd882 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.000s
I20251211 20:52:43.931185 911 raft_consensus.cc:3060] T bfb52935296443c58190d4d06e2e6e9c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:43.931556 1166 raft_consensus.cc:359] T 92efda2bd0c84d07a0af6b537c0cd882 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.932248 1168 tablet_bootstrap.cc:654] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.934850 1166 raft_consensus.cc:385] T 92efda2bd0c84d07a0af6b537c0cd882 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.935209 1166 raft_consensus.cc:740] T 92efda2bd0c84d07a0af6b537c0cd882 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.935736 1166 consensus_queue.cc:260] T 92efda2bd0c84d07a0af6b537c0cd882 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.936671 1166 ts_tablet_manager.cc:1434] T 92efda2bd0c84d07a0af6b537c0cd882 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20251211 20:52:43.937412 2429 consensus_queue.cc:1048] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:43.937762 1166 tablet_bootstrap.cc:492] T f25bde14792248b188660036f07ae902 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:43.939494 1166 tablet_bootstrap.cc:654] T f25bde14792248b188660036f07ae902 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.942799 2415 catalog_manager.cc:1269] Loaded cluster ID: 521ed6ceec724e0ab1d4cde545c10f31
I20251211 20:52:43.943404 1160 tablet_bootstrap.cc:492] T 5710fc2f24764f818f7e5c86c6fd985e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:43.943609 1160 ts_tablet_manager.cc:1403] T 5710fc2f24764f818f7e5c86c6fd985e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.065s user 0.008s sys 0.016s
I20251211 20:52:43.943627 2415 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251211 20:52:43.944408 1160 raft_consensus.cc:359] T 5710fc2f24764f818f7e5c86c6fd985e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.944607 1160 raft_consensus.cc:385] T 5710fc2f24764f818f7e5c86c6fd985e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:43.944687 1160 raft_consensus.cc:740] T 5710fc2f24764f818f7e5c86c6fd985e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:43.944914 1160 consensus_queue.cc:260] T 5710fc2f24764f818f7e5c86c6fd985e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.945427 1160 ts_tablet_manager.cc:1434] T 5710fc2f24764f818f7e5c86c6fd985e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:43.945820 1160 tablet_bootstrap.cc:492] T 645b08b33676420c96e880de3d460829 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:43.947567 1160 tablet_bootstrap.cc:654] T 645b08b33676420c96e880de3d460829 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:43.981753 1165 ts_tablet_manager.cc:1929] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251211 20:52:43.981902 1165 log.cc:1199] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/2630ad57845a4c31b20c8a104dd27335
I20251211 20:52:43.982252 1165 ts_tablet_manager.cc:1950] T 2630ad57845a4c31b20c8a104dd27335 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:43.982492 2494 raft_consensus.cc:515] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.983147 2494 leader_election.cc:290] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:43.983481 909 raft_consensus.cc:2468] T 5262c2241e524395a73c0eb368917169 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:43.983512 910 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f3855ffb51a74101b26fd28429e65bc2" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:43.983968 1042 raft_consensus.cc:2468] T 5262c2241e524395a73c0eb368917169 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:43.990692 710 leader_election.cc:304] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:43.991192 2494 raft_consensus.cc:2804] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:43.991320 2494 raft_consensus.cc:697] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:43.991545 2494 consensus_queue.cc:237] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:43.991736 2415 catalog_manager.cc:1514] Loading token signing keys...
I20251211 20:52:43.993216 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f3855ffb51a74101b26fd28429e65bc2" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:43.993438 1045 raft_consensus.cc:3060] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:43.999217 983 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 91613.4us
W20251211 20:52:44.002735 711 leader_election.cc:343] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:44.003249 711 consensus_peers.cc:597] T c7147571483e41ad970e573a08c368ca P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:44.010007 2560 raft_consensus.cc:493] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.010147 2560 raft_consensus.cc:515] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.010758 2560 leader_election.cc:290] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.011556 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0763a475527546bb83cee8fb823893ac" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:44.011632 910 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0763a475527546bb83cee8fb823893ac" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:44.013037 1160 tablet_bootstrap.cc:492] T 645b08b33676420c96e880de3d460829 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:44.013182 1160 ts_tablet_manager.cc:1403] T 645b08b33676420c96e880de3d460829 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.067s user 0.005s sys 0.006s
I20251211 20:52:44.013895 1160 raft_consensus.cc:359] T 645b08b33676420c96e880de3d460829 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.014057 1160 raft_consensus.cc:385] T 645b08b33676420c96e880de3d460829 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.014112 1160 raft_consensus.cc:740] T 645b08b33676420c96e880de3d460829 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.014308 1160 consensus_queue.cc:260] T 645b08b33676420c96e880de3d460829 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.014802 1160 ts_tablet_manager.cc:1434] T 645b08b33676420c96e880de3d460829 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:44.015103 1160 tablet_bootstrap.cc:492] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:44.017467 1160 tablet_bootstrap.cc:654] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:44.018132 711 leader_election.cc:343] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:44.021173 1166 tablet_bootstrap.cc:492] T f25bde14792248b188660036f07ae902 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:44.021306 1166 ts_tablet_manager.cc:1403] T f25bde14792248b188660036f07ae902 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.084s user 0.006s sys 0.002s
I20251211 20:52:44.022102 1166 raft_consensus.cc:359] T f25bde14792248b188660036f07ae902 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.022262 1166 raft_consensus.cc:385] T f25bde14792248b188660036f07ae902 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.022308 1166 raft_consensus.cc:740] T f25bde14792248b188660036f07ae902 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.022485 1166 consensus_queue.cc:260] T f25bde14792248b188660036f07ae902 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.022946 1166 ts_tablet_manager.cc:1434] T f25bde14792248b188660036f07ae902 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:44.028904 2429 raft_consensus.cc:493] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.029052 2429 raft_consensus.cc:515] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.029594 2429 leader_election.cc:290] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.030308 910 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3e6f62c2deff4c11b137fc0cba5ee234" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:44.030794 711 leader_election.cc:343] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:44.031186 1166 tablet_bootstrap.cc:492] T ce1799e6604f4ed7875ddedecab6f327 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:44.032428 2415 catalog_manager.cc:6038] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Loaded TSK: 0
I20251211 20:52:44.032833 1166 tablet_bootstrap.cc:654] T ce1799e6604f4ed7875ddedecab6f327 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:44.033622 2415 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251211 20:52:44.034014 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3e6f62c2deff4c11b137fc0cba5ee234" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:44.034204 1047 raft_consensus.cc:2468] T 3e6f62c2deff4c11b137fc0cba5ee234 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:44.034389 1168 tablet_bootstrap.cc:492] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:44.034521 1168 ts_tablet_manager.cc:1403] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.109s user 0.001s sys 0.005s
I20251211 20:52:44.034617 710 leader_election.cc:304] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:44.035270 1168 raft_consensus.cc:359] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.035436 1168 raft_consensus.cc:385] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.035486 1168 raft_consensus.cc:740] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.035770 1168 consensus_queue.cc:260] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.036082 2560 raft_consensus.cc:2804] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:44.036140 2560 raft_consensus.cc:493] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:44.036190 2560 raft_consensus.cc:3060] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.036741 1168 ts_tablet_manager.cc:1434] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:44.038911 2590 raft_consensus.cc:493] T b85af79c2574443998e847bc19c53b69 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.039043 2590 raft_consensus.cc:515] T b85af79c2574443998e847bc19c53b69 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.039644 2590 leader_election.cc:290] T b85af79c2574443998e847bc19c53b69 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:44.040212 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b85af79c2574443998e847bc19c53b69" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:44.040741 910 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b85af79c2574443998e847bc19c53b69" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:44.040848 982 leader_election.cc:343] T b85af79c2574443998e847bc19c53b69 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:44.041348 983 leader_election.cc:343] T b85af79c2574443998e847bc19c53b69 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:44.041450 983 leader_election.cc:304] T b85af79c2574443998e847bc19c53b69 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:44.041783 2590 raft_consensus.cc:2749] T b85af79c2574443998e847bc19c53b69 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:44.055148 910 raft_consensus.cc:3060] T 645b08b33676420c96e880de3d460829 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.058239 2429 consensus_queue.cc:1048] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:44.062942 1168 tablet_bootstrap.cc:492] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:44.067924 711 consensus_peers.cc:597] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:44.080289 2325 consensus_meta.cc:287] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919: Time spent flushing consensus metadata: real 0.547s user 0.002s sys 0.000s
I20251211 20:52:44.080431 2325 raft_consensus.cc:515] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.081020 2325 leader_election.cc:290] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.081995 1045 raft_consensus.cc:2468] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:44.082283 2462 raft_consensus.cc:493] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.082427 2462 raft_consensus.cc:515] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.082976 2462 leader_election.cc:290] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.082991 1166 tablet_bootstrap.cc:492] T ce1799e6604f4ed7875ddedecab6f327 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:44.083134 1166 ts_tablet_manager.cc:1403] T ce1799e6604f4ed7875ddedecab6f327 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.052s user 0.005s sys 0.005s
I20251211 20:52:44.083894 1166 raft_consensus.cc:359] T ce1799e6604f4ed7875ddedecab6f327 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.084053 1166 raft_consensus.cc:385] T ce1799e6604f4ed7875ddedecab6f327 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.084101 1166 raft_consensus.cc:740] T ce1799e6604f4ed7875ddedecab6f327 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.084168 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2e7e158809b9471fbd6e6ee9fb9ac569" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:44.084656 711 leader_election.cc:343] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:44.083069 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet ee7b93b0b87541d9af6eb9e68b4cea41 (table default.Table_b6e6aea68c88426fb01353146c06b6ac [id=a4f984b4263a4bc38f6248628f643ae3]). Expected version 3 got 0
I20251211 20:52:44.085630 710 leader_election.cc:304] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:44.086650 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7ec261837cf04bf88a48ff8add1b0d66" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:44.086834 772 raft_consensus.cc:2468] T 7ec261837cf04bf88a48ff8add1b0d66 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 0.
I20251211 20:52:44.086781 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2e7e158809b9471fbd6e6ee9fb9ac569" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:44.086956 1045 raft_consensus.cc:3060] T 2e7e158809b9471fbd6e6ee9fb9ac569 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.087141 1166 consensus_queue.cc:260] T ce1799e6604f4ed7875ddedecab6f327 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:44.087292 1810 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet ee7b93b0b87541d9af6eb9e68b4cea41 (table default.Table_b6e6aea68c88426fb01353146c06b6ac [id=a4f984b4263a4bc38f6248628f643ae3], current schema version=3): Not found: failed to reset TS proxy: No leader found: Tablet ee7b93b0b87541d9af6eb9e68b4cea41 consensus state has no leader
I20251211 20:52:44.087694 1166 ts_tablet_manager.cc:1434] T ce1799e6604f4ed7875ddedecab6f327 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:44.088686 824 heartbeater.cc:499] Master 127.0.104.61:40749 was elected leader, sending a full tablet report...
I20251211 20:52:44.088709 847 leader_election.cc:304] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters:
I20251211 20:52:44.089026 2460 raft_consensus.cc:2804] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:44.089116 2460 raft_consensus.cc:493] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:44.089200 2460 raft_consensus.cc:3060] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.089747 2325 raft_consensus.cc:2804] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:44.089887 2325 raft_consensus.cc:697] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:44.090103 2325 consensus_queue.cc:237] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.095346 1166 tablet_bootstrap.cc:492] T 71dc8d20d93243e289330d7b8c635511 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:44.095818 1806 master_service.cc:954] ReplaceTablet: received request to replace tablet c7bd218b3ff340b2a589c9517b49a649 from {username='slave'} at 127.0.0.1:46568
I20251211 20:52:44.096900 1166 tablet_bootstrap.cc:654] T 71dc8d20d93243e289330d7b8c635511 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:44.098613 1160 tablet_bootstrap.cc:492] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:44.098786 1160 ts_tablet_manager.cc:1403] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.084s user 0.007s sys 0.003s
I20251211 20:52:44.099633 1160 raft_consensus.cc:359] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.099833 1160 raft_consensus.cc:385] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.099915 1160 raft_consensus.cc:740] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
W20251211 20:52:44.100536 847 leader_election.cc:343] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 7ec261837cf04bf88a48ff8add1b0d66
I20251211 20:52:44.101364 1160 consensus_queue.cc:260] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.102984 2494 raft_consensus.cc:493] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.103026 1808 master_service.cc:954] ReplaceTablet: received request to replace tablet 76cfb26762d54593a150ec98125c6592 from {username='slave'} at 127.0.0.1:46568
I20251211 20:52:44.103119 2494 raft_consensus.cc:515] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.103710 2494 leader_election.cc:290] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.104415 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "215f8dc74a71482daa2ca80e15010f51" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:44.104589 1048 raft_consensus.cc:2468] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:44.104602 2494 consensus_queue.cc:1048] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:44.104902 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "215f8dc74a71482daa2ca80e15010f51" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:44.105252 710 leader_election.cc:304] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
W20251211 20:52:44.105629 711 leader_election.cc:343] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:44.105662 2494 raft_consensus.cc:2804] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:44.105760 2494 raft_consensus.cc:493] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:44.105813 2494 raft_consensus.cc:3060] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.106762 2590 raft_consensus.cc:493] T 46cb1d16a0e94b3a833f32c635213fd2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.106892 2590 raft_consensus.cc:515] T 46cb1d16a0e94b3a833f32c635213fd2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.106986 1160 ts_tablet_manager.cc:1434] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.008s user 0.000s sys 0.003s
I20251211 20:52:44.107434 2590 leader_election.cc:290] T 46cb1d16a0e94b3a833f32c635213fd2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:44.107484 1160 tablet_bootstrap.cc:492] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:44.108628 1795 master_service.cc:954] ReplaceTablet: received request to replace tablet 877aa237fe43452cbdbb4dac46f613ae from {username='slave'} at 127.0.0.1:46568
I20251211 20:52:44.108783 1168 tablet_bootstrap.cc:654] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:44.108830 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "46cb1d16a0e94b3a833f32c635213fd2" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:44.109169 1160 tablet_bootstrap.cc:654] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:44.109457 983 leader_election.cc:343] T 46cb1d16a0e94b3a833f32c635213fd2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:44.109652 2286 raft_consensus.cc:493] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.109768 2286 raft_consensus.cc:515] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.110282 2286 leader_election.cc:290] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.110918 1810 master_service.cc:954] ReplaceTablet: received request to replace tablet e9ead34065e240eeb1f946319537d32a from {username='slave'} at 127.0.0.1:46568
I20251211 20:52:44.110948 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c91ff29847d54e55b19321652a857e9f" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:44.111660 711 leader_election.cc:343] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:44.113680 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "46cb1d16a0e94b3a833f32c635213fd2" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:44.114785 982 leader_election.cc:343] T 46cb1d16a0e94b3a833f32c635213fd2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:44.114912 982 leader_election.cc:304] T 46cb1d16a0e94b3a833f32c635213fd2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:44.115221 2603 raft_consensus.cc:2749] T 46cb1d16a0e94b3a833f32c635213fd2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:44.115573 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c91ff29847d54e55b19321652a857e9f" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:44.115765 1044 raft_consensus.cc:2468] T c91ff29847d54e55b19321652a857e9f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:44.116786 710 leader_election.cc:304] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:44.117158 2499 raft_consensus.cc:2804] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:44.117237 2499 raft_consensus.cc:493] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:44.117290 2499 raft_consensus.cc:3060] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.103878 1808 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_cec07d512c8049e581be33a0214fad11" } new_table_name: "default.Table_68261841754847ad8d22e86c29886144" modify_external_catalogs: true
I20251211 20:52:44.108389 1796 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_19882e719bca4562a2fa3ff1f6e98590"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:44.123201 1796 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_19882e719bca4562a2fa3ff1f6e98590 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.107532 1797 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_ff29e8c9caa441708463eebe086f5ba9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:44.126245 1797 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff29e8c9caa441708463eebe086f5ba9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.126955 1042 raft_consensus.cc:2468] T 0763a475527546bb83cee8fb823893ac P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:44.127493 710 leader_election.cc:304] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:44.108533 1799 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_f611fe8c909d4af2af7c49dc28f4d2a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:44.128013 2561 raft_consensus.cc:2804] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:44.128091 2561 raft_consensus.cc:493] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:44.128149 2561 raft_consensus.cc:3060] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:44.128270 1799 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f611fe8c909d4af2af7c49dc28f4d2a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.109019 1798 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_89bf0fe9a0c54cae85b1b8305d2b91e8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:44.130030 1798 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_89bf0fe9a0c54cae85b1b8305d2b91e8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.109750 1795 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_a205c236aef84e30921a240403104033"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:44.132309 1795 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a205c236aef84e30921a240403104033 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.137979 909 raft_consensus.cc:3060] T 22273287aa9245a2a66131fed2c8861c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.111475 1810 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_34c853fcb87d480fb23827277ebed594"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:44.139305 1810 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_34c853fcb87d480fb23827277ebed594 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.110800 1802 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_37849576518b4320a3bac34baa096242" } new_table_name: "default.Table_22bed4b4e9e24a90818b19c6f5c91e1f" modify_external_catalogs: true
W20251211 20:52:44.148814 1784 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet ee7b93b0b87541d9af6eb9e68b4cea41 (table default.Table_b6e6aea68c88426fb01353146c06b6ac [id=a4f984b4263a4bc38f6248628f643ae3], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet ee7b93b0b87541d9af6eb9e68b4cea41 consensus state has no leader
I20251211 20:52:44.154742 2286 consensus_queue.cc:1048] T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251211 20:52:44.097229 1809 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_d8214c15312c49ce85f2c52b14cf4a78"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:44.158314 1809 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d8214c15312c49ce85f2c52b14cf4a78 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.198670 2601 raft_consensus.cc:493] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.198834 2601 raft_consensus.cc:515] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.199445 2601 leader_election.cc:290] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:44.200173 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7452dd736b0440fcb4626858e794676f" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:44.200562 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7452dd736b0440fcb4626858e794676f" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:44.200758 982 leader_election.cc:343] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:44.201042 983 leader_election.cc:343] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:44.201138 983 leader_election.cc:304] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:44.201639 2601 raft_consensus.cc:2749] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:44.206168 2462 raft_consensus.cc:493] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.206355 2462 raft_consensus.cc:515] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.207037 2462 leader_election.cc:290] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.208235 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4ce495233a5e4f3792faf44680df54ba" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:44.208447 772 raft_consensus.cc:2468] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 0.
I20251211 20:52:44.208993 847 leader_election.cc:304] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters:
W20251211 20:52:44.210106 847 leader_election.cc:343] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 4ce495233a5e4f3792faf44680df54ba
I20251211 20:52:44.210546 2462 raft_consensus.cc:2804] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:44.210654 2462 raft_consensus.cc:493] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:44.210733 2462 raft_consensus.cc:3060] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.211683 2601 raft_consensus.cc:493] T 92efda2bd0c84d07a0af6b537c0cd882 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.211854 2601 raft_consensus.cc:515] T 92efda2bd0c84d07a0af6b537c0cd882 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.212479 2601 leader_election.cc:290] T 92efda2bd0c84d07a0af6b537c0cd882 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:44.213124 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "92efda2bd0c84d07a0af6b537c0cd882" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:44.213346 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "92efda2bd0c84d07a0af6b537c0cd882" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:44.213840 983 leader_election.cc:343] T 92efda2bd0c84d07a0af6b537c0cd882 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:44.214313 982 leader_election.cc:343] T 92efda2bd0c84d07a0af6b537c0cd882 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:44.214414 982 leader_election.cc:304] T 92efda2bd0c84d07a0af6b537c0cd882 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:44.214895 2601 raft_consensus.cc:2749] T 92efda2bd0c84d07a0af6b537c0cd882 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:44.215541 2601 raft_consensus.cc:493] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.215737 2601 raft_consensus.cc:515] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.216353 2601 leader_election.cc:290] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:44.216792 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c26c3d0170f64a41ae8944c6668cc5f3" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:44.217360 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c26c3d0170f64a41ae8944c6668cc5f3" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:44.217828 983 leader_election.cc:343] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:44.218482 982 leader_election.cc:343] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:44.218587 982 leader_election.cc:304] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:44.219727 2601 raft_consensus.cc:2749] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:44.227929 911 raft_consensus.cc:3060] T 5710fc2f24764f818f7e5c86c6fd985e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.228838 1160 tablet_bootstrap.cc:492] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:44.229074 1168 tablet_bootstrap.cc:492] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:44.229274 1168 ts_tablet_manager.cc:1403] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.166s user 0.042s sys 0.025s
I20251211 20:52:44.229523 1160 ts_tablet_manager.cc:1403] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.122s user 0.011s sys 0.015s
I20251211 20:52:44.230084 1168 raft_consensus.cc:359] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.230283 1168 raft_consensus.cc:385] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.230358 1168 raft_consensus.cc:740] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.230605 1168 consensus_queue.cc:260] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.231707 1160 raft_consensus.cc:359] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.232486 1168 ts_tablet_manager.cc:1434] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:44.232712 1160 raft_consensus.cc:385] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.233777 1168 tablet_bootstrap.cc:492] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:44.235034 1160 raft_consensus.cc:740] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.235380 1160 consensus_queue.cc:260] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.235647 1168 tablet_bootstrap.cc:654] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:44.240438 1166 tablet_bootstrap.cc:492] T 71dc8d20d93243e289330d7b8c635511 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:44.240674 1166 ts_tablet_manager.cc:1403] T 71dc8d20d93243e289330d7b8c635511 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.145s user 0.014s sys 0.012s
I20251211 20:52:44.241511 1166 raft_consensus.cc:359] T 71dc8d20d93243e289330d7b8c635511 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.241740 1166 raft_consensus.cc:385] T 71dc8d20d93243e289330d7b8c635511 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.241828 1166 raft_consensus.cc:740] T 71dc8d20d93243e289330d7b8c635511 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.242055 1166 consensus_queue.cc:260] T 71dc8d20d93243e289330d7b8c635511 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.243003 1166 ts_tablet_manager.cc:1434] T 71dc8d20d93243e289330d7b8c635511 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:44.243315 1166 tablet_bootstrap.cc:492] T d43210ec61af423ebde624397dea9f2e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:44.244885 1168 tablet_bootstrap.cc:492] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:44.245071 1168 ts_tablet_manager.cc:1403] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.002s
I20251211 20:52:44.245860 1168 raft_consensus.cc:359] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.246102 1168 raft_consensus.cc:385] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.246193 1168 raft_consensus.cc:740] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.247011 1168 consensus_queue.cc:260] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:44.247917 711 consensus_peers.cc:597] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:44.248608 1168 ts_tablet_manager.cc:1434] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:44.250362 1168 tablet_bootstrap.cc:492] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:44.251029 2325 consensus_queue.cc:1048] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:44.253414 1168 tablet_bootstrap.cc:654] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:44.260036 1160 ts_tablet_manager.cc:1434] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.029s user 0.002s sys 0.000s
I20251211 20:52:44.260542 1160 tablet_bootstrap.cc:492] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:44.262342 2601 raft_consensus.cc:493] T d391d4e6a101462d8d1220f1e17d0daa P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.262507 2601 raft_consensus.cc:515] T d391d4e6a101462d8d1220f1e17d0daa P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.263154 2601 leader_election.cc:290] T d391d4e6a101462d8d1220f1e17d0daa P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:44.264364 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d391d4e6a101462d8d1220f1e17d0daa" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:44.264915 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d391d4e6a101462d8d1220f1e17d0daa" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:44.265475 983 leader_election.cc:343] T d391d4e6a101462d8d1220f1e17d0daa P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:44.266177 982 leader_election.cc:343] T d391d4e6a101462d8d1220f1e17d0daa P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:44.266284 982 leader_election.cc:304] T d391d4e6a101462d8d1220f1e17d0daa P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:44.266891 1168 tablet_bootstrap.cc:492] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:44.267051 1168 ts_tablet_manager.cc:1403] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.005s
I20251211 20:52:44.268203 913 raft_consensus.cc:3060] T a9bb22dce13f4c9da76ca446715cd99a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.268402 1160 tablet_bootstrap.cc:654] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:44.268834 2601 raft_consensus.cc:2749] T d391d4e6a101462d8d1220f1e17d0daa P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:44.269178 711 consensus_peers.cc:597] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:44.269316 1168 raft_consensus.cc:359] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.269474 1168 raft_consensus.cc:385] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.269554 1168 raft_consensus.cc:740] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.269873 1168 consensus_queue.cc:260] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.271202 1168 ts_tablet_manager.cc:1434] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:44.272325 1168 tablet_bootstrap.cc:492] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:44.274009 1168 tablet_bootstrap.cc:654] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:44.274714 2624 log.cc:927] Time spent T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Append to log took a long time: real 0.063s user 0.000s sys 0.000s
I20251211 20:52:44.278402 1166 tablet_bootstrap.cc:654] T d43210ec61af423ebde624397dea9f2e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:44.284512 1160 tablet_bootstrap.cc:492] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:44.284914 1160 ts_tablet_manager.cc:1403] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.001s
I20251211 20:52:44.286043 1160 raft_consensus.cc:359] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.287768 1160 raft_consensus.cc:385] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.290103 2325 consensus_queue.cc:1048] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:44.290387 1160 raft_consensus.cc:740] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.291004 1160 consensus_queue.cc:260] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.292237 1160 ts_tablet_manager.cc:1434] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20251211 20:52:44.293351 1160 tablet_bootstrap.cc:492] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:44.294137 1166 tablet_bootstrap.cc:492] T d43210ec61af423ebde624397dea9f2e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:44.297757 1166 ts_tablet_manager.cc:1403] T d43210ec61af423ebde624397dea9f2e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.055s user 0.006s sys 0.000s
I20251211 20:52:44.298667 1166 raft_consensus.cc:359] T d43210ec61af423ebde624397dea9f2e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.298899 1166 raft_consensus.cc:385] T d43210ec61af423ebde624397dea9f2e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.298970 1166 raft_consensus.cc:740] T d43210ec61af423ebde624397dea9f2e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.299228 1166 consensus_queue.cc:260] T d43210ec61af423ebde624397dea9f2e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.300179 1166 ts_tablet_manager.cc:1434] T d43210ec61af423ebde624397dea9f2e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:44.300721 1166 tablet_bootstrap.cc:492] T 18b977f463d5420186c007b38110e07e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:44.309334 2601 raft_consensus.cc:493] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.309495 2601 raft_consensus.cc:515] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.310109 2601 leader_election.cc:290] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:44.310997 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d69fb1f6d9594afcbfb844860b41a377" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:44.311314 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d69fb1f6d9594afcbfb844860b41a377" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:44.312065 983 leader_election.cc:343] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:44.312345 982 leader_election.cc:343] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:44.312434 982 leader_election.cc:304] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:44.312891 2601 raft_consensus.cc:2749] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:44.313865 1160 tablet_bootstrap.cc:654] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:44.322968 1160 tablet_bootstrap.cc:492] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:44.323212 1160 ts_tablet_manager.cc:1403] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.002s
I20251211 20:52:44.328286 1160 raft_consensus.cc:359] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.328568 1160 raft_consensus.cc:385] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.328694 1160 raft_consensus.cc:740] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.328965 1160 consensus_queue.cc:260] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.329619 1160 ts_tablet_manager.cc:1434] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20251211 20:52:44.331908 1160 tablet_bootstrap.cc:492] T c7147571483e41ad970e573a08c368ca P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:44.333338 1796 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet bfb52935296443c58190d4d06e2e6e9c (table default.Table_3721b5725e8c4297a392d3986a081128 [id=383392d835da4c50848068d83261cecc]). Expected version 7 got 0
I20251211 20:52:44.336052 1160 tablet_bootstrap.cc:654] T c7147571483e41ad970e573a08c368ca P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:44.336824 961 heartbeater.cc:499] Master 127.0.104.61:40749 was elected leader, sending a full tablet report...
I20251211 20:52:44.339707 2629 raft_consensus.cc:493] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.339922 2629 raft_consensus.cc:515] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.340631 2629 leader_election.cc:290] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.341979 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2207af105bc3471ca930d35a91d38e70" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:44.342170 772 raft_consensus.cc:2468] T 2207af105bc3471ca930d35a91d38e70 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 0.
W20251211 20:52:44.342896 847 leader_election.cc:343] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 2207af105bc3471ca930d35a91d38e70
I20251211 20:52:44.343166 847 leader_election.cc:304] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters: e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:44.343703 2629 raft_consensus.cc:2804] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:44.343830 2629 raft_consensus.cc:493] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:44.343909 2629 raft_consensus.cc:3060] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.339283 1796 catalog_manager.cc:5654] T 861d2d70a24946699dccd65cab05195e P e715a811a82446ce8d5bb4aafd5b81c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } }
I20251211 20:52:44.347549 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 861d2d70a24946699dccd65cab05195e (table default.Table_819c4206e91e41358ac71e82185b4140 [id=28be836b71ed4107af7adc3fcb948b5e]). Expected version 1 got 0
I20251211 20:52:44.347901 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet ce1799e6604f4ed7875ddedecab6f327 (table default.Table_5db1ac98e36d45b1b01889ee275ad83d [id=a6849726434d41d29dce12f47066a685]). Expected version 1 got 0
I20251211 20:52:44.356884 1160 tablet_bootstrap.cc:492] T c7147571483e41ad970e573a08c368ca P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:44.357136 1160 ts_tablet_manager.cc:1403] T c7147571483e41ad970e573a08c368ca P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.000s
I20251211 20:52:44.358059 1160 raft_consensus.cc:359] T c7147571483e41ad970e573a08c368ca P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.358304 1160 raft_consensus.cc:385] T c7147571483e41ad970e573a08c368ca P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.358409 1160 raft_consensus.cc:740] T c7147571483e41ad970e573a08c368ca P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.358716 1160 consensus_queue.cc:260] T c7147571483e41ad970e573a08c368ca P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.359692 1166 tablet_bootstrap.cc:654] T 18b977f463d5420186c007b38110e07e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:44.364717 2325 raft_consensus.cc:493] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.364876 2325 raft_consensus.cc:515] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.365487 2325 leader_election.cc:290] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.366686 1798 master_service.cc:954] ReplaceTablet: received request to replace tablet 2e29df291d8d40daba83e1d9f97f902b from {username='slave'} at 127.0.0.1:46568
I20251211 20:52:44.367884 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e484d499af1b4947adb36c32f8ea5704" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:44.368084 1047 raft_consensus.cc:2468] T e484d499af1b4947adb36c32f8ea5704 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:44.368597 710 leader_election.cc:304] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:44.368707 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e484d499af1b4947adb36c32f8ea5704" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:44.368960 1808 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 03184dcc56dd4e92adc7d0ad22aa8d8b has no consensus state
I20251211 20:52:44.369257 2429 raft_consensus.cc:2804] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:44.369333 2429 raft_consensus.cc:493] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:44.369390 2429 raft_consensus.cc:3060] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.369400 1168 tablet_bootstrap.cc:492] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:44.369535 1168 ts_tablet_manager.cc:1403] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.097s user 0.006s sys 0.016s
W20251211 20:52:44.370219 2637 log.cc:927] Time spent T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7: Append to log took a long time: real 0.062s user 0.000s sys 0.000s
I20251211 20:52:44.370297 1168 raft_consensus.cc:359] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.370522 1168 raft_consensus.cc:385] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.370572 1168 raft_consensus.cc:740] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
W20251211 20:52:44.370716 711 leader_election.cc:343] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:44.370806 1168 consensus_queue.cc:260] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.371295 1168 ts_tablet_manager.cc:1434] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:44.372146 1168 tablet_bootstrap.cc:492] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:44.374732 1168 tablet_bootstrap.cc:654] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:44.377424 2601 raft_consensus.cc:493] T 47779034ad89466e9e1801987128b4dd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.377578 2601 raft_consensus.cc:515] T 47779034ad89466e9e1801987128b4dd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.378201 2601 leader_election.cc:290] T 47779034ad89466e9e1801987128b4dd P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:44.379256 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "47779034ad89466e9e1801987128b4dd" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:44.379496 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "47779034ad89466e9e1801987128b4dd" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:44.379894 983 leader_election.cc:343] T 47779034ad89466e9e1801987128b4dd P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:44.380055 982 leader_election.cc:343] T 47779034ad89466e9e1801987128b4dd P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:44.380128 982 leader_election.cc:304] T 47779034ad89466e9e1801987128b4dd P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:44.380513 2601 raft_consensus.cc:2749] T 47779034ad89466e9e1801987128b4dd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:44.382047 1166 tablet_bootstrap.cc:492] T 18b977f463d5420186c007b38110e07e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:44.382177 1166 ts_tablet_manager.cc:1403] T 18b977f463d5420186c007b38110e07e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.082s user 0.003s sys 0.007s
I20251211 20:52:44.382889 1166 raft_consensus.cc:359] T 18b977f463d5420186c007b38110e07e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.383064 1166 raft_consensus.cc:385] T 18b977f463d5420186c007b38110e07e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.383111 1166 raft_consensus.cc:740] T 18b977f463d5420186c007b38110e07e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.383294 1166 consensus_queue.cc:260] T 18b977f463d5420186c007b38110e07e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.383805 1166 ts_tablet_manager.cc:1434] T 18b977f463d5420186c007b38110e07e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:44.386718 1806 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Sending DeleteTablet for 3 replicas of tablet c7bd218b3ff340b2a589c9517b49a649
I20251211 20:52:44.387208 1160 ts_tablet_manager.cc:1434] T c7147571483e41ad970e573a08c368ca P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.030s user 0.002s sys 0.001s
I20251211 20:52:44.387895 1160 tablet_bootstrap.cc:492] T 3de5b34b96404cdc9c191fed7f055638 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:44.388059 1806 catalog_manager.cc:6724] ReplaceTablet: tablet c7bd218b3ff340b2a589c9517b49a649 deleted and replaced by tablet dd1ae033dedc436fa3a3f864a7a208a2
W20251211 20:52:44.388679 1802 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet aab26df748724fe5b2bb7fad9e3bfc0c consensus state has no leader
I20251211 20:52:44.390823 756 tablet_service.cc:1552] Processing DeleteTablet for tablet c7bd218b3ff340b2a589c9517b49a649 with delete_type TABLET_DATA_DELETED (replaced by tablet dd1ae033dedc436fa3a3f864a7a208a2) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:44.392102 1160 tablet_bootstrap.cc:654] T 3de5b34b96404cdc9c191fed7f055638 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:44.398851 1166 tablet_bootstrap.cc:492] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:44.398846 2286 raft_consensus.cc:493] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.398998 2286 raft_consensus.cc:515] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.399322 1168 tablet_bootstrap.cc:492] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:44.399477 1168 ts_tablet_manager.cc:1403] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.002s
I20251211 20:52:44.400190 2286 leader_election.cc:290] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.400465 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "efe4d6bb07374294b8ab50a9fe605f87" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:44.401042 711 leader_election.cc:343] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:44.400274 1168 raft_consensus.cc:359] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.401788 1168 raft_consensus.cc:385] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.401877 1168 raft_consensus.cc:740] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.402100 1168 consensus_queue.cc:260] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.402637 1168 ts_tablet_manager.cc:1434] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:44.402832 1166 tablet_bootstrap.cc:654] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:44.407289 1168 tablet_bootstrap.cc:492] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:44.409245 2286 raft_consensus.cc:493] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.409423 2286 raft_consensus.cc:515] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.410063 2286 leader_election.cc:290] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.410466 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fa79b67b9eee45e2b41c4a90967606cc" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:44.411082 711 leader_election.cc:343] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:44.411275 1797 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_78f3b1065a0a43d891bb775169c0eaca" } new_table_name: "default.Table_1a73d567e8914a83974e3e8f5f58be0b" modify_external_catalogs: true
I20251211 20:52:44.411696 1160 tablet_bootstrap.cc:492] T 3de5b34b96404cdc9c191fed7f055638 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:44.411855 1160 ts_tablet_manager.cc:1403] T 3de5b34b96404cdc9c191fed7f055638 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.000s
I20251211 20:52:44.412662 1160 raft_consensus.cc:359] T 3de5b34b96404cdc9c191fed7f055638 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.412971 1160 raft_consensus.cc:385] T 3de5b34b96404cdc9c191fed7f055638 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.413062 1160 raft_consensus.cc:740] T 3de5b34b96404cdc9c191fed7f055638 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.413357 1160 consensus_queue.cc:260] T 3de5b34b96404cdc9c191fed7f055638 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.414038 1160 ts_tablet_manager.cc:1434] T 3de5b34b96404cdc9c191fed7f055638 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:44.414278 1806 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_a5b944a25f4c45cf8c70765ea43d1ac2" } modify_external_catalogs: true
I20251211 20:52:44.414497 1806 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_a5b944a25f4c45cf8c70765ea43d1ac2" } modify_external_catalogs: true
I20251211 20:52:44.416204 1802 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_87f092378445464ab48c155fae469f29" } modify_external_catalogs: true
I20251211 20:52:44.416393 1802 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_87f092378445464ab48c155fae469f29" } modify_external_catalogs: true
I20251211 20:52:44.416994 1808 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_cd866c64dfc7477782129c897b849771"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:44.417480 1043 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "efe4d6bb07374294b8ab50a9fe605f87" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
W20251211 20:52:44.417604 1808 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cd866c64dfc7477782129c897b849771 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.417675 1043 raft_consensus.cc:2468] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:44.418236 1795 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_f84cd3de863b446894133bb40aaf1a39" } modify_external_catalogs: true
I20251211 20:52:44.418424 1795 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_f84cd3de863b446894133bb40aaf1a39" } modify_external_catalogs: true
I20251211 20:52:44.418879 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fa79b67b9eee45e2b41c4a90967606cc" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:44.419044 1049 raft_consensus.cc:2468] T fa79b67b9eee45e2b41c4a90967606cc P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:44.416994 1809 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_50aa8aa5823a4e71b0f191ff52eefb75"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:44.421291 1798 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_bdb9aa9507d94416bbc3c8fe9cb11fa7" } modify_external_catalogs: true
I20251211 20:52:44.421464 710 leader_election.cc:304] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:44.410104 1168 tablet_bootstrap.cc:654] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:44.421784 2286 raft_consensus.cc:2804] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:44.421928 2286 raft_consensus.cc:493] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:44.422031 2286 raft_consensus.cc:3060] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.422183 1798 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_bdb9aa9507d94416bbc3c8fe9cb11fa7" } modify_external_catalogs: true
I20251211 20:52:44.422922 1799 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_717a9e3f73be421585b11e3cdad48b64" } new_table_name: "default.Table_c73d4009f4bc41b38aee5612313b30c8" modify_external_catalogs: true
I20251211 20:52:44.425678 1810 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_52e650ce0b824a41a39fcd4f1b8ba455" } modify_external_catalogs: true
I20251211 20:52:44.425974 1810 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_52e650ce0b824a41a39fcd4f1b8ba455" } modify_external_catalogs: true
W20251211 20:52:44.428992 1809 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_50aa8aa5823a4e71b0f191ff52eefb75 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.432163 710 leader_election.cc:304] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:44.432663 2325 raft_consensus.cc:2804] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:44.432797 2325 raft_consensus.cc:493] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:44.432879 2325 raft_consensus.cc:3060] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:44.432925 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet ce1799e6604f4ed7875ddedecab6f327 (table default.Table_5db1ac98e36d45b1b01889ee275ad83d [id=a6849726434d41d29dce12f47066a685], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet ce1799e6604f4ed7875ddedecab6f327 consensus state has no leader
I20251211 20:52:44.433573 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet c7bd218b3ff340b2a589c9517b49a649 with delete_type TABLET_DATA_DELETED (replaced by tablet dd1ae033dedc436fa3a3f864a7a208a2) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:44.433645 1015 tablet_service.cc:1552] Processing DeleteTablet for tablet 2e33606930774fe4aafc6e3819618af8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:44.434264 1096 heartbeater.cc:499] Master 127.0.104.61:40749 was elected leader, sending a full tablet report...
I20251211 20:52:44.435331 1166 tablet_bootstrap.cc:492] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:44.443331 1166 ts_tablet_manager.cc:1403] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.045s user 0.003s sys 0.002s
I20251211 20:52:44.444671 1168 tablet_bootstrap.cc:492] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:44.445770 1166 raft_consensus.cc:359] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.446858 1168 ts_tablet_manager.cc:1403] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.040s user 0.002s sys 0.005s
I20251211 20:52:44.448309 1168 raft_consensus.cc:359] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.449433 1166 raft_consensus.cc:385] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.449826 1166 raft_consensus.cc:740] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.450320 1166 consensus_queue.cc:260] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.451224 1168 raft_consensus.cc:385] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.452085 1168 raft_consensus.cc:740] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.452704 1166 ts_tablet_manager.cc:1434] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20251211 20:52:44.453356 1168 consensus_queue.cc:260] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.454573 1796 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_0be97b736b654803810e4c33d0b01df7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:44.455353 1797 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7b9f569554fb4836ac2b62542c3ecf7e consensus state has no leader
W20251211 20:52:44.455356 1796 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0be97b736b654803810e4c33d0b01df7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.455838 1797 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_f6f977edac0f45bea7e94de5bfa51ee4" } new_table_name: "default.Table_34e6ed4f13da437ea50d349b92e83bd6" modify_external_catalogs: true
I20251211 20:52:44.458060 1166 tablet_bootstrap.cc:492] T 95866de91f9d437d80f40b7962253610 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:44.460151 1166 tablet_bootstrap.cc:654] T 95866de91f9d437d80f40b7962253610 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:44.461019 1797 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_d87f970fdabb4d10871f55bb8193bd4d" } new_table_name: "default.Table_296bdf30a11541c389afe3df0b0f667a" modify_external_catalogs: true
I20251211 20:52:44.465435 894 tablet_service.cc:1552] Processing DeleteTablet for tablet c7bd218b3ff340b2a589c9517b49a649 with delete_type TABLET_DATA_DELETED (replaced by tablet dd1ae033dedc436fa3a3f864a7a208a2) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:44.468986 1806 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Sending DeleteTablet for 3 replicas of tablet 9b99860ee6be45a0b358e686332d3137
W20251211 20:52:44.474888 2622 log.cc:927] Time spent T 3de5b34b96404cdc9c191fed7f055638 P 6f8d784fd1cd492bbe3809403ec34919: Append to log took a long time: real 0.296s user 0.000s sys 0.000s
I20251211 20:52:44.475296 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 9b99860ee6be45a0b358e686332d3137 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:44.470574 1015 tablet_service.cc:1552] Processing DeleteTablet for tablet 9b99860ee6be45a0b358e686332d3137 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:44.475976 2662 tablet_replica.cc:333] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
W20251211 20:52:44.485685 710 consensus_peers.cc:597] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:44.485687 2662 raft_consensus.cc:2243] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:44.485908 2662 raft_consensus.cc:2272] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:44.486622 2662 ts_tablet_manager.cc:1916] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:44.490924 1799 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 41627cb27be8443fa37e514f9b3825a6 has no consensus state
I20251211 20:52:44.495229 1810 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Sending DeleteTablet for 3 replicas of tablet ca89f09cc0954389b679e8bb8b5e4dfb
I20251211 20:52:44.496587 1015 tablet_service.cc:1552] Processing DeleteTablet for tablet ca89f09cc0954389b679e8bb8b5e4dfb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:44.500929 1168 ts_tablet_manager.cc:1434] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.053s user 0.000s sys 0.005s
I20251211 20:52:44.502395 1168 tablet_bootstrap.cc:492] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:44.503947 2666 raft_consensus.cc:493] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.504174 2666 raft_consensus.cc:515] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.505486 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eb9da081cb9d43fd9a35c4bc51ea133e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:44.506054 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eb9da081cb9d43fd9a35c4bc51ea133e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
W20251211 20:52:44.506230 711 leader_election.cc:343] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:44.506287 1047 raft_consensus.cc:2468] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:44.506696 710 leader_election.cc:304] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:44.506814 1166 tablet_bootstrap.cc:492] T 95866de91f9d437d80f40b7962253610 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:44.506949 1166 ts_tablet_manager.cc:1403] T 95866de91f9d437d80f40b7962253610 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.049s user 0.004s sys 0.000s
I20251211 20:52:44.507779 1166 raft_consensus.cc:359] T 95866de91f9d437d80f40b7962253610 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.509682 1166 raft_consensus.cc:385] T 95866de91f9d437d80f40b7962253610 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.509815 1166 raft_consensus.cc:740] T 95866de91f9d437d80f40b7962253610 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.510104 1166 consensus_queue.cc:260] T 95866de91f9d437d80f40b7962253610 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.510798 1168 tablet_bootstrap.cc:654] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:44.511426 2667 raft_consensus.cc:493] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.513199 2667 raft_consensus.cc:515] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.514130 2667 leader_election.cc:290] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.515416 1166 ts_tablet_manager.cc:1434] T 95866de91f9d437d80f40b7962253610 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20251211 20:52:44.515957 2667 raft_consensus.cc:2804] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:44.516335 1166 tablet_bootstrap.cc:492] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:44.516587 2667 raft_consensus.cc:493] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:44.517012 2667 raft_consensus.cc:3060] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.518857 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4a39870c7eea4a379b284ea2e5ddf760" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:44.519093 1166 tablet_bootstrap.cc:654] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:44.519106 1047 raft_consensus.cc:2468] T 4a39870c7eea4a379b284ea2e5ddf760 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:44.523370 1810 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_870517d846e84832912515fdcf0d8b64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:44.524286 1810 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_870517d846e84832912515fdcf0d8b64 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:44.525120 1797 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5e47af616e1b4d2db3780c1d67cdcb47 consensus state has no leader
I20251211 20:52:44.525619 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4a39870c7eea4a379b284ea2e5ddf760" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:44.520619 710 leader_election.cc:304] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
W20251211 20:52:44.531863 711 leader_election.cc:343] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:44.521517 1168 tablet_bootstrap.cc:492] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:44.530158 2666 leader_election.cc:290] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.537235 1168 ts_tablet_manager.cc:1403] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.035s user 0.004s sys 0.000s
I20251211 20:52:44.537847 2666 raft_consensus.cc:2804] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:44.538532 2666 raft_consensus.cc:493] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:44.538693 2666 raft_consensus.cc:3060] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.541723 2674 raft_consensus.cc:493] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.541934 2674 raft_consensus.cc:515] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.542632 2674 leader_election.cc:290] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.542824 2674 raft_consensus.cc:493] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.542954 2674 raft_consensus.cc:515] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.543452 2674 leader_election.cc:290] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.544188 2668 tablet_replica.cc:333] T c7bd218b3ff340b2a589c9517b49a649 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:44.544428 2668 raft_consensus.cc:2243] T c7bd218b3ff340b2a589c9517b49a649 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:44.544665 2668 raft_consensus.cc:2272] T c7bd218b3ff340b2a589c9517b49a649 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:44.545295 2668 ts_tablet_manager.cc:1916] T c7bd218b3ff340b2a589c9517b49a649 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:44.550314 894 tablet_service.cc:1552] Processing DeleteTablet for tablet ca89f09cc0954389b679e8bb8b5e4dfb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:44.551077 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f5916a4d9b24d7b8db3fcf028bbe3c9" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:44.552271 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68792ccef54944a09af9b5456e9ff1fe" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:44.553018 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f5916a4d9b24d7b8db3fcf028bbe3c9" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:44.553272 774 raft_consensus.cc:2468] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 0.
I20251211 20:52:44.553381 1810 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_0514969896e648faa4dc0327ecaf8993"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:44.553787 847 leader_election.cc:343] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 68792ccef54944a09af9b5456e9ff1fe
I20251211 20:52:44.553748 1168 raft_consensus.cc:359] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:44.554160 1810 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0514969896e648faa4dc0327ecaf8993 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.554697 847 leader_election.cc:304] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:44.555099 1168 raft_consensus.cc:385] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.555125 847 leader_election.cc:304] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters: e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:44.555195 1168 raft_consensus.cc:740] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.555557 2670 raft_consensus.cc:2749] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:44.555748 2670 raft_consensus.cc:2804] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:44.555939 2670 raft_consensus.cc:493] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:44.555912 1168 consensus_queue.cc:260] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.556103 2670 raft_consensus.cc:3060] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.556499 1796 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_4585c080cc0f44a3af3d60f96f928eb2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:44.556707 2673 tablet.cc:1722] T bfb52935296443c58190d4d06e2e6e9c P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 7
W20251211 20:52:44.557622 1796 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4585c080cc0f44a3af3d60f96f928eb2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.562276 1168 ts_tablet_manager.cc:1434] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.019s user 0.002s sys 0.000s
I20251211 20:52:44.567759 1168 tablet_bootstrap.cc:492] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:44.570063 1810 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_9386ee8bce224f50bc410ab74a565d56"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:44.570323 1797 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_acd9171c49ff48d2a7ed21684903193b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:44.570724 1810 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9386ee8bce224f50bc410ab74a565d56 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:44.570724 1797 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_acd9171c49ff48d2a7ed21684903193b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.573057 1809 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_8ead90e9bdef45bf8bf6c5d4c26c347a" } new_table_name: "default.Table_3ac4746e17264f5a93890f3600505781" modify_external_catalogs: true
W20251211 20:52:44.573457 2616 log.cc:927] Time spent T 861d2d70a24946699dccd65cab05195e P e715a811a82446ce8d5bb4aafd5b81c7: Append to log took a long time: real 0.370s user 0.000s sys 0.000s
I20251211 20:52:44.578286 2674 raft_consensus.cc:493] T a3b1aadf17ea4bf0a896d23cd543fca7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.578498 2674 raft_consensus.cc:515] T a3b1aadf17ea4bf0a896d23cd543fca7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.579139 2674 leader_election.cc:290] T a3b1aadf17ea4bf0a896d23cd543fca7 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:44.579252 777 consensus_meta.cc:287] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919: Time spent flushing consensus metadata: real 0.888s user 0.003s sys 0.000s
I20251211 20:52:44.580608 2682 tablet.cc:1722] T bfb52935296443c58190d4d06e2e6e9c P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 7
I20251211 20:52:44.580915 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a3b1aadf17ea4bf0a896d23cd543fca7" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:44.582273 1166 tablet_bootstrap.cc:492] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:44.584540 1166 ts_tablet_manager.cc:1403] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.068s user 0.004s sys 0.002s
I20251211 20:52:44.587622 1166 raft_consensus.cc:359] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.590229 1166 raft_consensus.cc:385] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.590719 1166 raft_consensus.cc:740] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.591944 1166 consensus_queue.cc:260] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.593762 1166 ts_tablet_manager.cc:1434] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20251211 20:52:44.593919 2601 raft_consensus.cc:493] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.595350 2601 raft_consensus.cc:515] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.595964 2601 leader_election.cc:290] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:44.596191 739 tablet_service.cc:1552] Processing DeleteTablet for tablet ca89f09cc0954389b679e8bb8b5e4dfb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:44.599962 2662 ts_tablet_manager.cc:1929] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:44.600075 2662 log.cc:1199] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/c7bd218b3ff340b2a589c9517b49a649
I20251211 20:52:44.600391 2662 ts_tablet_manager.cc:1950] T c7bd218b3ff340b2a589c9517b49a649 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:44.604578 2672 tablet.cc:1722] T bfb52935296443c58190d4d06e2e6e9c P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 7
I20251211 20:52:44.604662 2681 consensus_queue.cc:1048] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20251211 20:52:44.604611 777 raft_consensus.cc:2468] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 1.
I20251211 20:52:44.605162 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 9b99860ee6be45a0b358e686332d3137 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:44.605289 1166 tablet_bootstrap.cc:492] T 44e2d2bdecf247baaac26e06bccc94de P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:44.606101 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1aeae8c18b6499e9dc1accabdc3f633" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:44.607156 847 leader_election.cc:304] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters: e715a811a82446ce8d5bb4aafd5b81c7
W20251211 20:52:44.607625 982 leader_election.cc:343] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:44.608145 1799 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_4b288130d07345eab081b9d7ccea6c2b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:44.608806 1799 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4b288130d07345eab081b9d7ccea6c2b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.610108 1808 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_23402fab961249179b57df4d79b53a18" } new_table_name: "default.Table_0d7df3fba38849a49b65d8290812b95c" modify_external_catalogs: true
I20251211 20:52:44.611081 1166 tablet_bootstrap.cc:654] T 44e2d2bdecf247baaac26e06bccc94de P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:44.611682 1796 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_171c2a61012b444caf720781072205a3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:44.612381 1796 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_171c2a61012b444caf720781072205a3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.608978 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a3b1aadf17ea4bf0a896d23cd543fca7" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:44.634445 2684 raft_consensus.cc:493] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.636826 1015 tablet_service.cc:1505] Processing CreateTablet for tablet 728b4d34376b4f00b5556adc25b46315 (DEFAULT_TABLE table=default.Table_84d90fc27f874a01bf3187d6d02b2053 [id=fa858cefc5bc458ca620f77a25b697a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.637027 2684 raft_consensus.cc:515] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.637701 2684 leader_election.cc:290] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.637864 2684 raft_consensus.cc:2804] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:44.637966 2684 raft_consensus.cc:697] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41 [term 1 LEADER]: Becoming Leader. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Running, Role: LEADER
I20251211 20:52:44.638231 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 5b5fb5f22c824fd6af16db188cf32e45 (DEFAULT_TABLE table=default.Table_a32f150ce5604ec999ac5f3edc6324b4 [id=f98e1455f27a4c98a9c484931ed502b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.638226 2684 consensus_queue.cc:237] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.640360 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 7e24d782602f4a639a60c797a7c92b5b (DEFAULT_TABLE table=default.Table_ca4526eee78f4811a98c7a64f76431d8 [id=eb61893aa74349e79064946f7d8334a6]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:44.641728 771 consensus_meta.cc:287] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919: Time spent flushing consensus metadata: real 0.850s user 0.002s sys 0.001s
I20251211 20:52:44.641873 771 raft_consensus.cc:2468] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 1.
W20251211 20:52:44.644757 914 consensus_meta.cc:287] T 28ddb8a59a214576a25d613d42c2fef5 P eb0f1933d9a74fccaa4618047fc05c41: Time spent flushing consensus metadata: real 0.716s user 0.003s sys 0.000s
I20251211 20:52:44.646456 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 5d225553d0984b6e9685b5841616c9d8 (DEFAULT_TABLE table=default.Table_25d3985a897841b8b55ad7c44bbff154 [id=fde40e5a01ee40c48ca653928036c9a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.647208 756 tablet_service.cc:1505] Processing CreateTablet for tablet 728b4d34376b4f00b5556adc25b46315 (DEFAULT_TABLE table=default.Table_84d90fc27f874a01bf3187d6d02b2053 [id=fa858cefc5bc458ca620f77a25b697a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.647742 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 728b4d34376b4f00b5556adc25b46315. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:44.647811 910 consensus_meta.cc:287] T 645b08b33676420c96e880de3d460829 P eb0f1933d9a74fccaa4618047fc05c41: Time spent flushing consensus metadata: real 0.592s user 0.003s sys 0.000s
I20251211 20:52:44.647892 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 728b4d34376b4f00b5556adc25b46315. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:44.650770 2460 consensus_meta.cc:287] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41: Time spent flushing consensus metadata: real 0.561s user 0.000s sys 0.000s
I20251211 20:52:44.650887 2460 raft_consensus.cc:515] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.651548 2686 tablet.cc:1722] T 861d2d70a24946699dccd65cab05195e P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:44.651727 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b5fb5f22c824fd6af16db188cf32e45. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:44.652695 847 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.1:42177, user_credentials={real_user=slave}} blocked reactor thread for 45577us
I20251211 20:52:44.653842 2462 raft_consensus.cc:515] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.654376 2462 leader_election.cc:290] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Requested vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:44.654692 847 leader_election.cc:343] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 7ec261837cf04bf88a48ff8add1b0d66
I20251211 20:52:44.655028 847 leader_election.cc:304] T a3b1aadf17ea4bf0a896d23cd543fca7 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:44.655355 847 leader_election.cc:304] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters: e715a811a82446ce8d5bb4aafd5b81c7
W20251211 20:52:44.656653 2501 consensus_meta.cc:287] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919: Time spent flushing consensus metadata: real 0.841s user 0.002s sys 0.000s
I20251211 20:52:44.656144 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b1ce6d2a12a4c7ea044b349412b4bf5" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:44.656767 2501 raft_consensus.cc:515] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.657099 2674 raft_consensus.cc:2749] T a3b1aadf17ea4bf0a896d23cd543fca7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:44.657285 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7ec261837cf04bf88a48ff8add1b0d66" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:44.657451 2501 leader_election.cc:290] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.657492 772 raft_consensus.cc:3060] T 7ec261837cf04bf88a48ff8add1b0d66 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.657246 2674 raft_consensus.cc:2804] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:44.657608 2674 raft_consensus.cc:697] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 LEADER]: Becoming Leader. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Running, Role: LEADER
I20251211 20:52:44.657631 1810 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_bcb5c15cfcc2448a9f814e5ff5665f93" } new_table_name: "default.Table_bf827b216f1b43c196923dc1828ff0d0" modify_external_catalogs: true
I20251211 20:52:44.657953 2674 consensus_queue.cc:237] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:44.659569 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 58198us
W20251211 20:52:44.659893 2500 consensus_meta.cc:287] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919: Time spent flushing consensus metadata: real 0.809s user 0.002s sys 0.000s
I20251211 20:52:44.660023 2500 raft_consensus.cc:515] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.660576 2500 leader_election.cc:290] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:44.660912 2273 consensus_meta.cc:287] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919: Time spent flushing consensus metadata: real 1.195s user 0.000s sys 0.003s
I20251211 20:52:44.661010 2273 raft_consensus.cc:515] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.661550 2273 leader_election.cc:290] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.661491 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d37a7d0d662e45079b0cfc02765d2e01" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:44.661665 1044 raft_consensus.cc:3060] T d37a7d0d662e45079b0cfc02765d2e01 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.661923 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 964) took 3609 ms. Trace:
I20251211 20:52:44.662061 1165 rpcz_store.cc:276] 1211 20:52:41.052857 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:41.054838 (+ 1981us) service_pool.cc:225] Handling call
1211 20:52:43.805466 (+2750628us) ts_tablet_manager.cc:1151] Deleting tablet 2630ad57845a4c31b20c8a104dd27335
1211 20:52:43.805475 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:43.886306 (+ 80831us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:43.981612 (+ 95306us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:44.661896 (+680284us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2750448}
I20251211 20:52:44.662398 1165 tablet_replica.cc:333] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:44.662416 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b7dd8232d974a43b99ec9b28075dc8d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:44.662554 1048 raft_consensus.cc:3060] T 6b7dd8232d974a43b99ec9b28075dc8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:44.667402 2560 consensus_meta.cc:287] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919: Time spent flushing consensus metadata: real 0.631s user 0.000s sys 0.000s
I20251211 20:52:44.668745 2629 raft_consensus.cc:515] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.669309 2629 leader_election.cc:290] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Requested vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:44.670282 2494 consensus_meta.cc:287] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919: Time spent flushing consensus metadata: real 0.564s user 0.000s sys 0.000s
I20251211 20:52:44.670264 1043 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "04086eaa32a74edd84235d3b28f9f965" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:44.670423 1043 raft_consensus.cc:3060] T 04086eaa32a74edd84235d3b28f9f965 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.670409 2494 raft_consensus.cc:515] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.671042 2494 leader_election.cc:290] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.672574 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "215f8dc74a71482daa2ca80e15010f51" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:44.672744 1047 raft_consensus.cc:3060] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:44.653843 909 consensus_meta.cc:287] T 22273287aa9245a2a66131fed2c8861c P eb0f1933d9a74fccaa4618047fc05c41: Time spent flushing consensus metadata: real 0.516s user 0.002s sys 0.000s
W20251211 20:52:44.673588 2499 consensus_meta.cc:287] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919: Time spent flushing consensus metadata: real 0.556s user 0.000s sys 0.000s
I20251211 20:52:44.673698 2499 raft_consensus.cc:515] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.673755 2682 tablet.cc:1722] T 861d2d70a24946699dccd65cab05195e P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:44.674263 2499 leader_election.cc:290] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.674582 1797 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_7e2c82ae1dbe44078774e5229684130d" } modify_external_catalogs: true
I20251211 20:52:44.674796 1797 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_7e2c82ae1dbe44078774e5229684130d" } modify_external_catalogs: true
I20251211 20:52:44.675850 772 raft_consensus.cc:2468] T 7ec261837cf04bf88a48ff8add1b0d66 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 1.
I20251211 20:52:44.676208 2494 consensus_queue.cc:1048] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251211 20:52:44.676643 2561 consensus_meta.cc:287] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919: Time spent flushing consensus metadata: real 0.548s user 0.002s sys 0.000s
I20251211 20:52:44.676771 2561 raft_consensus.cc:515] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.677347 2561 leader_election.cc:290] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.677386 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c91ff29847d54e55b19321652a857e9f" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:44.677536 1042 raft_consensus.cc:3060] T c91ff29847d54e55b19321652a857e9f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.679895 2429 raft_consensus.cc:515] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.680498 2429 leader_election.cc:290] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:44.680502 1809 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ff4d5ec1f87247e896f9c099c532265a consensus state has no leader
I20251211 20:52:44.683012 2286 raft_consensus.cc:515] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.684461 1809 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_ff2dfead5c37456d91374a4bc62c85ea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:44.685036 1809 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff2dfead5c37456d91374a4bc62c85ea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.686041 2325 raft_consensus.cc:515] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.686709 2325 leader_election.cc:290] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:44.686996 1045 consensus_meta.cc:287] T 2e7e158809b9471fbd6e6ee9fb9ac569 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent flushing consensus metadata: real 0.600s user 0.003s sys 0.000s
I20251211 20:52:44.687100 1045 raft_consensus.cc:2468] T 2e7e158809b9471fbd6e6ee9fb9ac569 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:44.687798 2662 tablet_replica.cc:333] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:44.687954 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet c7bd218b3ff340b2a589c9517b49a649 (table default.Table_233d5484bd4e4456aed925bc271edfa5 [id=ca0c3c6978874d93a04b57cc0c425c83]) successfully deleted
I20251211 20:52:44.688000 2662 raft_consensus.cc:2243] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:44.688154 2662 raft_consensus.cc:2272] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:44.688863 713 connection.cc:441] server connection from 127.0.0.1:37886 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 964) could send its response
I20251211 20:52:44.689324 2601 raft_consensus.cc:493] T 18b977f463d5420186c007b38110e07e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.689445 2601 raft_consensus.cc:515] T 18b977f463d5420186c007b38110e07e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.690014 2601 leader_election.cc:290] T 18b977f463d5420186c007b38110e07e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:44.692160 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4ce495233a5e4f3792faf44680df54ba" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:44.692354 774 raft_consensus.cc:3060] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.695135 774 raft_consensus.cc:2468] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 1.
I20251211 20:52:44.695729 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e24d782602f4a639a60c797a7c92b5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.697536 739 tablet_service.cc:1505] Processing CreateTablet for tablet 7e24d782602f4a639a60c797a7c92b5b (DEFAULT_TABLE table=default.Table_ca4526eee78f4811a98c7a64f76431d8 [id=eb61893aa74349e79064946f7d8334a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.697957 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e24d782602f4a639a60c797a7c92b5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.698781 710 leader_election.cc:304] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:44.699090 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d225553d0984b6e9685b5841616c9d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.702245 1799 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_280921729cb94c62b206fc68ef348c61"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:44.702869 1799 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_280921729cb94c62b206fc68ef348c61 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.703023 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b1ce6d2a12a4c7ea044b349412b4bf5" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:44.703997 2325 raft_consensus.cc:2804] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:44.704157 2325 raft_consensus.cc:697] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:44.704222 1806 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_c64075b948a646458740b55e6970fd94" } new_table_name: "default.Table_af0df4d1ee144455a6041e5a6a0b9b80" modify_external_catalogs: true
I20251211 20:52:44.704416 2325 consensus_queue.cc:237] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.704911 2662 ts_tablet_manager.cc:1916] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:44.703948 756 tablet_service.cc:1505] Processing CreateTablet for tablet 5b5fb5f22c824fd6af16db188cf32e45 (DEFAULT_TABLE table=default.Table_a32f150ce5604ec999ac5f3edc6324b4 [id=f98e1455f27a4c98a9c484931ed502b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.705911 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b5fb5f22c824fd6af16db188cf32e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.709277 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2207af105bc3471ca930d35a91d38e70" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:44.709487 771 raft_consensus.cc:3060] T 2207af105bc3471ca930d35a91d38e70 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.710448 747 tablet_service.cc:1505] Processing CreateTablet for tablet e43140b127cc462096008617ab8e6253 (DEFAULT_TABLE table=default.Table_60f0bdc6a1c849fd851ca0d31eeeaf79 [id=e6d99593c9aa45c7bca5a782bad9c904]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.710856 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e43140b127cc462096008617ab8e6253. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:44.711261 1808 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3c865c99e9314f5f91658b9126da4d39 consensus state has no leader
I20251211 20:52:44.712601 771 raft_consensus.cc:2468] T 2207af105bc3471ca930d35a91d38e70 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 1.
W20251211 20:52:44.714056 2338 consensus_meta.cc:287] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919: Time spent flushing consensus metadata: real 0.910s user 0.002s sys 0.000s
I20251211 20:52:44.714210 2338 raft_consensus.cc:515] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.714458 741 tablet_service.cc:1505] Processing CreateTablet for tablet 84cca3ee8fc446fcaf9971171235cf55 (DEFAULT_TABLE table=default.Table_68f5e21962ce48b89834785bf8dd332f [id=df1df827d2194032b668c7579bc1dcda]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.714841 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84cca3ee8fc446fcaf9971171235cf55. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.715343 1808 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_c73d4009f4bc41b38aee5612313b30c8" } modify_external_catalogs: true
I20251211 20:52:44.715528 1808 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_c73d4009f4bc41b38aee5612313b30c8" } modify_external_catalogs: true
I20251211 20:52:44.719686 1044 raft_consensus.cc:2468] T d37a7d0d662e45079b0cfc02765d2e01 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:44.722124 1048 raft_consensus.cc:2468] T 6b7dd8232d974a43b99ec9b28075dc8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:44.722548 2338 leader_election.cc:290] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.723899 739 tablet_service.cc:1505] Processing CreateTablet for tablet 5d225553d0984b6e9685b5841616c9d8 (DEFAULT_TABLE table=default.Table_25d3985a897841b8b55ad7c44bbff154 [id=fde40e5a01ee40c48ca653928036c9a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.724287 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d225553d0984b6e9685b5841616c9d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.724506 1043 raft_consensus.cc:2468] T 04086eaa32a74edd84235d3b28f9f965 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:44.726848 1047 raft_consensus.cc:2468] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:44.727679 1042 raft_consensus.cc:2468] T c91ff29847d54e55b19321652a857e9f P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:44.667517 2560 raft_consensus.cc:515] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.728370 2560 leader_election.cc:290] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.730746 1796 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_19882e719bca4562a2fa3ff1f6e98590" } new_table_name: "default.Table_4e1f03b9d87b4fe59dbcb3a947ee299d" modify_external_catalogs: true
I20251211 20:52:44.733409 2286 leader_election.cc:290] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.651523 2460 leader_election.cc:290] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Requested vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.736694 2647 raft_consensus.cc:493] T 645b08b33676420c96e880de3d460829 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6f8d784fd1cd492bbe3809403ec34919)
I20251211 20:52:44.736878 2647 raft_consensus.cc:515] T 645b08b33676420c96e880de3d460829 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.737142 2668 ts_tablet_manager.cc:1929] T c7bd218b3ff340b2a589c9517b49a649 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:44.737241 2668 log.cc:1199] T c7bd218b3ff340b2a589c9517b49a649 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/c7bd218b3ff340b2a589c9517b49a649
I20251211 20:52:44.737557 2668 ts_tablet_manager.cc:1950] T c7bd218b3ff340b2a589c9517b49a649 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:44.737744 2647 leader_election.cc:290] T 645b08b33676420c96e880de3d460829 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.741487 1160 tablet_bootstrap.cc:492] T 861d2d70a24946699dccd65cab05195e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:44.741456 1798 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Sending DeleteTablet for 3 replicas of tablet 06ba98a726424dd89450053c5c5793f4
I20251211 20:52:44.741779 1783 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet c7bd218b3ff340b2a589c9517b49a649 (table default.Table_233d5484bd4e4456aed925bc271edfa5 [id=ca0c3c6978874d93a04b57cc0c425c83]) successfully deleted
W20251211 20:52:44.742473 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 9b99860ee6be45a0b358e686332d3137 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9b99860ee6be45a0b358e686332d3137 already in progress: creating tablet
I20251211 20:52:44.742883 1160 tablet_bootstrap.cc:654] T 861d2d70a24946699dccd65cab05195e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:44.742890 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet ca89f09cc0954389b679e8bb8b5e4dfb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ca89f09cc0954389b679e8bb8b5e4dfb already in progress: creating tablet
I20251211 20:52:44.743558 894 tablet_service.cc:1505] Processing CreateTablet for tablet 728b4d34376b4f00b5556adc25b46315 (DEFAULT_TABLE table=default.Table_84d90fc27f874a01bf3187d6d02b2053 [id=fa858cefc5bc458ca620f77a25b697a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.743760 1798 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_e133e518b951424cba3117dfdf77d480"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:44.743969 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 728b4d34376b4f00b5556adc25b46315. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:44.744364 1798 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e133e518b951424cba3117dfdf77d480 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.744588 2666 raft_consensus.cc:515] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.745253 2666 leader_election.cc:290] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.750907 1160 tablet_bootstrap.cc:492] T 861d2d70a24946699dccd65cab05195e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:44.751056 1160 ts_tablet_manager.cc:1403] T 861d2d70a24946699dccd65cab05195e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.001s
I20251211 20:52:44.751747 1160 raft_consensus.cc:359] T 861d2d70a24946699dccd65cab05195e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.751936 1160 raft_consensus.cc:385] T 861d2d70a24946699dccd65cab05195e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.751994 1160 raft_consensus.cc:740] T 861d2d70a24946699dccd65cab05195e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.752161 1160 consensus_queue.cc:260] T 861d2d70a24946699dccd65cab05195e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.752454 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 06ba98a726424dd89450053c5c5793f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:44.753378 1802 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Sending DeleteTablet for 3 replicas of tablet 516f7181aaac48cdadf9fbc08d0d3f57
I20251211 20:52:44.754973 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 84cca3ee8fc446fcaf9971171235cf55 (DEFAULT_TABLE table=default.Table_68f5e21962ce48b89834785bf8dd332f [id=df1df827d2194032b668c7579bc1dcda]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.755226 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 516f7181aaac48cdadf9fbc08d0d3f57 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:44.755052 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 06ba98a726424dd89450053c5c5793f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:44.755477 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84cca3ee8fc446fcaf9971171235cf55. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.755409 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "18b977f463d5420186c007b38110e07e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:44.755550 1802 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_7759f916c14543c38f2a391980a1b38b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:44.755954 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 4d03f2f064c3420d817a46e4153054e8 (DEFAULT_TABLE table=default.Table_4ec0d87202fc419380c2542d827087de [id=d9829c79b55844ec92b1b0cf594ab96b]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:44.756297 1802 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7759f916c14543c38f2a391980a1b38b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.756415 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d03f2f064c3420d817a46e4153054e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.757112 1160 ts_tablet_manager.cc:1434] T 861d2d70a24946699dccd65cab05195e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20251211 20:52:44.757465 1160 tablet_bootstrap.cc:492] T 2e7e158809b9471fbd6e6ee9fb9ac569 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:44.759759 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0763a475527546bb83cee8fb823893ac" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:44.759943 1042 raft_consensus.cc:3060] T 0763a475527546bb83cee8fb823893ac P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.760382 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e484d499af1b4947adb36c32f8ea5704" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:44.760521 1047 raft_consensus.cc:3060] T e484d499af1b4947adb36c32f8ea5704 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.762310 1042 raft_consensus.cc:2468] T 0763a475527546bb83cee8fb823893ac P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:44.762594 2666 raft_consensus.cc:493] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.762733 1047 raft_consensus.cc:2468] T e484d499af1b4947adb36c32f8ea5704 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:44.762758 2666 raft_consensus.cc:515] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.763362 2666 leader_election.cc:290] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.763818 1015 tablet_service.cc:1505] Processing CreateTablet for tablet 12d77bf2d30843daab1beb01c396faa5 (DEFAULT_TABLE table=default.Table_af66deaa476a4ea986b403ef120654ff [id=dc94606dc32f4fd0ae8d0885323ab561]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.764176 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12d77bf2d30843daab1beb01c396faa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.765288 1795 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Sending DeleteTablet for 3 replicas of tablet 59e955aa07c14fbbb347b88eca59a74e
I20251211 20:52:44.759025 1160 tablet_bootstrap.cc:654] T 2e7e158809b9471fbd6e6ee9fb9ac569 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:44.767673 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fa79b67b9eee45e2b41c4a90967606cc" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:44.767941 1047 raft_consensus.cc:3060] T fa79b67b9eee45e2b41c4a90967606cc P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.768119 1795 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_d424820106284c94adee000401d8ab22"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:44.768729 1795 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d424820106284c94adee000401d8ab22 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.769094 747 tablet_service.cc:1505] Processing CreateTablet for tablet 4d03f2f064c3420d817a46e4153054e8 (DEFAULT_TABLE table=default.Table_4ec0d87202fc419380c2542d827087de [id=d9829c79b55844ec92b1b0cf594ab96b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.769455 1043 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3562b5effa124378beac7ccf0d27ee7e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:44.769634 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d03f2f064c3420d817a46e4153054e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.769647 1043 raft_consensus.cc:3060] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.770272 1160 tablet_bootstrap.cc:492] T 2e7e158809b9471fbd6e6ee9fb9ac569 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:44.770399 1160 ts_tablet_manager.cc:1403] T 2e7e158809b9471fbd6e6ee9fb9ac569 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.001s
I20251211 20:52:44.771054 1160 raft_consensus.cc:359] T 2e7e158809b9471fbd6e6ee9fb9ac569 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.771206 1160 raft_consensus.cc:385] T 2e7e158809b9471fbd6e6ee9fb9ac569 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.771255 1160 raft_consensus.cc:740] T 2e7e158809b9471fbd6e6ee9fb9ac569 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.771431 1160 consensus_queue.cc:260] T 2e7e158809b9471fbd6e6ee9fb9ac569 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.773978 1047 raft_consensus.cc:2468] T fa79b67b9eee45e2b41c4a90967606cc P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
W20251211 20:52:44.774588 1810 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0302cc8613934c43a0650f081cf98797 consensus state has no leader
I20251211 20:52:44.775506 741 tablet_service.cc:1505] Processing CreateTablet for tablet 12d77bf2d30843daab1beb01c396faa5 (DEFAULT_TABLE table=default.Table_af66deaa476a4ea986b403ef120654ff [id=dc94606dc32f4fd0ae8d0885323ab561]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.769140 2667 raft_consensus.cc:515] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.775954 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12d77bf2d30843daab1beb01c396faa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.777455 1160 ts_tablet_manager.cc:1434] T 2e7e158809b9471fbd6e6ee9fb9ac569 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20251211 20:52:44.777730 1160 tablet_bootstrap.cc:492] T fed300d31c414f4a9b27c66e78097688 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:44.779090 2667 leader_election.cc:290] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.779255 1160 tablet_bootstrap.cc:654] T fed300d31c414f4a9b27c66e78097688 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:44.775672 2699 raft_consensus.cc:493] T f25bde14792248b188660036f07ae902 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.777022 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 59e955aa07c14fbbb347b88eca59a74e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:44.777539 2662 ts_tablet_manager.cc:1929] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:44.784055 2699 raft_consensus.cc:515] T f25bde14792248b188660036f07ae902 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.784171 2662 log.cc:1199] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/2e33606930774fe4aafc6e3819618af8
I20251211 20:52:44.784547 2662 ts_tablet_manager.cc:1950] T 2e33606930774fe4aafc6e3819618af8 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:44.784679 2699 leader_election.cc:290] T f25bde14792248b188660036f07ae902 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:44.785100 1043 raft_consensus.cc:2468] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:44.780356 1019 tablet_service.cc:1505] Processing CreateTablet for tablet e43140b127cc462096008617ab8e6253 (DEFAULT_TABLE table=default.Table_60f0bdc6a1c849fd851ca0d31eeeaf79 [id=e6d99593c9aa45c7bca5a782bad9c904]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.786144 1168 tablet_bootstrap.cc:654] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:44.786638 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e43140b127cc462096008617ab8e6253. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.787302 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 2e33606930774fe4aafc6e3819618af8 (table default.Table_290ed4f01870438ca11f93a98c4ce8e8 [id=9e23a3e5095f4a358fe7859424c27d59]) successfully deleted
I20251211 20:52:44.787911 881 tablet_service.cc:1505] Processing CreateTablet for tablet 5d225553d0984b6e9685b5841616c9d8 (DEFAULT_TABLE table=default.Table_25d3985a897841b8b55ad7c44bbff154 [id=fde40e5a01ee40c48ca653928036c9a1]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:44.788257 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 9b99860ee6be45a0b358e686332d3137 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9b99860ee6be45a0b358e686332d3137 already in progress: creating tablet
I20251211 20:52:44.788318 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d225553d0984b6e9685b5841616c9d8. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:44.788739 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet ca89f09cc0954389b679e8bb8b5e4dfb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ca89f09cc0954389b679e8bb8b5e4dfb already in progress: creating tablet
W20251211 20:52:44.789145 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 516f7181aaac48cdadf9fbc08d0d3f57 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 516f7181aaac48cdadf9fbc08d0d3f57 already in progress: creating tablet
W20251211 20:52:44.789577 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 06ba98a726424dd89450053c5c5793f4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06ba98a726424dd89450053c5c5793f4 already in progress: creating tablet
W20251211 20:52:44.789909 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 59e955aa07c14fbbb347b88eca59a74e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 59e955aa07c14fbbb347b88eca59a74e already in progress: creating tablet
I20251211 20:52:44.787914 883 tablet_service.cc:1505] Processing CreateTablet for tablet 7e24d782602f4a639a60c797a7c92b5b (DEFAULT_TABLE table=default.Table_ca4526eee78f4811a98c7a64f76431d8 [id=eb61893aa74349e79064946f7d8334a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.792029 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e24d782602f4a639a60c797a7c92b5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.796823 1160 tablet_bootstrap.cc:492] T fed300d31c414f4a9b27c66e78097688 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:44.796963 1160 ts_tablet_manager.cc:1403] T fed300d31c414f4a9b27c66e78097688 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.000s
I20251211 20:52:44.797093 1797 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Sending DeleteTablet for 3 replicas of tablet 683d0f2068fb4a6ebc9c169ad7b455ce
I20251211 20:52:44.797930 2647 raft_consensus.cc:493] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.798038 2647 raft_consensus.cc:515] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.798571 2647 leader_election.cc:290] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.800024 910 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d37a7d0d662e45079b0cfc02765d2e01" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:44.800488 894 tablet_service.cc:1505] Processing CreateTablet for tablet 5b5fb5f22c824fd6af16db188cf32e45 (DEFAULT_TABLE table=default.Table_a32f150ce5604ec999ac5f3edc6324b4 [id=f98e1455f27a4c98a9c484931ed502b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.800623 910 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b7dd8232d974a43b99ec9b28075dc8d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:44.800982 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b5fb5f22c824fd6af16db188cf32e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.801167 910 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "04086eaa32a74edd84235d3b28f9f965" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:44.802603 2699 raft_consensus.cc:493] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.802743 2699 raft_consensus.cc:515] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.802959 1797 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_7b6d7c6f71514a118834aa645c8405e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:44.803458 2699 leader_election.cc:290] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
W20251211 20:52:44.803560 1797 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7b6d7c6f71514a118834aa645c8405e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.691464 2670 raft_consensus.cc:515] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.803933 910 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "215f8dc74a71482daa2ca80e15010f51" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:44.804616 908 raft_consensus.cc:3060] T a3b1aadf17ea4bf0a896d23cd543fca7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.806718 886 tablet_service.cc:1505] Processing CreateTablet for tablet 12d77bf2d30843daab1beb01c396faa5 (DEFAULT_TABLE table=default.Table_af66deaa476a4ea986b403ef120654ff [id=dc94606dc32f4fd0ae8d0885323ab561]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.807071 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12d77bf2d30843daab1beb01c396faa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.813231 881 tablet_service.cc:1505] Processing CreateTablet for tablet 84cca3ee8fc446fcaf9971171235cf55 (DEFAULT_TABLE table=default.Table_68f5e21962ce48b89834785bf8dd332f [id=df1df827d2194032b668c7579bc1dcda]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.813691 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84cca3ee8fc446fcaf9971171235cf55. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.815764 910 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c91ff29847d54e55b19321652a857e9f" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:44.815901 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0763a475527546bb83cee8fb823893ac" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:44.816195 907 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e484d499af1b4947adb36c32f8ea5704" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:44.817373 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3e6f62c2deff4c11b137fc0cba5ee234" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:44.817376 883 tablet_service.cc:1505] Processing CreateTablet for tablet e43140b127cc462096008617ab8e6253 (DEFAULT_TABLE table=default.Table_60f0bdc6a1c849fd851ca0d31eeeaf79 [id=e6d99593c9aa45c7bca5a782bad9c904]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.817631 1047 raft_consensus.cc:3060] T 3e6f62c2deff4c11b137fc0cba5ee234 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.817750 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e43140b127cc462096008617ab8e6253. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.820648 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "efe4d6bb07374294b8ab50a9fe605f87" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:44.820709 1043 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4a39870c7eea4a379b284ea2e5ddf760" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:44.820740 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b85af79c2574443998e847bc19c53b69" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:44.820844 1042 raft_consensus.cc:3060] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.820876 1043 raft_consensus.cc:3060] T 4a39870c7eea4a379b284ea2e5ddf760 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.823798 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "efe4d6bb07374294b8ab50a9fe605f87" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:44.824375 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fa79b67b9eee45e2b41c4a90967606cc" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:44.804629 2670 leader_election.cc:290] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Requested vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.830771 2626 raft_consensus.cc:493] T 5710fc2f24764f818f7e5c86c6fd985e P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6f8d784fd1cd492bbe3809403ec34919)
W20251211 20:52:44.831028 1806 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d43210ec61af423ebde624397dea9f2e consensus state has no leader
I20251211 20:52:44.830938 2626 raft_consensus.cc:515] T 5710fc2f24764f818f7e5c86c6fd985e P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.831773 2626 leader_election.cc:290] T 5710fc2f24764f818f7e5c86c6fd985e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.838438 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5bcc94ec9630484cb7a3e8bf6d4f378a" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:44.838671 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 683d0f2068fb4a6ebc9c169ad7b455ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:44.839084 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f25bde14792248b188660036f07ae902" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:44.839301 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 516f7181aaac48cdadf9fbc08d0d3f57 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:44.839345 1808 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Sending DeleteTablet for 3 replicas of tablet 41627cb27be8443fa37e514f9b3825a6
I20251211 20:52:44.839567 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 59e955aa07c14fbbb347b88eca59a74e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:44.839960 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 683d0f2068fb4a6ebc9c169ad7b455ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:50110
W20251211 20:52:44.842728 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 516f7181aaac48cdadf9fbc08d0d3f57 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 516f7181aaac48cdadf9fbc08d0d3f57 already in progress: creating tablet
W20251211 20:52:44.842957 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 59e955aa07c14fbbb347b88eca59a74e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 59e955aa07c14fbbb347b88eca59a74e already in progress: creating tablet
W20251211 20:52:44.843099 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 683d0f2068fb4a6ebc9c169ad7b455ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 683d0f2068fb4a6ebc9c169ad7b455ce already in progress: creating tablet
I20251211 20:52:44.846717 1808 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_7f9e421192224c908c84e08e5b787787"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:44.847388 1808 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7f9e421192224c908c84e08e5b787787 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.820889 1046 raft_consensus.cc:2468] T b85af79c2574443998e847bc19c53b69 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:44.848747 1042 raft_consensus.cc:2468] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:44.849572 1165 raft_consensus.cc:2243] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:44.849969 1165 raft_consensus.cc:2272] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:44.850337 1043 raft_consensus.cc:2468] T 4a39870c7eea4a379b284ea2e5ddf760 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:44.850566 1165 ts_tablet_manager.cc:1916] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:44.852206 1047 raft_consensus.cc:2468] T 3e6f62c2deff4c11b137fc0cba5ee234 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
W20251211 20:52:44.853129 1796 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 46a2a070cf6a42fe8bc54a42eb802fbc consensus state has no leader
I20251211 20:52:44.821331 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eb9da081cb9d43fd9a35c4bc51ea133e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:44.855877 1044 raft_consensus.cc:3060] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.857805 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 41627cb27be8443fa37e514f9b3825a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:44.858414 1044 raft_consensus.cc:2468] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
W20251211 20:52:44.858536 1783 catalog_manager.cc:4952] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 41627cb27be8443fa37e514f9b3825a6 because the tablet was not found. No further retry: Not found: Tablet not found: 41627cb27be8443fa37e514f9b3825a6
I20251211 20:52:44.860603 1165 ts_tablet_manager.cc:1929] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251211 20:52:44.860958 1165 log.cc:1199] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/8308ad46551447439a248d20ed1e5922
I20251211 20:52:44.861434 1165 ts_tablet_manager.cc:1950] T 8308ad46551447439a248d20ed1e5922 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
W20251211 20:52:44.865383 982 leader_election.cc:343] T 18b977f463d5420186c007b38110e07e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:44.868686 1806 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_28bfcb62d3324b46af3d8e28ab649bf4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:44.869376 1806 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_28bfcb62d3324b46af3d8e28ab649bf4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.872726 887 tablet_service.cc:1505] Processing CreateTablet for tablet 31e860227d874abf8d97901fa432948a (DEFAULT_TABLE table=default.Table_ebf689834b574c408bb00777fac24c84 [id=d20c23f61583488f8c8104a18ad2461c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.873194 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31e860227d874abf8d97901fa432948a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.867532 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 41627cb27be8443fa37e514f9b3825a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:44.875852 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 965) took 3822 ms. Trace:
I20251211 20:52:44.876055 1165 rpcz_store.cc:276] 1211 20:52:41.052922 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:41.055054 (+ 2132us) service_pool.cc:225] Handling call
1211 20:52:44.662332 (+3607278us) ts_tablet_manager.cc:1151] Deleting tablet 8308ad46551447439a248d20ed1e5922
1211 20:52:44.662341 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:44.853593 (+191252us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:44.860455 (+ 6862us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:44.875763 (+ 15308us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":186918,"tablet-delete.queue_time_us":3607147}
W20251211 20:52:44.876613 713 connection.cc:441] server connection from 127.0.0.1:37886 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 965) could send its response
I20251211 20:52:44.876717 1165 tablet_replica.cc:333] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:44.876931 1165 raft_consensus.cc:2243] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:44.877280 1165 raft_consensus.cc:2272] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:44.877837 1165 ts_tablet_manager.cc:1916] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:44.821605 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 683d0f2068fb4a6ebc9c169ad7b455ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:44.879539 886 tablet_service.cc:1505] Processing CreateTablet for tablet 46a2a070cf6a42fe8bc54a42eb802fbc (DEFAULT_TABLE table=default.Table_4e1f03b9d87b4fe59dbcb3a947ee299d [id=8e66ea58499b40e59294a135324aa527]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.879927 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46a2a070cf6a42fe8bc54a42eb802fbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.880350 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3562b5effa124378beac7ccf0d27ee7e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:44.880836 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3e6f62c2deff4c11b137fc0cba5ee234" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:44.881251 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4a39870c7eea4a379b284ea2e5ddf760" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:44.882587 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b85af79c2574443998e847bc19c53b69" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:44.883033 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eb9da081cb9d43fd9a35c4bc51ea133e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:44.883386 1799 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_8db9acb9d35149fdabcb9a1d24061b46" } new_table_name: "default.Table_25e38d0576684687890bae04bd81d4f4" modify_external_catalogs: true
I20251211 20:52:44.884043 906 raft_consensus.cc:3060] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.884851 887 tablet_service.cc:1505] Processing CreateTablet for tablet ea03b0f5edb84116ae650e76092970fc (DEFAULT_TABLE table=default.Table_e3de355d0a69496c842b4a35daa99b90 [id=c34e32ee8d9f41b68288ced87e28069f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.885185 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea03b0f5edb84116ae650e76092970fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.888584 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "645b08b33676420c96e880de3d460829" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:44.889173 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb4004212441473483a209d23c0800ce" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:44.889374 774 raft_consensus.cc:2468] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 0.
W20251211 20:52:44.891232 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 61042.8us
I20251211 20:52:44.891626 710 leader_election.cc:304] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:44.896025 710 leader_election.cc:304] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:44.899127 2626 raft_consensus.cc:493] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.899271 2626 raft_consensus.cc:515] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.899912 2626 leader_election.cc:290] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.900056 2626 raft_consensus.cc:493] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6f8d784fd1cd492bbe3809403ec34919)
I20251211 20:52:44.900152 2626 raft_consensus.cc:515] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.900616 2626 leader_election.cc:290] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.900472 1809 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_c78768d721bd46a99b00e5b810e654db"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:44.901219 1809 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c78768d721bd46a99b00e5b810e654db in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:44.901366 2626 raft_consensus.cc:493] T 3de5b34b96404cdc9c191fed7f055638 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.901480 2626 raft_consensus.cc:515] T 3de5b34b96404cdc9c191fed7f055638 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.901928 2626 leader_election.cc:290] T 3de5b34b96404cdc9c191fed7f055638 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.910874 1160 raft_consensus.cc:359] T fed300d31c414f4a9b27c66e78097688 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.911214 1160 raft_consensus.cc:385] T fed300d31c414f4a9b27c66e78097688 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.911312 1160 raft_consensus.cc:740] T fed300d31c414f4a9b27c66e78097688 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.911811 1160 consensus_queue.cc:260] T fed300d31c414f4a9b27c66e78097688 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.912542 1160 ts_tablet_manager.cc:1434] T fed300d31c414f4a9b27c66e78097688 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.116s user 0.002s sys 0.000s
I20251211 20:52:44.912917 1160 tablet_bootstrap.cc:492] T f3855ffb51a74101b26fd28429e65bc2 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:44.913519 1810 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_b26cc06e1b0745e08181124e1cbce934" } new_table_name: "default.Table_014b6db9f3294658b965c701384b98bb" modify_external_catalogs: true
I20251211 20:52:44.914435 1015 tablet_service.cc:1552] Processing DeleteTablet for tablet 41627cb27be8443fa37e514f9b3825a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:36220
W20251211 20:52:44.915252 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 683d0f2068fb4a6ebc9c169ad7b455ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 683d0f2068fb4a6ebc9c169ad7b455ce already in progress: creating tablet
I20251211 20:52:44.915915 2666 raft_consensus.cc:2804] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
W20251211 20:52:44.916062 1780 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 41627cb27be8443fa37e514f9b3825a6 because the tablet was not found. No further retry: Not found: Tablet not found: 41627cb27be8443fa37e514f9b3825a6
I20251211 20:52:44.916121 2666 raft_consensus.cc:697] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:44.917212 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5710fc2f24764f818f7e5c86c6fd985e" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:44.916501 2666 consensus_queue.cc:237] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:44.917326 982 leader_election.cc:343] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:44.917507 982 leader_election.cc:343] T f25bde14792248b188660036f07ae902 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:44.918290 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "645b08b33676420c96e880de3d460829" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:44.896386 2675 raft_consensus.cc:2804] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:44.898501 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68792ccef54944a09af9b5456e9ff1fe" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:44.918884 774 raft_consensus.cc:3060] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:44.921137 756 tablet_service.cc:1505] Processing CreateTablet for tablet ea03b0f5edb84116ae650e76092970fc (DEFAULT_TABLE table=default.Table_e3de355d0a69496c842b4a35daa99b90 [id=c34e32ee8d9f41b68288ced87e28069f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.921447 774 raft_consensus.cc:2468] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 1.
I20251211 20:52:44.921569 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea03b0f5edb84116ae650e76092970fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.898837 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5710fc2f24764f818f7e5c86c6fd985e" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:44.922489 2675 raft_consensus.cc:697] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:44.922842 2675 consensus_queue.cc:237] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.903239 2667 raft_consensus.cc:493] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.927487 2667 raft_consensus.cc:515] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.928206 2667 leader_election.cc:290] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.928980 1015 tablet_service.cc:1505] Processing CreateTablet for tablet 31e860227d874abf8d97901fa432948a (DEFAULT_TABLE table=default.Table_ebf689834b574c408bb00777fac24c84 [id=d20c23f61583488f8c8104a18ad2461c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.929683 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31e860227d874abf8d97901fa432948a. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:44.938603 847 leader_election.cc:343] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 4ce495233a5e4f3792faf44680df54ba
W20251211 20:52:44.938920 847 leader_election.cc:343] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 2207af105bc3471ca930d35a91d38e70
W20251211 20:52:44.939191 847 leader_election.cc:343] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: bb4004212441473483a209d23c0800ce
W20251211 20:52:44.939352 847 leader_election.cc:343] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 68792ccef54944a09af9b5456e9ff1fe
I20251211 20:52:44.940668 710 leader_election.cc:304] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:44.941047 2667 raft_consensus.cc:2804] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:44.941275 2667 raft_consensus.cc:697] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:44.941644 2667 consensus_queue.cc:237] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.942967 1166 tablet_bootstrap.cc:492] T 44e2d2bdecf247baaac26e06bccc94de P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:44.943163 1166 ts_tablet_manager.cc:1403] T 44e2d2bdecf247baaac26e06bccc94de P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.340s user 0.005s sys 0.003s
I20251211 20:52:44.943193 710 leader_election.cc:304] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:44.943610 710 leader_election.cc:304] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:44.943876 2325 raft_consensus.cc:2804] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:44.944128 2501 raft_consensus.cc:2804] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:44.944674 2501 raft_consensus.cc:697] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:44.945312 2500 raft_consensus.cc:493] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:44.945448 2500 raft_consensus.cc:515] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.946089 2500 leader_election.cc:290] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:44.944973 2501 consensus_queue.cc:237] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.948632 2325 raft_consensus.cc:697] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:44.948904 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 06ba98a726424dd89450053c5c5793f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:44.948930 2325 consensus_queue.cc:237] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:44.949717 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 06ba98a726424dd89450053c5c5793f4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06ba98a726424dd89450053c5c5793f4 already in progress: creating tablet
I20251211 20:52:44.950940 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 59e955aa07c14fbbb347b88eca59a74e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:44.956394 710 leader_election.cc:304] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:44.956756 2500 raft_consensus.cc:2804] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:44.956954 2500 raft_consensus.cc:697] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:44.957046 883 tablet_service.cc:1505] Processing CreateTablet for tablet 284e34970f634abcb2288c864f73e963 (DEFAULT_TABLE table=default.Table_e250eb3a69ef4e6da3893852edc00469 [id=ba41048fe47d4749b5e084582989eb38]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.957204 2500 consensus_queue.cc:237] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.957468 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 284e34970f634abcb2288c864f73e963. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.959369 910 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee7b93b0b87541d9af6eb9e68b4cea41" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:44.960952 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 284e34970f634abcb2288c864f73e963 (DEFAULT_TABLE table=default.Table_e250eb3a69ef4e6da3893852edc00469 [id=ba41048fe47d4749b5e084582989eb38]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.961144 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 46a2a070cf6a42fe8bc54a42eb802fbc (DEFAULT_TABLE table=default.Table_4e1f03b9d87b4fe59dbcb3a947ee299d [id=8e66ea58499b40e59294a135324aa527]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.961361 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 284e34970f634abcb2288c864f73e963. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.961668 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46a2a070cf6a42fe8bc54a42eb802fbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.962481 1168 tablet_bootstrap.cc:492] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:44.962685 1168 ts_tablet_manager.cc:1403] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.400s user 0.004s sys 0.002s
I20251211 20:52:44.963548 1168 raft_consensus.cc:359] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.963847 1168 raft_consensus.cc:385] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.963968 1168 raft_consensus.cc:740] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.964325 1168 consensus_queue.cc:260] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.968359 905 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1aeae8c18b6499e9dc1accabdc3f633" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:44.959897 910 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4d2b7e2a0abe4592bc806d5ef3f894d8" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:44.969503 881 tablet_service.cc:1505] Processing CreateTablet for tablet 91254cc6ce5946b1bf3069b2202a6a11 (DEFAULT_TABLE table=default.Table_4ecbff2ab6c243f994ef4fae46276e92 [id=b3230934455d40598c6213630dd3108b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.969687 847 leader_election.cc:304] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters: e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:44.969892 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91254cc6ce5946b1bf3069b2202a6a11. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.970132 847 leader_election.cc:304] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters: e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:44.970705 2626 raft_consensus.cc:2804] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:44.970822 2626 raft_consensus.cc:697] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41 [term 1 LEADER]: Becoming Leader. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Running, Role: LEADER
I20251211 20:52:44.951459 1022 tablet_service.cc:1505] Processing CreateTablet for tablet ea03b0f5edb84116ae650e76092970fc (DEFAULT_TABLE table=default.Table_e3de355d0a69496c842b4a35daa99b90 [id=c34e32ee8d9f41b68288ced87e28069f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.973515 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea03b0f5edb84116ae650e76092970fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.976209 1166 raft_consensus.cc:359] T 44e2d2bdecf247baaac26e06bccc94de P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.976388 1166 raft_consensus.cc:385] T 44e2d2bdecf247baaac26e06bccc94de P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:44.976490 1166 raft_consensus.cc:740] T 44e2d2bdecf247baaac26e06bccc94de P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:44.976730 1166 consensus_queue.cc:260] T 44e2d2bdecf247baaac26e06bccc94de P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.977264 1166 ts_tablet_manager.cc:1434] T 44e2d2bdecf247baaac26e06bccc94de P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.034s user 0.001s sys 0.000s
I20251211 20:52:44.984967 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 91254cc6ce5946b1bf3069b2202a6a11 (DEFAULT_TABLE table=default.Table_4ecbff2ab6c243f994ef4fae46276e92 [id=b3230934455d40598c6213630dd3108b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.985687 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 9d00e07cfc23457eb09e570dfecdcb9e (DEFAULT_TABLE table=default.Table_eaad557efa374b0bbd5653d90a967f8b [id=b187d831069447e39a00bc5e91f1fcf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.971119 2696 raft_consensus.cc:2804] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:44.987695 2696 raft_consensus.cc:697] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 LEADER]: Becoming Leader. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Running, Role: LEADER
I20251211 20:52:44.973111 894 tablet_service.cc:1505] Processing CreateTablet for tablet 4d03f2f064c3420d817a46e4153054e8 (DEFAULT_TABLE table=default.Table_4ec0d87202fc419380c2542d827087de [id=d9829c79b55844ec92b1b0cf594ab96b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.988363 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d03f2f064c3420d817a46e4153054e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.988677 847 leader_election.cc:304] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:44.989163 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d00e07cfc23457eb09e570dfecdcb9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.995076 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91254cc6ce5946b1bf3069b2202a6a11. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:44.996064 1168 ts_tablet_manager.cc:1434] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.033s user 0.000s sys 0.002s
I20251211 20:52:44.996438 1168 tablet_bootstrap.cc:492] T 5453f68789fd4e89b69e98860050865b P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:44.956782 710 leader_election.cc:304] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:44.997902 710 leader_election.cc:304] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:44.998723 2429 raft_consensus.cc:2804] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:44.998874 2429 raft_consensus.cc:697] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:44.998939 2561 raft_consensus.cc:2804] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:44.999058 2561 raft_consensus.cc:697] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:44.999161 2429 consensus_queue.cc:237] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.999547 2561 consensus_queue.cc:237] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.968017 741 tablet_service.cc:1505] Processing CreateTablet for tablet 31e860227d874abf8d97901fa432948a (DEFAULT_TABLE table=default.Table_ebf689834b574c408bb00777fac24c84 [id=d20c23f61583488f8c8104a18ad2461c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.971326 756 tablet_service.cc:1505] Processing CreateTablet for tablet 284e34970f634abcb2288c864f73e963 (DEFAULT_TABLE table=default.Table_e250eb3a69ef4e6da3893852edc00469 [id=ba41048fe47d4749b5e084582989eb38]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.971983 754 tablet_service.cc:1505] Processing CreateTablet for tablet 91254cc6ce5946b1bf3069b2202a6a11 (DEFAULT_TABLE table=default.Table_4ecbff2ab6c243f994ef4fae46276e92 [id=b3230934455d40598c6213630dd3108b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:44.972112 757 tablet_service.cc:1505] Processing CreateTablet for tablet 9d00e07cfc23457eb09e570dfecdcb9e (DEFAULT_TABLE table=default.Table_eaad557efa374b0bbd5653d90a967f8b [id=b187d831069447e39a00bc5e91f1fcf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.000694 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91254cc6ce5946b1bf3069b2202a6a11. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.000949 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d00e07cfc23457eb09e570dfecdcb9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.004972 710 leader_election.cc:304] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.006137 710 leader_election.cc:304] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.007315 710 leader_election.cc:304] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.007411 2601 raft_consensus.cc:493] T 71dc8d20d93243e289330d7b8c635511 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.007555 2601 raft_consensus.cc:515] T 71dc8d20d93243e289330d7b8c635511 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.008085 710 leader_election.cc:304] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.008200 2601 leader_election.cc:290] T 71dc8d20d93243e289330d7b8c635511 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:44.972687 739 tablet_service.cc:1552] Processing DeleteTablet for tablet 516f7181aaac48cdadf9fbc08d0d3f57 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:45.006421 2666 raft_consensus.cc:2804] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:45.007510 2675 raft_consensus.cc:2804] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:44.988934 2692 raft_consensus.cc:2749] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:44.972263 747 tablet_service.cc:1505] Processing CreateTablet for tablet 46a2a070cf6a42fe8bc54a42eb802fbc (DEFAULT_TABLE table=default.Table_4e1f03b9d87b4fe59dbcb3a947ee299d [id=8e66ea58499b40e59294a135324aa527]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.015991 2601 raft_consensus.cc:493] T ce1799e6604f4ed7875ddedecab6f327 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.016161 2601 raft_consensus.cc:515] T ce1799e6604f4ed7875ddedecab6f327 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.989145 847 leader_election.cc:304] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters: e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:45.016731 2692 raft_consensus.cc:2804] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:45.016829 2601 leader_election.cc:290] T ce1799e6604f4ed7875ddedecab6f327 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:45.014896 2626 consensus_queue.cc:237] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:44.980003 1165 ts_tablet_manager.cc:1929] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251211 20:52:45.022547 1166 tablet_bootstrap.cc:492] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:45.029862 2286 raft_consensus.cc:2804] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:45.032111 1166 tablet_bootstrap.cc:654] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:44.988884 914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.104.1:38217 (request call id 519) took 1060 ms. Trace:
I20251211 20:52:45.035626 911 raft_consensus.cc:3060] T 861d2d70a24946699dccd65cab05195e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.009461 887 tablet_service.cc:1505] Processing CreateTablet for tablet 9d00e07cfc23457eb09e570dfecdcb9e (DEFAULT_TABLE table=default.Table_eaad557efa374b0bbd5653d90a967f8b [id=b187d831069447e39a00bc5e91f1fcf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.044432 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d00e07cfc23457eb09e570dfecdcb9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.046586 847 leader_election.cc:304] T 645b08b33676420c96e880de3d460829 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:45.046947 2647 raft_consensus.cc:2749] T 645b08b33676420c96e880de3d460829 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:52:45.047389 847 leader_election.cc:304] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters: e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:45.047665 2647 raft_consensus.cc:2804] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:45.047729 2647 raft_consensus.cc:493] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:45.047780 2647 raft_consensus.cc:3060] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.048595 847 leader_election.cc:304] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters: e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:45.048874 2670 raft_consensus.cc:2804] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:45.048974 2670 raft_consensus.cc:697] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 [term 1 LEADER]: Becoming Leader. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Running, Role: LEADER
I20251211 20:52:45.049732 847 leader_election.cc:304] T 5710fc2f24764f818f7e5c86c6fd985e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:45.050065 2460 raft_consensus.cc:2749] T 5710fc2f24764f818f7e5c86c6fd985e P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:52:45.052160 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3de5b34b96404cdc9c191fed7f055638" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
W20251211 20:52:45.052258 847 leader_election.cc:343] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 4df68d1b6a91458e9f212ae81aa25438
W20251211 20:52:45.054345 847 leader_election.cc:343] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 4f807596fa6a4e66b89566b5d5dd1080
I20251211 20:52:45.055478 910 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f25bde14792248b188660036f07ae902" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:45.055830 1784 consensus_peers.cc:597] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 -> Peer 9230f94a6a1a49cead51aeea9f069087 (127.0.104.60:42087): Couldn't send request to peer 9230f94a6a1a49cead51aeea9f069087. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.60:42087: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20251211 20:52:45.055474 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "18b977f463d5420186c007b38110e07e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:45.056381 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5bcc94ec9630484cb7a3e8bf6d4f378a" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:45.059137 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 284e34970f634abcb2288c864f73e963. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.060520 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4df68d1b6a91458e9f212ae81aa25438" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:45.061640 2666 raft_consensus.cc:697] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:45.061960 1165 log.cc:1199] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/c8e17645a26947ab8c01f9b093fa422c
I20251211 20:52:45.062383 2286 raft_consensus.cc:697] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:45.062933 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46a2a070cf6a42fe8bc54a42eb802fbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.063167 2560 consensus_queue.cc:1048] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20251211 20:52:45.063894 710 leader_election.cc:304] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.065039 1166 tablet_bootstrap.cc:492] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:45.065307 2501 raft_consensus.cc:2804] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:45.065753 1166 ts_tablet_manager.cc:1403] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.043s user 0.006s sys 0.000s
I20251211 20:52:45.066980 1798 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_08dc6bc7d022495db033bda27f21c84c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:45.068976 1808 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_68b9d0d7258a4b5caa1779a4769bdd7f" } modify_external_catalogs: true
I20251211 20:52:45.069190 1808 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_68b9d0d7258a4b5caa1779a4769bdd7f" } modify_external_catalogs: true
I20251211 20:52:45.071750 1166 raft_consensus.cc:359] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.071919 1166 raft_consensus.cc:385] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.071966 1166 raft_consensus.cc:740] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.072204 1166 consensus_queue.cc:260] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.072995 1797 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_b43dda4aa0674e3f8ef0d55e0e2431e3" } modify_external_catalogs: true
I20251211 20:52:45.073161 1797 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_b43dda4aa0674e3f8ef0d55e0e2431e3" } modify_external_catalogs: true
I20251211 20:52:45.074687 1795 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_760e3e4714ea4128bd1de3da5a7ccccc" } modify_external_catalogs: true
I20251211 20:52:45.074836 1795 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_760e3e4714ea4128bd1de3da5a7ccccc" } modify_external_catalogs: true
I20251211 20:52:45.077975 1802 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_339c5a45d0294fe7adf34fdbfa8e3d98" } new_table_name: "default.Table_8bb5b228d60e4d29a41d9ba40c33d45d" modify_external_catalogs: true
I20251211 20:52:45.081452 1166 ts_tablet_manager.cc:1434] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.016s user 0.002s sys 0.000s
I20251211 20:52:45.081468 2325 raft_consensus.cc:493] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.081633 2325 raft_consensus.cc:515] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.082391 2325 leader_election.cc:290] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.082566 2678 raft_consensus.cc:493] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.082715 2678 raft_consensus.cc:515] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.083289 2678 leader_election.cc:290] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.083454 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3de5b34b96404cdc9c191fed7f055638" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:45.083825 771 raft_consensus.cc:2468] T 4df68d1b6a91458e9f212ae81aa25438 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 0.
I20251211 20:52:45.083878 2666 consensus_queue.cc:237] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:45.088294 1798 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_08dc6bc7d022495db033bda27f21c84c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:45.090099 1166 tablet_bootstrap.cc:492] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:45.092303 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71dc8d20d93243e289330d7b8c635511" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:45.094167 1166 tablet_bootstrap.cc:654] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.094303 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31e860227d874abf8d97901fa432948a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.095920 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71dc8d20d93243e289330d7b8c635511" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:45.096491 2286 consensus_queue.cc:237] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.097404 1166 tablet_bootstrap.cc:492] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:45.097618 2675 raft_consensus.cc:697] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:45.097900 2501 raft_consensus.cc:697] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:45.097538 1166 ts_tablet_manager.cc:1403] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20251211 20:52:45.098976 1166 raft_consensus.cc:359] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.099217 1166 raft_consensus.cc:385] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.099320 1166 raft_consensus.cc:740] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.099648 1166 consensus_queue.cc:260] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:45.103142 982 leader_election.cc:343] T 71dc8d20d93243e289330d7b8c635511 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.111234 1166 ts_tablet_manager.cc:1434] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20251211 20:52:45.111527 2601 raft_consensus.cc:493] T 95866de91f9d437d80f40b7962253610 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.111974 2601 raft_consensus.cc:515] T 95866de91f9d437d80f40b7962253610 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.112610 2601 leader_election.cc:290] T 95866de91f9d437d80f40b7962253610 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:45.113780 1166 tablet_bootstrap.cc:492] T a7979798e92e45eb969197c8c7ac2580 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:45.115772 1166 tablet_bootstrap.cc:654] T a7979798e92e45eb969197c8c7ac2580 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.119450 886 tablet_service.cc:1505] Processing CreateTablet for tablet 6ab29c688eb14134836ed661f9193f9e (DEFAULT_TABLE table=default.Table_6bc2fc890a7a485ba23c3214f772b758 [id=a703dcf9f6244f34878332575ff2324c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.119941 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ab29c688eb14134836ed661f9193f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.127331 1017 tablet_service.cc:1505] Processing CreateTablet for tablet d8943d496a254ccebc987ed3ea17fe3d (DEFAULT_TABLE table=default.Table_be91bd80b0374d5eae01b605de36b203 [id=eb7478f4f95f45479e8c830ba6575a40]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.127848 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8943d496a254ccebc987ed3ea17fe3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.128679 1010 tablet_service.cc:1505] Processing CreateTablet for tablet ff4d5ec1f87247e896f9c099c532265a (DEFAULT_TABLE table=default.Table_3ac4746e17264f5a93890f3600505781 [id=475e0a1ec0344d10addbcfa42fe05b68]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.129057 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff4d5ec1f87247e896f9c099c532265a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.107092 2560 raft_consensus.cc:2804] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20251211 20:52:45.136394 711 consensus_peers.cc:597] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:45.140537 1165 ts_tablet_manager.cc:1950] T c8e17645a26947ab8c01f9b093fa422c P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:45.107021 710 leader_election.cc:304] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.141844 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 6ab29c688eb14134836ed661f9193f9e (DEFAULT_TABLE table=default.Table_6bc2fc890a7a485ba23c3214f772b758 [id=a703dcf9f6244f34878332575ff2324c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.142449 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ab29c688eb14134836ed661f9193f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.143672 2675 consensus_queue.cc:237] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.144734 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "95866de91f9d437d80f40b7962253610" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:45.049186 2670 consensus_queue.cc:237] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.147377 2560 raft_consensus.cc:493] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:45.147914 2678 raft_consensus.cc:2804] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:45.147953 757 tablet_service.cc:1505] Processing CreateTablet for tablet 6ab29c688eb14134836ed661f9193f9e (DEFAULT_TABLE table=default.Table_6bc2fc890a7a485ba23c3214f772b758 [id=a703dcf9f6244f34878332575ff2324c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.148047 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1799e6604f4ed7875ddedecab6f327" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:45.148360 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 76858.4us
I20251211 20:52:45.149004 1165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 968) took 3957 ms. Trace:
I20251211 20:52:45.106878 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1799e6604f4ed7875ddedecab6f327" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:45.152132 891 tablet_service.cc:1505] Processing CreateTablet for tablet d8943d496a254ccebc987ed3ea17fe3d (DEFAULT_TABLE table=default.Table_be91bd80b0374d5eae01b605de36b203 [id=eb7478f4f95f45479e8c830ba6575a40]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.151518 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee7b93b0b87541d9af6eb9e68b4cea41" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:45.152372 1046 raft_consensus.cc:2468] T ee7b93b0b87541d9af6eb9e68b4cea41 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:45.152551 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8943d496a254ccebc987ed3ea17fe3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.152515 1036 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f22e4d34cde24b37a025c1b5a1046b1b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:45.152652 1036 raft_consensus.cc:2468] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:45.153353 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4d2b7e2a0abe4592bc806d5ef3f894d8" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:45.153503 1049 raft_consensus.cc:2468] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:45.155157 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3ecf0ea8ce19486c81d1d1a91d57a7ce" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:45.155304 1047 raft_consensus.cc:2468] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:45.035866 2696 consensus_queue.cc:237] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.159844 2501 consensus_queue.cc:237] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.161432 890 tablet_service.cc:1505] Processing CreateTablet for tablet ff4d5ec1f87247e896f9c099c532265a (DEFAULT_TABLE table=default.Table_3ac4746e17264f5a93890f3600505781 [id=475e0a1ec0344d10addbcfa42fe05b68]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.161895 2325 raft_consensus.cc:493] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.162387 1168 tablet_bootstrap.cc:654] T 5453f68789fd4e89b69e98860050865b P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:45.164696 1780 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet d43210ec61af423ebde624397dea9f2e (table default.Table_af0df4d1ee144455a6041e5a6a0b9b80 [id=7c5b2e33fa1c4b12b3941577360908a8], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet d43210ec61af423ebde624397dea9f2e consensus state has no leader
I20251211 20:52:45.166785 2560 raft_consensus.cc:3060] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:45.167176 983 leader_election.cc:343] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.167289 983 leader_election.cc:304] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:45.167554 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ab29c688eb14134836ed661f9193f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.167681 2601 raft_consensus.cc:2749] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:45.168344 983 leader_election.cc:343] T f25bde14792248b188660036f07ae902 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.168443 983 leader_election.cc:304] T f25bde14792248b188660036f07ae902 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:45.168745 2601 raft_consensus.cc:2749] T f25bde14792248b188660036f07ae902 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:45.169528 1165 rpcz_store.cc:276] 1211 20:52:41.183048 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:41.183111 (+ 63us) service_pool.cc:225] Handling call
1211 20:52:44.876592 (+3693481us) ts_tablet_manager.cc:1151] Deleting tablet c8e17645a26947ab8c01f9b093fa422c
1211 20:52:44.876608 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:44.976724 (+100116us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:44.979822 (+ 3098us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:45.140911 (+161089us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3693274}
I20251211 20:52:45.169946 1165 tablet_replica.cc:333] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:45.170568 1165 raft_consensus.cc:2243] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:45.170890 1165 raft_consensus.cc:2272] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:45.172963 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff4d5ec1f87247e896f9c099c532265a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.175767 754 tablet_service.cc:1505] Processing CreateTablet for tablet ff4d5ec1f87247e896f9c099c532265a (DEFAULT_TABLE table=default.Table_3ac4746e17264f5a93890f3600505781 [id=475e0a1ec0344d10addbcfa42fe05b68]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.176388 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff4d5ec1f87247e896f9c099c532265a. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:45.176381 982 leader_election.cc:343] T 95866de91f9d437d80f40b7962253610 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:45.176597 982 leader_election.cc:343] T ce1799e6604f4ed7875ddedecab6f327 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.178573 739 tablet_service.cc:1505] Processing CreateTablet for tablet d8943d496a254ccebc987ed3ea17fe3d (DEFAULT_TABLE table=default.Table_be91bd80b0374d5eae01b605de36b203 [id=eb7478f4f95f45479e8c830ba6575a40]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.179082 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8943d496a254ccebc987ed3ea17fe3d. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:45.181244 983 leader_election.cc:343] T 18b977f463d5420186c007b38110e07e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.181380 983 leader_election.cc:304] T 18b977f463d5420186c007b38110e07e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:45.181762 2601 raft_consensus.cc:2749] T 18b977f463d5420186c007b38110e07e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:45.182332 983 leader_election.cc:343] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.182438 983 leader_election.cc:304] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:45.182770 2601 raft_consensus.cc:2749] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:45.184217 847 leader_election.cc:343] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 4f807596fa6a4e66b89566b5d5dd1080
W20251211 20:52:45.185246 983 leader_election.cc:343] T 71dc8d20d93243e289330d7b8c635511 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.185359 983 leader_election.cc:304] T 71dc8d20d93243e289330d7b8c635511 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:45.185590 711 leader_election.cc:343] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.185721 2601 raft_consensus.cc:2749] T 71dc8d20d93243e289330d7b8c635511 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:45.185881 711 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 45096us
W20251211 20:52:45.186090 711 leader_election.cc:343] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:45.186322 711 leader_election.cc:343] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:45.186537 711 leader_election.cc:343] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:45.186749 711 leader_election.cc:343] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:45.186980 711 leader_election.cc:343] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:45.187196 711 leader_election.cc:343] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.187399 2499 consensus_queue.cc:1048] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:45.187630 2325 raft_consensus.cc:515] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.187745 2273 consensus_queue.cc:1048] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:45.188126 2325 leader_election.cc:290] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.188271 2678 raft_consensus.cc:697] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:45.188526 2678 consensus_queue.cc:237] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:45.189630 713 connection.cc:441] server connection from 127.0.0.1:37886 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37886 (request call id 968) could send its response
I20251211 20:52:45.187412 710 leader_election.cc:304] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.190208 2499 raft_consensus.cc:2804] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:45.190320 2499 raft_consensus.cc:493] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:45.190342 710 leader_election.cc:304] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.190419 2499 raft_consensus.cc:3060] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.190840 710 leader_election.cc:304] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.191092 2338 raft_consensus.cc:2804] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:45.191184 2338 raft_consensus.cc:493] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:45.191303 2338 raft_consensus.cc:3060] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.191807 2663 raft_consensus.cc:2804] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:45.191912 2663 raft_consensus.cc:493] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:45.191947 1806 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_a06aeab492cf4bd59dbc1274c6c6827d" } modify_external_catalogs: true
I20251211 20:52:45.192030 2663 raft_consensus.cc:3060] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.192129 1806 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_a06aeab492cf4bd59dbc1274c6c6827d" } modify_external_catalogs: true
I20251211 20:52:45.193075 2494 consensus_queue.cc:1048] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:45.193548 2494 consensus_queue.cc:1048] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:45.193812 1809 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_d8d6a529a8934b64a5a430cb54ac0954"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:45.194378 1809 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d8d6a529a8934b64a5a430cb54ac0954 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:45.186115 2460 raft_consensus.cc:493] T c7147571483e41ad970e573a08c368ca P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.197171 847 leader_election.cc:304] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
W20251211 20:52:45.192047 711 leader_election.cc:343] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:45.200706 711 leader_election.cc:343] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:45.201009 711 consensus_peers.cc:597] T 50f9f29c064c4f938a0f20f302ab5e8a P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:45.201843 711 leader_election.cc:343] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:45.202179 711 leader_election.cc:343] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:45.202441 711 leader_election.cc:343] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:45.203337 711 consensus_peers.cc:597] T 41c30bded6504bbc9f0af8f34dc81e8d P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:45.208902 2601 raft_consensus.cc:493] T d43210ec61af423ebde624397dea9f2e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.209075 2601 raft_consensus.cc:515] T d43210ec61af423ebde624397dea9f2e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:45.209404 711 consensus_peers.cc:597] T 940dfd8781de444a9caa78dd363c7451 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:45.209542 711 leader_election.cc:343] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.209712 2601 leader_election.cc:290] T d43210ec61af423ebde624397dea9f2e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
W20251211 20:52:45.209796 711 leader_election.cc:343] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.214468 2663 raft_consensus.cc:515] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.215116 2663 leader_election.cc:290] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.218146 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d43210ec61af423ebde624397dea9f2e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:45.218575 2499 raft_consensus.cc:515] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:45.218793 982 leader_election.cc:343] T d43210ec61af423ebde624397dea9f2e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.219147 2499 leader_election.cc:290] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.222229 2338 raft_consensus.cc:515] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.222859 2338 leader_election.cc:290] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.206780 2561 raft_consensus.cc:493] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.228401 2561 raft_consensus.cc:515] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.229060 2561 leader_election.cc:290] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.231199 2560 raft_consensus.cc:515] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:45.240861 1802 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bc6a1c84501d4e179cd2828b50fddc23 consensus state has no leader
I20251211 20:52:45.243408 2560 leader_election.cc:290] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:45.249907 711 leader_election.cc:343] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:45.250473 711 leader_election.cc:343] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.260985 754 tablet_service.cc:1505] Processing CreateTablet for tablet 73a4bf965f0b4ac0a63d2c48aeb9767a (DEFAULT_TABLE table=default.Table_0d3816dfd69844518591138a3a844a9a [id=2b6bb7b58170467abe6463a912f608c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.261152 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 73a4bf965f0b4ac0a63d2c48aeb9767a (DEFAULT_TABLE table=default.Table_0d3816dfd69844518591138a3a844a9a [id=2b6bb7b58170467abe6463a912f608c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.261507 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73a4bf965f0b4ac0a63d2c48aeb9767a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.261682 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73a4bf965f0b4ac0a63d2c48aeb9767a. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:45.267824 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 76580.8us
I20251211 20:52:45.268146 710 leader_election.cc:304] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.268443 2494 raft_consensus.cc:2804] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:45.268503 2494 raft_consensus.cc:493] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:45.268553 2494 raft_consensus.cc:3060] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.270861 2494 raft_consensus.cc:515] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.270989 1165 ts_tablet_manager.cc:1916] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:45.271453 2494 leader_election.cc:290] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.282204 2647 raft_consensus.cc:515] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.282876 2647 leader_election.cc:290] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Requested vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.037269 914 rpcz_store.cc:276] 1211 20:52:43.928406 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:43.928464 (+ 58us) service_pool.cc:225] Handling call
1211 20:52:43.928604 (+ 140us) raft_consensus.cc:1471] Updating replica for 1 ops
1211 20:52:44.691534 (+762930us) raft_consensus.cc:1534] Early marking committed up to index 0
1211 20:52:44.691539 (+ 5us) raft_consensus.cc:1539] Triggering prepare for 1 ops
1211 20:52:44.691751 (+ 212us) log.cc:844] Serialized 26 byte log entry
1211 20:52:44.939736 (+247985us) raft_consensus.cc:1645] Marking committed up to 1
1211 20:52:44.987707 (+ 47971us) log.cc:844] Serialized 14 byte log entry
1211 20:52:44.987853 (+ 146us) raft_consensus.cc:1697] Filling consensus response to leader.
1211 20:52:44.988154 (+ 301us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
1211 20:52:44.988741 (+ 587us) raft_consensus.cc:1685] finished
1211 20:52:44.988743 (+ 2us) raft_consensus.cc:1691] UpdateReplicas() finished
1211 20:52:44.988860 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":15802,"prepare.queue_time_us":15934,"prepare.run_cpu_time_us":7,"prepare.run_wall_time_us":7,"raft.queue_time_us":926,"raft.run_cpu_time_us":60,"raft.run_wall_time_us":61,"thread_start_us":247881,"threads_started":1,"wal-append.queue_time_us":247979}
I20251211 20:52:45.196974 2460 raft_consensus.cc:515] T c7147571483e41ad970e573a08c368ca P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.284538 2460 leader_election.cc:290] T c7147571483e41ad970e573a08c368ca P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.291571 1165 ts_tablet_manager.cc:1929] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:45.297806 1015 tablet_service.cc:1505] Processing CreateTablet for tablet 09cd11f8b23c4f8a98a2b2eb77bb7b8c (DEFAULT_TABLE table=default.Table_440d3f9c240c448bb5bf5bfbdaf35d00 [id=276a75404b564adcb63880b9bbed2359]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.298291 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09cd11f8b23c4f8a98a2b2eb77bb7b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.296828 757 tablet_service.cc:1505] Processing CreateTablet for tablet 09cd11f8b23c4f8a98a2b2eb77bb7b8c (DEFAULT_TABLE table=default.Table_440d3f9c240c448bb5bf5bfbdaf35d00 [id=276a75404b564adcb63880b9bbed2359]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.299301 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09cd11f8b23c4f8a98a2b2eb77bb7b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.306973 1166 tablet_bootstrap.cc:492] T a7979798e92e45eb969197c8c7ac2580 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:45.307225 1166 ts_tablet_manager.cc:1403] T a7979798e92e45eb969197c8c7ac2580 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.194s user 0.024s sys 0.052s
I20251211 20:52:45.308061 1166 raft_consensus.cc:359] T a7979798e92e45eb969197c8c7ac2580 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.308157 1165 log.cc:1199] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/c7bd218b3ff340b2a589c9517b49a649
I20251211 20:52:45.308387 1166 raft_consensus.cc:385] T a7979798e92e45eb969197c8c7ac2580 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.308494 1166 raft_consensus.cc:740] T a7979798e92e45eb969197c8c7ac2580 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.308557 1165 ts_tablet_manager.cc:1950] T c7bd218b3ff340b2a589c9517b49a649 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:45.308766 1166 consensus_queue.cc:260] T a7979798e92e45eb969197c8c7ac2580 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.309719 1166 ts_tablet_manager.cc:1434] T a7979798e92e45eb969197c8c7ac2580 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:45.310523 1166 tablet_bootstrap.cc:492] T 7be44df39a8f4d6da793479d425fa684 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:45.311739 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4d2b7e2a0abe4592bc806d5ef3f894d8" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:45.311903 1047 raft_consensus.cc:3060] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.312374 1038 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee7b93b0b87541d9af6eb9e68b4cea41" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:45.312507 1038 raft_consensus.cc:3060] T ee7b93b0b87541d9af6eb9e68b4cea41 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.313112 1166 tablet_bootstrap.cc:654] T 7be44df39a8f4d6da793479d425fa684 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.314482 1047 raft_consensus.cc:2468] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:45.314738 1038 raft_consensus.cc:2468] T ee7b93b0b87541d9af6eb9e68b4cea41 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:45.315014 1037 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbe3529c301a4a65b58b634a443e1fb6" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:45.315197 1037 raft_consensus.cc:2468] T bbe3529c301a4a65b58b634a443e1fb6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:45.316964 1038 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75dd170c5828409fa69bd68fdf28a72b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:45.317126 1038 raft_consensus.cc:2468] T 75dd170c5828409fa69bd68fdf28a72b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:45.318701 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b85af79c2574443998e847bc19c53b69" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:45.319188 1047 raft_consensus.cc:3060] T b85af79c2574443998e847bc19c53b69 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.321105 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3ecf0ea8ce19486c81d1d1a91d57a7ce" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:45.321275 1049 raft_consensus.cc:3060] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.321803 1166 tablet_bootstrap.cc:492] T 7be44df39a8f4d6da793479d425fa684 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:45.321923 1166 ts_tablet_manager.cc:1403] T 7be44df39a8f4d6da793479d425fa684 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.007s
I20251211 20:52:45.322693 1166 raft_consensus.cc:359] T 7be44df39a8f4d6da793479d425fa684 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.322839 1166 raft_consensus.cc:385] T 7be44df39a8f4d6da793479d425fa684 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.322888 1166 raft_consensus.cc:740] T 7be44df39a8f4d6da793479d425fa684 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.322996 1047 raft_consensus.cc:2468] T b85af79c2574443998e847bc19c53b69 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:45.323076 1166 consensus_queue.cc:260] T 7be44df39a8f4d6da793479d425fa684 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.324966 1166 ts_tablet_manager.cc:1434] T 7be44df39a8f4d6da793479d425fa684 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:45.325716 1166 tablet_bootstrap.cc:492] T 497c10742f01476db885bd70c4d2c491 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:45.327342 1166 tablet_bootstrap.cc:654] T 497c10742f01476db885bd70c4d2c491 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.331290 1049 raft_consensus.cc:2468] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
W20251211 20:52:45.335042 2707 log.cc:927] Time spent T a3b1aadf17ea4bf0a896d23cd543fca7 P eb0f1933d9a74fccaa4618047fc05c41: Append to log took a long time: real 0.250s user 0.000s sys 0.000s
I20251211 20:52:45.343505 1168 tablet_bootstrap.cc:492] T 5453f68789fd4e89b69e98860050865b P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:45.343801 1168 ts_tablet_manager.cc:1403] T 5453f68789fd4e89b69e98860050865b P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.347s user 0.008s sys 0.019s
I20251211 20:52:45.344574 1168 raft_consensus.cc:359] T 5453f68789fd4e89b69e98860050865b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.347373 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet c7bd218b3ff340b2a589c9517b49a649 (table default.Table_233d5484bd4e4456aed925bc271edfa5 [id=ca0c3c6978874d93a04b57cc0c425c83]) successfully deleted
W20251211 20:52:45.348225 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet ca89f09cc0954389b679e8bb8b5e4dfb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ca89f09cc0954389b679e8bb8b5e4dfb already in progress: creating tablet
W20251211 20:52:45.348906 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 9b99860ee6be45a0b358e686332d3137 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9b99860ee6be45a0b358e686332d3137 already in progress: creating tablet
W20251211 20:52:45.349557 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 06ba98a726424dd89450053c5c5793f4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06ba98a726424dd89450053c5c5793f4 already in progress: creating tablet
W20251211 20:52:45.350207 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 683d0f2068fb4a6ebc9c169ad7b455ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 683d0f2068fb4a6ebc9c169ad7b455ce already in progress: creating tablet
W20251211 20:52:45.350734 1780 catalog_manager.cc:4952] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 41627cb27be8443fa37e514f9b3825a6 because the tablet was not found. No further retry: Not found: Tablet not found: 41627cb27be8443fa37e514f9b3825a6
W20251211 20:52:45.351329 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 59e955aa07c14fbbb347b88eca59a74e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 59e955aa07c14fbbb347b88eca59a74e already in progress: creating tablet
W20251211 20:52:45.351943 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 516f7181aaac48cdadf9fbc08d0d3f57 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 516f7181aaac48cdadf9fbc08d0d3f57 already in progress: creating tablet
I20251211 20:52:45.353896 1168 raft_consensus.cc:385] T 5453f68789fd4e89b69e98860050865b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.354058 1168 raft_consensus.cc:740] T 5453f68789fd4e89b69e98860050865b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.358006 2460 raft_consensus.cc:2749] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20251211 20:52:45.358428 847 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.1:42177, user_credentials={real_user=slave}} blocked reactor thread for 174203us
I20251211 20:52:45.359725 710 leader_election.cc:304] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.366214 2460 raft_consensus.cc:493] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.366401 2460 raft_consensus.cc:515] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.367035 2460 leader_election.cc:290] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.035779 2692 raft_consensus.cc:697] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 LEADER]: Becoming Leader. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Running, Role: LEADER
I20251211 20:52:45.370368 890 tablet_service.cc:1505] Processing CreateTablet for tablet 73a4bf965f0b4ac0a63d2c48aeb9767a (DEFAULT_TABLE table=default.Table_0d3816dfd69844518591138a3a844a9a [id=2b6bb7b58170467abe6463a912f608c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.370988 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73a4bf965f0b4ac0a63d2c48aeb9767a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.372814 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f22e4d34cde24b37a025c1b5a1046b1b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:45.373435 2429 raft_consensus.cc:2804] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:45.373595 2429 raft_consensus.cc:697] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:45.373842 2429 consensus_queue.cc:237] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.374143 1037 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f22e4d34cde24b37a025c1b5a1046b1b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:45.374387 1037 raft_consensus.cc:3060] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.377403 1037 raft_consensus.cc:2468] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:45.379549 710 leader_election.cc:304] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.379895 2501 raft_consensus.cc:2804] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:45.380093 2501 raft_consensus.cc:697] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:45.380484 2501 consensus_queue.cc:237] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.387441 883 tablet_service.cc:1505] Processing CreateTablet for tablet 09cd11f8b23c4f8a98a2b2eb77bb7b8c (DEFAULT_TABLE table=default.Table_440d3f9c240c448bb5bf5bfbdaf35d00 [id=276a75404b564adcb63880b9bbed2359]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.388583 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09cd11f8b23c4f8a98a2b2eb77bb7b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.379891 710 leader_election.cc:304] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.373347 847 leader_election.cc:304] T 3de5b34b96404cdc9c191fed7f055638 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:45.390635 710 leader_election.cc:304] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.390664 2460 raft_consensus.cc:2749] T 3de5b34b96404cdc9c191fed7f055638 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:45.391239 847 leader_election.cc:304] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters: e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:45.390916 2736 raft_consensus.cc:2804] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:45.391921 2736 raft_consensus.cc:493] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:45.392069 2736 raft_consensus.cc:3060] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.392653 1168 consensus_queue.cc:260] T 5453f68789fd4e89b69e98860050865b P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.393134 2494 raft_consensus.cc:2804] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:45.393939 2494 raft_consensus.cc:493] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:45.393927 914 raft_consensus.cc:3060] T c7147571483e41ad970e573a08c368ca P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.394064 2494 raft_consensus.cc:3060] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.394409 906 raft_consensus.cc:3060] T 3de5b34b96404cdc9c191fed7f055638 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.396684 2494 raft_consensus.cc:515] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.397150 2460 raft_consensus.cc:2804] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:45.397246 2460 raft_consensus.cc:493] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:45.397305 2460 raft_consensus.cc:3060] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.397571 2494 leader_election.cc:290] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:45.397948 710 consensus_peers.cc:597] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2e33606930774fe4aafc6e3819618af8. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:45.399205 777 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb4004212441473483a209d23c0800ce" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:45.400557 777 raft_consensus.cc:3060] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.403183 777 raft_consensus.cc:2468] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 1.
I20251211 20:52:45.399397 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7147571483e41ad970e573a08c368ca" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:45.404771 710 leader_election.cc:304] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.405120 2675 raft_consensus.cc:2804] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:45.405223 2675 raft_consensus.cc:697] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:45.405457 2675 consensus_queue.cc:237] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.407402 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3ecf0ea8ce19486c81d1d1a91d57a7ce" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:45.407546 710 leader_election.cc:304] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.407801 2675 raft_consensus.cc:2804] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:45.407914 2675 raft_consensus.cc:697] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:45.408124 2675 consensus_queue.cc:237] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.382797 2692 consensus_queue.cc:237] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.413151 2647 raft_consensus.cc:493] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.413316 2647 raft_consensus.cc:515] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.413924 2647 leader_election.cc:290] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.416652 2460 raft_consensus.cc:515] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.418006 1168 ts_tablet_manager.cc:1434] T 5453f68789fd4e89b69e98860050865b P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.074s user 0.001s sys 0.001s
I20251211 20:52:45.418995 1168 tablet_bootstrap.cc:492] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:45.419440 847 leader_election.cc:343] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 87863daf77304a399318cdf56da9f9bb
I20251211 20:52:45.419780 847 leader_election.cc:304] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters:
I20251211 20:52:45.420228 2647 raft_consensus.cc:2804] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:45.423338 1798 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_5da0e9ddf96549c88878f7136d0ba415" } new_table_name: "default.Table_d5276e3cc46d4331b36a8878680fc55e" modify_external_catalogs: true
I20251211 20:52:45.424854 1168 tablet_bootstrap.cc:654] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.425051 1802 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_282dd63df9ff4557a5ab2761d63df7b1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:45.425695 1802 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_282dd63df9ff4557a5ab2761d63df7b1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:45.393179 754 tablet_service.cc:1505] Processing CreateTablet for tablet a87fe058d1de4781b24f02424eaeeccd (DEFAULT_TABLE table=default.Table_d87c2c33f4ff4aee9ff82b0653d4cc43 [id=2876f1fc09634b03afe5664e82c017d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.429989 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a87fe058d1de4781b24f02424eaeeccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.421993 2736 raft_consensus.cc:515] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.442004 2736 leader_election.cc:290] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.453538 1168 tablet_bootstrap.cc:492] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:45.455837 1168 ts_tablet_manager.cc:1403] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.037s user 0.003s sys 0.000s
I20251211 20:52:45.457520 1168 raft_consensus.cc:359] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.460956 1168 raft_consensus.cc:385] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.461491 1168 raft_consensus.cc:740] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.464052 1168 consensus_queue.cc:260] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.466226 1168 ts_tablet_manager.cc:1434] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.009s user 0.001s sys 0.000s
I20251211 20:52:45.466219 1802 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_7287bc8f28a64682b352f9adae2ec224" } modify_external_catalogs: true
I20251211 20:52:45.466497 1802 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_7287bc8f28a64682b352f9adae2ec224" } modify_external_catalogs: true
I20251211 20:52:45.467722 1168 tablet_bootstrap.cc:492] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:45.470710 1168 tablet_bootstrap.cc:654] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.483734 754 tablet_service.cc:1505] Processing CreateTablet for tablet 11af6cfd71a94e768c66b36862b048bb (DEFAULT_TABLE table=default.Table_bae5e6fe430247c99eae40ae86192b39 [id=6212dd14e00341be8a2c71adcf7f4c61]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.484367 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11af6cfd71a94e768c66b36862b048bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.486251 757 tablet_service.cc:1505] Processing CreateTablet for tablet 1a1016ef8bf94d2e95d4774df44f01f1 (DEFAULT_TABLE table=default.Table_b1d96cf6ac344427a92df55d1dfcee90 [id=679d27d782fb480086443c78df21fa52]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.486624 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a1016ef8bf94d2e95d4774df44f01f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.515074 1168 tablet_bootstrap.cc:492] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:45.516299 1168 ts_tablet_manager.cc:1403] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.049s user 0.003s sys 0.004s
I20251211 20:52:45.521971 1168 raft_consensus.cc:359] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.525540 1168 raft_consensus.cc:385] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.525703 1168 raft_consensus.cc:740] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.526018 1168 consensus_queue.cc:260] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.526557 1168 ts_tablet_manager.cc:1434] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251211 20:52:45.526986 1168 tablet_bootstrap.cc:492] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:45.528681 1168 tablet_bootstrap.cc:654] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.529750 894 tablet_service.cc:1505] Processing CreateTablet for tablet 11af6cfd71a94e768c66b36862b048bb (DEFAULT_TABLE table=default.Table_bae5e6fe430247c99eae40ae86192b39 [id=6212dd14e00341be8a2c71adcf7f4c61]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.530714 905 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4d2b7e2a0abe4592bc806d5ef3f894d8" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:45.535562 1168 tablet_bootstrap.cc:492] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:45.536206 1168 ts_tablet_manager.cc:1403] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20251211 20:52:45.537230 1168 raft_consensus.cc:359] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.538416 1168 raft_consensus.cc:385] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.538597 1168 raft_consensus.cc:740] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.539003 1168 consensus_queue.cc:260] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.540128 1168 ts_tablet_manager.cc:1434] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:45.540822 1168 tablet_bootstrap.cc:492] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:45.542775 1168 tablet_bootstrap.cc:654] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.564033 1168 tablet_bootstrap.cc:492] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:45.564966 1168 ts_tablet_manager.cc:1403] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.003s
I20251211 20:52:45.566020 1168 raft_consensus.cc:359] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.567463 910 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbe3529c301a4a65b58b634a443e1fb6" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:45.567858 1168 raft_consensus.cc:385] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.568120 1168 raft_consensus.cc:740] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.568502 907 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b85af79c2574443998e847bc19c53b69" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:45.568547 1168 consensus_queue.cc:260] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.569029 907 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3ecf0ea8ce19486c81d1d1a91d57a7ce" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:45.569516 1168 ts_tablet_manager.cc:1434] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251211 20:52:45.570326 1168 tablet_bootstrap.cc:492] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:45.572407 1168 tablet_bootstrap.cc:654] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:45.572978 847 leader_election.cc:343] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: bb4004212441473483a209d23c0800ce
I20251211 20:52:45.576406 910 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75dd170c5828409fa69bd68fdf28a72b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:45.578313 907 raft_consensus.cc:3060] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:45.579537 847 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 155478us
I20251211 20:52:45.582710 1168 tablet_bootstrap.cc:492] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:45.583757 1168 ts_tablet_manager.cc:1403] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.004s
I20251211 20:52:45.584723 1168 raft_consensus.cc:359] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.585933 1168 raft_consensus.cc:385] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.586177 1168 raft_consensus.cc:740] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.586658 1168 consensus_queue.cc:260] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.587730 1168 ts_tablet_manager.cc:1434] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:45.588510 1168 tablet_bootstrap.cc:492] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:45.589547 847 leader_election.cc:343] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 87863daf77304a399318cdf56da9f9bb
I20251211 20:52:45.590124 847 leader_election.cc:304] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:45.590744 2626 raft_consensus.cc:2749] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:45.591269 1168 tablet_bootstrap.cc:654] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.599138 1160 tablet_bootstrap.cc:654] T f3855ffb51a74101b26fd28429e65bc2 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.600185 1168 tablet_bootstrap.cc:492] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:45.600605 886 tablet_service.cc:1505] Processing CreateTablet for tablet 1a1016ef8bf94d2e95d4774df44f01f1 (DEFAULT_TABLE table=default.Table_b1d96cf6ac344427a92df55d1dfcee90 [id=679d27d782fb480086443c78df21fa52]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.600909 1168 ts_tablet_manager.cc:1403] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20251211 20:52:45.601928 1168 raft_consensus.cc:359] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.603073 1168 raft_consensus.cc:385] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.603327 1168 raft_consensus.cc:740] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.603987 1168 consensus_queue.cc:260] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.605134 1168 ts_tablet_manager.cc:1434] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:45.605835 1168 tablet_bootstrap.cc:492] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:45.608315 908 raft_consensus.cc:3060] T fed300d31c414f4a9b27c66e78097688 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.609413 1168 tablet_bootstrap.cc:654] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.611903 2460 leader_election.cc:290] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Requested vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.612778 2769 tablet.cc:1722] T 861d2d70a24946699dccd65cab05195e P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:45.399052 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c7147571483e41ad970e573a08c368ca" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:45.404711 1166 tablet_bootstrap.cc:492] T 497c10742f01476db885bd70c4d2c491 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:45.435601 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75dd170c5828409fa69bd68fdf28a72b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:45.438416 2601 raft_consensus.cc:493] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.616832 1166 ts_tablet_manager.cc:1403] T 497c10742f01476db885bd70c4d2c491 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.291s user 0.036s sys 0.040s
I20251211 20:52:45.616844 1045 raft_consensus.cc:3060] T 75dd170c5828409fa69bd68fdf28a72b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.617594 1166 raft_consensus.cc:359] T 497c10742f01476db885bd70c4d2c491 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.617780 1166 raft_consensus.cc:385] T 497c10742f01476db885bd70c4d2c491 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.617839 1166 raft_consensus.cc:740] T 497c10742f01476db885bd70c4d2c491 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.618068 1166 consensus_queue.cc:260] T 497c10742f01476db885bd70c4d2c491 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.443982 2699 raft_consensus.cc:493] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.445580 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbe3529c301a4a65b58b634a443e1fb6" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:45.618568 1049 raft_consensus.cc:3060] T bbe3529c301a4a65b58b634a443e1fb6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.618544 2699 raft_consensus.cc:515] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.618810 1168 tablet_bootstrap.cc:492] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:45.618939 1168 ts_tablet_manager.cc:1403] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20251211 20:52:45.619117 2699 leader_election.cc:290] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:45.619702 1168 raft_consensus.cc:359] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.619568 1045 raft_consensus.cc:2468] T 75dd170c5828409fa69bd68fdf28a72b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:45.616847 2601 raft_consensus.cc:515] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.620548 2601 leader_election.cc:290] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:45.620968 1049 raft_consensus.cc:2468] T bbe3529c301a4a65b58b634a443e1fb6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:45.621438 1168 raft_consensus.cc:385] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.621657 1168 raft_consensus.cc:740] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.622045 1168 consensus_queue.cc:260] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.623412 1168 ts_tablet_manager.cc:1434] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:45.623948 1168 tablet_bootstrap.cc:492] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:45.626012 1168 tablet_bootstrap.cc:654] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.616107 1019 tablet_service.cc:1505] Processing CreateTablet for tablet a87fe058d1de4781b24f02424eaeeccd (DEFAULT_TABLE table=default.Table_d87c2c33f4ff4aee9ff82b0653d4cc43 [id=2876f1fc09634b03afe5664e82c017d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.626469 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4df68d1b6a91458e9f212ae81aa25438" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:45.627852 771 raft_consensus.cc:3060] T 4df68d1b6a91458e9f212ae81aa25438 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.628366 1166 ts_tablet_manager.cc:1434] T 497c10742f01476db885bd70c4d2c491 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.011s user 0.001s sys 0.005s
I20251211 20:52:45.631413 771 raft_consensus.cc:2468] T 4df68d1b6a91458e9f212ae81aa25438 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb0f1933d9a74fccaa4618047fc05c41 in term 1.
I20251211 20:52:45.631827 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a87fe058d1de4781b24f02424eaeeccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.632613 1166 tablet_bootstrap.cc:492] T fd43a27db69044b68826cda87e16a086 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:45.622367 909 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.653s user 0.012s sys 0.002s
I20251211 20:52:45.634716 1166 tablet_bootstrap.cc:654] T fd43a27db69044b68826cda87e16a086 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.643971 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 11af6cfd71a94e768c66b36862b048bb (DEFAULT_TABLE table=default.Table_bae5e6fe430247c99eae40ae86192b39 [id=6212dd14e00341be8a2c71adcf7f4c61]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.644502 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11af6cfd71a94e768c66b36862b048bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.646659 905 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee7b93b0b87541d9af6eb9e68b4cea41" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:45.647359 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f22e4d34cde24b37a025c1b5a1046b1b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:45.648602 1168 tablet_bootstrap.cc:492] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:45.648763 1168 ts_tablet_manager.cc:1403] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.004s
W20251211 20:52:45.649271 849 thread.cc:641] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.654s user 0.000s sys 0.012s
I20251211 20:52:45.649425 887 tablet_service.cc:1505] Processing CreateTablet for tablet a87fe058d1de4781b24f02424eaeeccd (DEFAULT_TABLE table=default.Table_d87c2c33f4ff4aee9ff82b0653d4cc43 [id=2876f1fc09634b03afe5664e82c017d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.649523 1168 raft_consensus.cc:359] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.649711 1168 raft_consensus.cc:385] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.649767 1168 raft_consensus.cc:740] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.649966 1168 consensus_queue.cc:260] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.650470 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11af6cfd71a94e768c66b36862b048bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.650650 1168 ts_tablet_manager.cc:1434] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:45.650911 1168 tablet_bootstrap.cc:492] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:45.652884 1168 tablet_bootstrap.cc:654] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.653421 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a1016ef8bf94d2e95d4774df44f01f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.653421 905 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75dd170c5828409fa69bd68fdf28a72b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:45.654953 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbe3529c301a4a65b58b634a443e1fb6" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:45.656960 2501 raft_consensus.cc:493] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.657115 2501 raft_consensus.cc:515] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.657739 2501 leader_election.cc:290] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.663219 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3621735fcb294e3a8c0180ba9ef86442" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:45.663877 710 leader_election.cc:304] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.664176 2675 raft_consensus.cc:2804] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:45.664194 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "373d3f03d7e0408cbe06955f8f60b16c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:45.664280 2675 raft_consensus.cc:697] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:45.664515 2675 consensus_queue.cc:237] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.665863 710 leader_election.cc:304] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.666189 710 leader_election.cc:304] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.666406 2736 raft_consensus.cc:2804] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:45.666497 2736 raft_consensus.cc:697] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:45.666707 2736 consensus_queue.cc:237] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.667789 2678 raft_consensus.cc:2804] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:45.667918 2678 raft_consensus.cc:697] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:45.668133 2678 consensus_queue.cc:237] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.668402 1166 tablet_bootstrap.cc:492] T fd43a27db69044b68826cda87e16a086 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:45.668538 1166 ts_tablet_manager.cc:1403] T fd43a27db69044b68826cda87e16a086 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.036s user 0.001s sys 0.001s
I20251211 20:52:45.669088 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "46cb1d16a0e94b3a833f32c635213fd2" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:45.669236 1166 raft_consensus.cc:359] T fd43a27db69044b68826cda87e16a086 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.669380 1166 raft_consensus.cc:385] T fd43a27db69044b68826cda87e16a086 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.669430 1166 raft_consensus.cc:740] T fd43a27db69044b68826cda87e16a086 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.669592 1166 consensus_queue.cc:260] T fd43a27db69044b68826cda87e16a086 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.669245 1049 raft_consensus.cc:2468] T 46cb1d16a0e94b3a833f32c635213fd2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:45.670084 1166 ts_tablet_manager.cc:1434] T fd43a27db69044b68826cda87e16a086 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251211 20:52:45.670703 1166 tablet_bootstrap.cc:492] T 845b5ac4a5984c0cb4ab188925103dc9 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:45.673637 1166 tablet_bootstrap.cc:654] T 845b5ac4a5984c0cb4ab188925103dc9 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.677716 1168 tablet_bootstrap.cc:492] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:45.677868 1168 ts_tablet_manager.cc:1403] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.027s user 0.007s sys 0.003s
I20251211 20:52:45.678325 1166 tablet_bootstrap.cc:492] T 845b5ac4a5984c0cb4ab188925103dc9 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:45.678463 1166 ts_tablet_manager.cc:1403] T 845b5ac4a5984c0cb4ab188925103dc9 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20251211 20:52:45.678714 1168 raft_consensus.cc:359] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.678896 1168 raft_consensus.cc:385] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.678942 1168 raft_consensus.cc:740] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.679131 1168 consensus_queue.cc:260] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.679143 1166 raft_consensus.cc:359] T 845b5ac4a5984c0cb4ab188925103dc9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.679309 1166 raft_consensus.cc:385] T 845b5ac4a5984c0cb4ab188925103dc9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.679363 1166 raft_consensus.cc:740] T 845b5ac4a5984c0cb4ab188925103dc9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.679664 1166 consensus_queue.cc:260] T 845b5ac4a5984c0cb4ab188925103dc9 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.679874 1168 ts_tablet_manager.cc:1434] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:45.680333 1168 tablet_bootstrap.cc:492] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:45.680668 1166 ts_tablet_manager.cc:1434] T 845b5ac4a5984c0cb4ab188925103dc9 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:45.681030 1166 tablet_bootstrap.cc:492] T 94658fcf512a48d0a992a7e2e0f7c176 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:45.682992 1166 tablet_bootstrap.cc:654] T 94658fcf512a48d0a992a7e2e0f7c176 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.683071 1168 tablet_bootstrap.cc:654] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.686508 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a87fe058d1de4781b24f02424eaeeccd. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:45.694597 982 leader_election.cc:343] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:45.694818 982 leader_election.cc:343] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.695465 710 leader_election.cc:304] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.696115 1166 tablet_bootstrap.cc:492] T 94658fcf512a48d0a992a7e2e0f7c176 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:45.696239 1166 ts_tablet_manager.cc:1403] T 94658fcf512a48d0a992a7e2e0f7c176 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.000s
I20251211 20:52:45.696218 2494 raft_consensus.cc:2804] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:45.696311 2494 raft_consensus.cc:493] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:45.696369 2494 raft_consensus.cc:3060] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.697124 1166 raft_consensus.cc:359] T 94658fcf512a48d0a992a7e2e0f7c176 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.697280 1166 raft_consensus.cc:385] T 94658fcf512a48d0a992a7e2e0f7c176 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.697321 1166 raft_consensus.cc:740] T 94658fcf512a48d0a992a7e2e0f7c176 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.697515 1166 consensus_queue.cc:260] T 94658fcf512a48d0a992a7e2e0f7c176 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.696233 1168 tablet_bootstrap.cc:492] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:45.698132 1168 ts_tablet_manager.cc:1403] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.000s
I20251211 20:52:45.698907 2494 raft_consensus.cc:515] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.699011 1166 ts_tablet_manager.cc:1434] T 94658fcf512a48d0a992a7e2e0f7c176 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:45.699519 2494 leader_election.cc:290] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.699692 1166 tablet_bootstrap.cc:492] T 79cc81e91e1846038621af7a0a9f7a2e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:45.699926 1168 raft_consensus.cc:359] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.700783 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 1a1016ef8bf94d2e95d4774df44f01f1 (DEFAULT_TABLE table=default.Table_b1d96cf6ac344427a92df55d1dfcee90 [id=679d27d782fb480086443c78df21fa52]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.701481 754 tablet_service.cc:1505] Processing CreateTablet for tablet d539d7e75d6d4e2b8486b134f971acfc (DEFAULT_TABLE table=default.Table_c1fc29564f5e4ee5b8f8cf8b19c744f0 [id=7a100133ee30444794457344356629ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.702414 1168 raft_consensus.cc:385] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.702486 1168 raft_consensus.cc:740] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.701761 757 tablet_service.cc:1505] Processing CreateTablet for tablet f4e9123ee4124d7dad8eb19b577d2664 (DEFAULT_TABLE table=default.Table_655c466dc1b54183b7e91ddeca33ae5e [id=be92c660569a46e7981df62185f0cbfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.702698 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a1016ef8bf94d2e95d4774df44f01f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.702669 1168 consensus_queue.cc:260] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.702680 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d539d7e75d6d4e2b8486b134f971acfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.703152 1168 ts_tablet_manager.cc:1434] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251211 20:52:45.703655 1168 tablet_bootstrap.cc:492] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:45.691147 913 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.681s user 0.004s sys 0.008s
I20251211 20:52:45.705217 1168 tablet_bootstrap.cc:654] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.707715 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4e9123ee4124d7dad8eb19b577d2664. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:45.709158 847 leader_election.cc:343] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 2d849c62f7434099a16f62c2a2fc5e85
W20251211 20:52:45.709878 913 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.762s user 0.005s sys 0.008s
I20251211 20:52:45.710687 1166 tablet_bootstrap.cc:654] T 79cc81e91e1846038621af7a0a9f7a2e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.713243 1166 tablet_bootstrap.cc:492] T 79cc81e91e1846038621af7a0a9f7a2e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:45.713445 1166 ts_tablet_manager.cc:1403] T 79cc81e91e1846038621af7a0a9f7a2e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.002s
I20251211 20:52:45.714509 1166 raft_consensus.cc:359] T 79cc81e91e1846038621af7a0a9f7a2e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.714684 1166 raft_consensus.cc:385] T 79cc81e91e1846038621af7a0a9f7a2e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.714735 1166 raft_consensus.cc:740] T 79cc81e91e1846038621af7a0a9f7a2e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.714979 1166 consensus_queue.cc:260] T 79cc81e91e1846038621af7a0a9f7a2e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.716058 1166 ts_tablet_manager.cc:1434] T 79cc81e91e1846038621af7a0a9f7a2e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
W20251211 20:52:45.716014 847 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.1:42177, user_credentials={real_user=slave}} blocked reactor thread for 82271.3us
I20251211 20:52:45.716650 1166 tablet_bootstrap.cc:492] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:45.717638 909 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.666s user 0.012s sys 0.002s
I20251211 20:52:45.718442 1166 tablet_bootstrap.cc:654] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.718434 913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.104.1:38217 (request call id 540) took 1442 ms. Trace:
I20251211 20:52:45.718533 913 rpcz_store.cc:276] 1211 20:52:44.267549 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:44.267991 (+ 442us) service_pool.cc:225] Handling call
1211 20:52:44.268116 (+ 125us) raft_consensus.cc:1471] Updating replica for 1 ops
1211 20:52:44.941319 (+673203us) raft_consensus.cc:1534] Early marking committed up to index 0
1211 20:52:44.941324 (+ 5us) raft_consensus.cc:1539] Triggering prepare for 1 ops
1211 20:52:44.941497 (+ 173us) log.cc:844] Serialized 26 byte log entry
1211 20:52:45.710013 (+768516us) raft_consensus.cc:1645] Marking committed up to 1
1211 20:52:45.710125 (+ 112us) log.cc:844] Serialized 14 byte log entry
1211 20:52:45.710184 (+ 59us) raft_consensus.cc:1697] Filling consensus response to leader.
1211 20:52:45.710210 (+ 26us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
1211 20:52:45.710219 (+ 9us) raft_consensus.cc:1685] finished
1211 20:52:45.710221 (+ 2us) raft_consensus.cc:1691] UpdateReplicas() finished
1211 20:52:45.710258 (+ 37us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":234,"prepare.run_cpu_time_us":12,"prepare.run_wall_time_us":14,"raft.queue_time_us":3628,"raft.run_cpu_time_us":50,"raft.run_wall_time_us":51,"thread_start_us":1047834,"threads_started":2,"wal-append.queue_time_us":582597}
I20251211 20:52:45.721174 909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.104.3:33817 (request call id 193) took 1580 ms. Trace:
I20251211 20:52:45.721444 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "46cb1d16a0e94b3a833f32c635213fd2" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:45.721618 1049 raft_consensus.cc:3060] T 46cb1d16a0e94b3a833f32c635213fd2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.722180 1166 tablet_bootstrap.cc:492] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:45.722286 1166 ts_tablet_manager.cc:1403] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20251211 20:52:45.722992 1166 raft_consensus.cc:359] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.723130 1166 raft_consensus.cc:385] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.723172 1166 raft_consensus.cc:740] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.723366 1166 consensus_queue.cc:260] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.724236 2764 raft_consensus.cc:493] T a7979798e92e45eb969197c8c7ac2580 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.724277 757 tablet_service.cc:1505] Processing CreateTablet for tablet 77bd176108f24e79a632c798566e9e31 (DEFAULT_TABLE table=default.Table_1021d997007b41208355c31d22f4e26c [id=8c644777215b4617a8ce57691c34ce8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.724345 2764 raft_consensus.cc:515] T a7979798e92e45eb969197c8c7ac2580 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.724419 1049 raft_consensus.cc:2468] T 46cb1d16a0e94b3a833f32c635213fd2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:45.724706 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77bd176108f24e79a632c798566e9e31. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.724896 2764 leader_election.cc:290] T a7979798e92e45eb969197c8c7ac2580 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:45.725136 847 leader_election.cc:304] T c7147571483e41ad970e573a08c368ca P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:45.725378 1166 ts_tablet_manager.cc:1434] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
W20251211 20:52:45.725586 847 leader_election.cc:343] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 2d849c62f7434099a16f62c2a2fc5e85
I20251211 20:52:45.726568 741 tablet_service.cc:1505] Processing CreateTablet for tablet de61a3b7ea634a9d9f329db19f74b630 (DEFAULT_TABLE table=default.Table_087c001a257049babda598744c761be8 [id=0e1a601c016d4b0e8e7d05969a4ea590]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:45.726779 849 thread.cc:608] client-negotiator [worker] (thread pool) Time spent starting thread: real 0.680s user 0.000s sys 0.012s
I20251211 20:52:45.726955 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de61a3b7ea634a9d9f329db19f74b630. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.727198 2670 raft_consensus.cc:493] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6f8d784fd1cd492bbe3809403ec34919)
I20251211 20:52:45.731187 847 leader_election.cc:304] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
W20251211 20:52:45.731685 847 leader_election.cc:343] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 4df68d1b6a91458e9f212ae81aa25438
I20251211 20:52:45.732414 2696 raft_consensus.cc:2749] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:45.733206 739 tablet_service.cc:1505] Processing CreateTablet for tablet bc6a1c84501d4e179cd2828b50fddc23 (DEFAULT_TABLE table=default.Table_8bb5b228d60e4d29a41d9ba40c33d45d [id=6ef058fada9941629e0ea4d1906abdac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.733668 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc6a1c84501d4e179cd2828b50fddc23. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.734592 754 tablet_service.cc:1505] Processing CreateTablet for tablet 40475640009142439755cba2f870822c (DEFAULT_TABLE table=default.Table_37fd7b6c68b64deaa3fb0ac9a1965eff [id=05826ee49d07424aa08e105c165c497c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.734934 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40475640009142439755cba2f870822c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.740190 1166 tablet_bootstrap.cc:492] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:45.741936 1166 tablet_bootstrap.cc:654] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.744165 2460 raft_consensus.cc:493] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.750352 2429 consensus_queue.cc:1048] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:45.751631 2429 consensus_queue.cc:1048] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:45.752239 710 leader_election.cc:304] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.752472 2429 raft_consensus.cc:2804] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:45.752557 2429 raft_consensus.cc:697] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:45.752933 2429 consensus_queue.cc:237] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.756114 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a7979798e92e45eb969197c8c7ac2580" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:45.757362 741 tablet_service.cc:1505] Processing CreateTablet for tablet 4acbc679266b4efd815e00a855751ace (DEFAULT_TABLE table=default.Table_d44ac0044b77421f88b8a7b00811c8d3 [id=9800f52437ab4fb6b5a46e4861154750]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.757762 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4acbc679266b4efd815e00a855751ace. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.761132 1015 tablet_service.cc:1505] Processing CreateTablet for tablet d539d7e75d6d4e2b8486b134f971acfc (DEFAULT_TABLE table=default.Table_c1fc29564f5e4ee5b8f8cf8b19c744f0 [id=7a100133ee30444794457344356629ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.761556 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d539d7e75d6d4e2b8486b134f971acfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.769433 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 77bd176108f24e79a632c798566e9e31 (DEFAULT_TABLE table=default.Table_1021d997007b41208355c31d22f4e26c [id=8c644777215b4617a8ce57691c34ce8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.769508 1029 tablet_service.cc:1505] Processing CreateTablet for tablet de61a3b7ea634a9d9f329db19f74b630 (DEFAULT_TABLE table=default.Table_087c001a257049babda598744c761be8 [id=0e1a601c016d4b0e8e7d05969a4ea590]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.769433 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 40475640009142439755cba2f870822c (DEFAULT_TABLE table=default.Table_37fd7b6c68b64deaa3fb0ac9a1965eff [id=05826ee49d07424aa08e105c165c497c]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:45.769718 982 leader_election.cc:343] T a7979798e92e45eb969197c8c7ac2580 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.769910 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de61a3b7ea634a9d9f329db19f74b630. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.769899 1020 tablet_service.cc:1505] Processing CreateTablet for tablet bc6a1c84501d4e179cd2828b50fddc23 (DEFAULT_TABLE table=default.Table_8bb5b228d60e4d29a41d9ba40c33d45d [id=6ef058fada9941629e0ea4d1906abdac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.770169 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40475640009142439755cba2f870822c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.770340 1021 tablet_service.cc:1505] Processing CreateTablet for tablet 9527db28329245e7b8343e69d3881123 (DEFAULT_TABLE table=default.Table_b9de49d730424e2b863b64535f2ba732 [id=fd5a2aedcd31426394ad1ffdd0936893]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.770622 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9527db28329245e7b8343e69d3881123. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.775914 1013 tablet_service.cc:1505] Processing CreateTablet for tablet 4acbc679266b4efd815e00a855751ace (DEFAULT_TABLE table=default.Table_d44ac0044b77421f88b8a7b00811c8d3 [id=9800f52437ab4fb6b5a46e4861154750]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.769542 1022 tablet_service.cc:1505] Processing CreateTablet for tablet f4e9123ee4124d7dad8eb19b577d2664 (DEFAULT_TABLE table=default.Table_655c466dc1b54183b7e91ddeca33ae5e [id=be92c660569a46e7981df62185f0cbfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.776289 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4acbc679266b4efd815e00a855751ace. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.776481 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4e9123ee4124d7dad8eb19b577d2664. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.777977 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc6a1c84501d4e179cd2828b50fddc23. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.781733 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77bd176108f24e79a632c798566e9e31. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.789458 870 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:43876 (request call id 164) took 1469 ms. Trace:
I20251211 20:52:45.796273 739 tablet_service.cc:1505] Processing CreateTablet for tablet 9527db28329245e7b8343e69d3881123 (DEFAULT_TABLE table=default.Table_b9de49d730424e2b863b64535f2ba732 [id=fd5a2aedcd31426394ad1ffdd0936893]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.796892 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9527db28329245e7b8343e69d3881123. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.805248 1168 tablet_bootstrap.cc:492] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:45.805390 1168 ts_tablet_manager.cc:1403] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.102s user 0.006s sys 0.006s
I20251211 20:52:45.806141 1168 raft_consensus.cc:359] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.806317 1168 raft_consensus.cc:385] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.806375 1168 raft_consensus.cc:740] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.806548 1168 consensus_queue.cc:260] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.808368 1168 ts_tablet_manager.cc:1434] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:45.809011 1168 tablet_bootstrap.cc:492] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:45.810693 1168 tablet_bootstrap.cc:654] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.813907 2764 raft_consensus.cc:493] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.814064 2764 raft_consensus.cc:515] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.814673 2764 leader_election.cc:290] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:45.816365 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "58e35de0792a464a818170886ca8d8c9" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:45.816891 982 leader_election.cc:343] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.818894 2764 raft_consensus.cc:493] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.819042 2764 raft_consensus.cc:515] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.819700 2764 leader_election.cc:290] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:45.820389 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "06b8f8e46e524545aeab6c931efaf116" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:45.820897 982 leader_election.cc:343] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.821401 1168 tablet_bootstrap.cc:492] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:45.821538 1168 ts_tablet_manager.cc:1403] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.001s
I20251211 20:52:45.822360 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 4c0455a5b4f44412803475d1c4852b48 (DEFAULT_TABLE table=default.Table_e00092673b0e4f09b8d7eec63e1ee034 [id=18497e06d7254441a4655ce1cdc5a9bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.822780 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c0455a5b4f44412803475d1c4852b48. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.823189 741 tablet_service.cc:1505] Processing CreateTablet for tablet 4c0455a5b4f44412803475d1c4852b48 (DEFAULT_TABLE table=default.Table_e00092673b0e4f09b8d7eec63e1ee034 [id=18497e06d7254441a4655ce1cdc5a9bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.823518 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c0455a5b4f44412803475d1c4852b48. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:45.824863 983 leader_election.cc:343] T ce1799e6604f4ed7875ddedecab6f327 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.824977 983 leader_election.cc:304] T ce1799e6604f4ed7875ddedecab6f327 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:45.825194 2764 raft_consensus.cc:2749] T ce1799e6604f4ed7875ddedecab6f327 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:45.825975 711 consensus_peers.cc:597] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:45.826186 2494 raft_consensus.cc:493] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.826293 2500 raft_consensus.cc:493] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.826295 2494 raft_consensus.cc:515] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.826426 2500 raft_consensus.cc:515] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:45.826849 711 leader_election.cc:343] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.826933 2494 leader_election.cc:290] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.827631 2494 consensus_queue.cc:1048] T 5262c2241e524395a73c0eb368917169 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:45.827700 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 21cb988566a247d0b9ac9b337b1188fb (DEFAULT_TABLE table=default.Table_11a7f002cdb8441eabe140368c795751 [id=d25542cf6b174800a1c0d46f07397bfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.828091 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21cb988566a247d0b9ac9b337b1188fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.826937 2500 leader_election.cc:290] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.828783 754 tablet_service.cc:1505] Processing CreateTablet for tablet 21cb988566a247d0b9ac9b337b1188fb (DEFAULT_TABLE table=default.Table_11a7f002cdb8441eabe140368c795751 [id=d25542cf6b174800a1c0d46f07397bfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.829139 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21cb988566a247d0b9ac9b337b1188fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.829743 1037 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0aac6fb3972e4aba8950828c9e49d507" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:45.801164 871 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:43876 (request call id 163) took 1485 ms. Trace:
W20251211 20:52:45.802834 847 consensus_peers.cc:597] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7d4bba0aa7ea403e8dfcb74cec2e1f9b. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:45.803752 2647 raft_consensus.cc:697] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41 [term 1 LEADER]: Becoming Leader. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Running, Role: LEADER
I20251211 20:52:45.829932 1037 raft_consensus.cc:2468] T 0aac6fb3972e4aba8950828c9e49d507 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:45.803886 1160 tablet_bootstrap.cc:492] T f3855ffb51a74101b26fd28429e65bc2 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:45.804371 894 tablet_service.cc:1505] Processing CreateTablet for tablet f4e9123ee4124d7dad8eb19b577d2664 (DEFAULT_TABLE table=default.Table_655c466dc1b54183b7e91ddeca33ae5e [id=be92c660569a46e7981df62185f0cbfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.808579 2692 raft_consensus.cc:493] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.808717 2460 raft_consensus.cc:515] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.830365 1037 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1799e6604f4ed7875ddedecab6f327" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:45.830502 1037 raft_consensus.cc:2468] T ce1799e6604f4ed7875ddedecab6f327 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:45.816726 2670 raft_consensus.cc:515] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.831305 710 leader_election.cc:304] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.805620 909 rpcz_store.cc:276] 1211 20:52:44.137610 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:44.137684 (+ 74us) service_pool.cc:225] Handling call
1211 20:52:44.137859 (+ 175us) raft_consensus.cc:1471] Updating replica for 1 ops
1211 20:52:44.673722 (+535863us) raft_consensus.cc:1534] Early marking committed up to index 0
1211 20:52:44.673728 (+ 6us) raft_consensus.cc:1539] Triggering prepare for 1 ops
1211 20:52:44.673920 (+ 192us) log.cc:844] Serialized 26 byte log entry
1211 20:52:44.941683 (+267763us) raft_consensus.cc:1645] Marking committed up to 1
1211 20:52:45.717875 (+776192us) log.cc:844] Serialized 14 byte log entry
1211 20:52:45.717976 (+ 101us) raft_consensus.cc:1697] Filling consensus response to leader.
1211 20:52:45.718094 (+ 118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
1211 20:52:45.718105 (+ 11us) raft_consensus.cc:1685] finished
1211 20:52:45.718107 (+ 2us) raft_consensus.cc:1691] UpdateReplicas() finished
1211 20:52:45.718158 (+ 51us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":13970,"prepare.queue_time_us":14027,"prepare.run_cpu_time_us":4,"prepare.run_wall_time_us":4,"raft.queue_time_us":3730,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"thread_start_us":1016571,"threads_started":2,"wal-append.queue_time_us":264590}
I20251211 20:52:45.820003 913 raft_consensus.cc:3060] T 2e7e158809b9471fbd6e6ee9fb9ac569 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.820151 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "46cb1d16a0e94b3a833f32c635213fd2" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:45.831837 710 leader_election.cc:304] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.820744 905 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "46cb1d16a0e94b3a833f32c635213fd2" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:45.827682 874 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:43876 (request call id 165) took 1499 ms. Trace:
I20251211 20:52:45.827766 886 tablet_service.cc:1505] Processing CreateTablet for tablet d539d7e75d6d4e2b8486b134f971acfc (DEFAULT_TABLE table=default.Table_c1fc29564f5e4ee5b8f8cf8b19c744f0 [id=7a100133ee30444794457344356629ce]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:45.832376 711 consensus_peers.cc:597] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:45.832530 2429 raft_consensus.cc:2804] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:45.832599 2429 raft_consensus.cc:493] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:45.832669 2429 raft_consensus.cc:3060] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.832628 883 tablet_service.cc:1505] Processing CreateTablet for tablet 77bd176108f24e79a632c798566e9e31 (DEFAULT_TABLE table=default.Table_1021d997007b41208355c31d22f4e26c [id=8c644777215b4617a8ce57691c34ce8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.832957 1160 ts_tablet_manager.cc:1403] T f3855ffb51a74101b26fd28429e65bc2 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.917s user 0.002s sys 0.014s
I20251211 20:52:45.833465 2675 raft_consensus.cc:2804] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:45.833649 2675 raft_consensus.cc:493] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:45.833799 2675 raft_consensus.cc:3060] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.835038 2429 raft_consensus.cc:515] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.836181 1168 raft_consensus.cc:359] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.835624 2647 consensus_queue.cc:237] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.836355 1168 raft_consensus.cc:385] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.836408 1168 raft_consensus.cc:740] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.836588 1168 consensus_queue.cc:260] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.837268 1168 ts_tablet_manager.cc:1434] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.016s user 0.001s sys 0.001s
I20251211 20:52:45.837268 2692 raft_consensus.cc:515] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.837527 1168 tablet_bootstrap.cc:492] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:45.838021 847 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 98451.7us
I20251211 20:52:45.838690 2670 leader_election.cc:290] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.839008 1037 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0aac6fb3972e4aba8950828c9e49d507" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:45.839184 1037 raft_consensus.cc:3060] T 0aac6fb3972e4aba8950828c9e49d507 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.839278 2460 leader_election.cc:290] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.839545 1168 tablet_bootstrap.cc:654] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.841871 1037 raft_consensus.cc:2468] T 0aac6fb3972e4aba8950828c9e49d507 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:45.842517 710 leader_election.cc:304] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:45.843016 2764 raft_consensus.cc:493] T fd43a27db69044b68826cda87e16a086 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.843147 2764 raft_consensus.cc:515] T fd43a27db69044b68826cda87e16a086 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.843778 2764 leader_election.cc:290] T fd43a27db69044b68826cda87e16a086 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:45.844434 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd43a27db69044b68826cda87e16a086" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:45.844973 982 leader_election.cc:343] T fd43a27db69044b68826cda87e16a086 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.850293 2736 raft_consensus.cc:2804] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:45.850551 2736 raft_consensus.cc:697] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:45.850853 2736 consensus_queue.cc:237] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.851078 891 tablet_service.cc:1505] Processing CreateTablet for tablet bc6a1c84501d4e179cd2828b50fddc23 (DEFAULT_TABLE table=default.Table_8bb5b228d60e4d29a41d9ba40c33d45d [id=6ef058fada9941629e0ea4d1906abdac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.852054 1168 tablet_bootstrap.cc:492] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:45.852190 1168 ts_tablet_manager.cc:1403] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.001s
I20251211 20:52:45.852152 2759 raft_consensus.cc:2749] T c7147571483e41ad970e573a08c368ca P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:45.852955 881 tablet_service.cc:1505] Processing CreateTablet for tablet de61a3b7ea634a9d9f329db19f74b630 (DEFAULT_TABLE table=default.Table_087c001a257049babda598744c761be8 [id=0e1a601c016d4b0e8e7d05969a4ea590]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.853981 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4e9123ee4124d7dad8eb19b577d2664. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.854343 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de61a3b7ea634a9d9f329db19f74b630. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.854462 1168 raft_consensus.cc:359] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.854636 1168 raft_consensus.cc:385] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.854707 1168 raft_consensus.cc:740] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.854914 1168 consensus_queue.cc:260] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.855167 757 tablet_service.cc:1505] Processing CreateTablet for tablet acc952c98f414a05aa863372de1ab397 (DEFAULT_TABLE table=default.Table_ee37a9ef24e548ebb2b6b55b54fa6f91 [id=162bfc57131c46c9bbf360299e331097]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.855520 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acc952c98f414a05aa863372de1ab397. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.855686 1015 tablet_service.cc:1505] Processing CreateTablet for tablet acc952c98f414a05aa863372de1ab397 (DEFAULT_TABLE table=default.Table_ee37a9ef24e548ebb2b6b55b54fa6f91 [id=162bfc57131c46c9bbf360299e331097]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.856063 1168 ts_tablet_manager.cc:1434] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:45.856112 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acc952c98f414a05aa863372de1ab397. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.856336 1168 tablet_bootstrap.cc:492] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:45.857867 1168 tablet_bootstrap.cc:654] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.859731 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc6a1c84501d4e179cd2828b50fddc23. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.863525 2764 raft_consensus.cc:493] T 845b5ac4a5984c0cb4ab188925103dc9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.863700 2764 raft_consensus.cc:515] T 845b5ac4a5984c0cb4ab188925103dc9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.864266 2764 leader_election.cc:290] T 845b5ac4a5984c0cb4ab188925103dc9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:45.864558 1168 tablet_bootstrap.cc:492] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:45.864693 1168 ts_tablet_manager.cc:1403] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.001s
I20251211 20:52:45.865217 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "845b5ac4a5984c0cb4ab188925103dc9" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:45.865455 1168 raft_consensus.cc:359] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.865628 1168 raft_consensus.cc:385] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.865680 1168 raft_consensus.cc:740] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
W20251211 20:52:45.865751 982 leader_election.cc:343] T 845b5ac4a5984c0cb4ab188925103dc9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.865854 1168 consensus_queue.cc:260] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.866426 1168 ts_tablet_manager.cc:1434] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:45.866693 1168 tablet_bootstrap.cc:492] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:45.868302 1168 tablet_bootstrap.cc:654] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.870052 1096 heartbeater.cc:499] Master 127.0.104.61:40749 was elected leader, sending a full tablet report...
I20251211 20:52:45.870543 757 tablet_service.cc:1505] Processing CreateTablet for tablet 3c865c99e9314f5f91658b9126da4d39 (DEFAULT_TABLE table=default.Table_0d7df3fba38849a49b65d8290812b95c [id=7f966d5cdad3423bb770414802a3cf7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.871088 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c865c99e9314f5f91658b9126da4d39. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.871748 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77bd176108f24e79a632c798566e9e31. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.877918 1168 tablet_bootstrap.cc:492] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:45.878093 1168 ts_tablet_manager.cc:1403] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20251211 20:52:45.878962 1168 raft_consensus.cc:359] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.879160 1168 raft_consensus.cc:385] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.879216 1168 raft_consensus.cc:740] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.880023 1168 consensus_queue.cc:260] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.880527 1168 ts_tablet_manager.cc:1434] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:45.881526 890 tablet_service.cc:1505] Processing CreateTablet for tablet 40475640009142439755cba2f870822c (DEFAULT_TABLE table=default.Table_37fd7b6c68b64deaa3fb0ac9a1965eff [id=05826ee49d07424aa08e105c165c497c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.876166 2759 raft_consensus.cc:493] T 2e7e158809b9471fbd6e6ee9fb9ac569 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.882632 2759 raft_consensus.cc:515] T 2e7e158809b9471fbd6e6ee9fb9ac569 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.883201 756 tablet_service.cc:1505] Processing CreateTablet for tablet b65b142f801e4111ad153164dfe9b211 (DEFAULT_TABLE table=default.Table_0910ec0ec3e64c139587947350a39ae5 [id=4ce959831e384a14854ce5570dededaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.883330 2759 leader_election.cc:290] T 2e7e158809b9471fbd6e6ee9fb9ac569 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.883567 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65b142f801e4111ad153164dfe9b211. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.884162 1168 tablet_bootstrap.cc:492] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:45.885839 1168 tablet_bootstrap.cc:654] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.886116 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40475640009142439755cba2f870822c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.886785 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d539d7e75d6d4e2b8486b134f971acfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.895327 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 3c865c99e9314f5f91658b9126da4d39 (DEFAULT_TABLE table=default.Table_0d7df3fba38849a49b65d8290812b95c [id=7f966d5cdad3423bb770414802a3cf7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.895902 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c865c99e9314f5f91658b9126da4d39. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.896306 872 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:43876 (request call id 167) took 1241 ms. Trace:
I20251211 20:52:45.901376 2678 consensus_queue.cc:1048] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:45.905366 2678 consensus_queue.cc:1048] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:45.906703 2736 consensus_queue.cc:1048] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:45.909612 2764 raft_consensus.cc:493] T 7be44df39a8f4d6da793479d425fa684 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.909775 2764 raft_consensus.cc:515] T 7be44df39a8f4d6da793479d425fa684 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.910400 2764 leader_election.cc:290] T 7be44df39a8f4d6da793479d425fa684 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:45.911300 2764 raft_consensus.cc:493] T 44e2d2bdecf247baaac26e06bccc94de P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.911311 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7be44df39a8f4d6da793479d425fa684" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:45.911444 2764 raft_consensus.cc:515] T 44e2d2bdecf247baaac26e06bccc94de P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.912035 2764 leader_election.cc:290] T 44e2d2bdecf247baaac26e06bccc94de P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
W20251211 20:52:45.912626 982 leader_election.cc:343] T 7be44df39a8f4d6da793479d425fa684 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.913137 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "44e2d2bdecf247baaac26e06bccc94de" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:45.913615 982 leader_election.cc:343] T 44e2d2bdecf247baaac26e06bccc94de P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:45.918632 1019 tablet_service.cc:1505] Processing CreateTablet for tablet b65b142f801e4111ad153164dfe9b211 (DEFAULT_TABLE table=default.Table_0910ec0ec3e64c139587947350a39ae5 [id=4ce959831e384a14854ce5570dededaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.919076 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65b142f801e4111ad153164dfe9b211. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.923005 2675 raft_consensus.cc:515] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.924433 2675 leader_election.cc:290] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.924505 1043 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1799e6604f4ed7875ddedecab6f327" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:45.924760 1043 raft_consensus.cc:3060] T ce1799e6604f4ed7875ddedecab6f327 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.925179 1022 tablet_service.cc:1505] Processing CreateTablet for tablet d1391fcd70ab420bbfcb30738a871b6f (DEFAULT_TABLE table=default.Table_d15e45922b4340a59fce3b9dc10faf71 [id=8e1b3fd5c43a446ab4cd6a8a4b4ee8e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.925554 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1391fcd70ab420bbfcb30738a871b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.927345 1043 raft_consensus.cc:2468] T ce1799e6604f4ed7875ddedecab6f327 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:45.931437 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 13269a20421240d1a98215129443248b (DEFAULT_TABLE table=default.Table_d887c953026046dc883ede774a58fd8e [id=1d8a79f716a0447cb0393f4759178598]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.931829 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13269a20421240d1a98215129443248b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.935907 1013 tablet_service.cc:1505] Processing CreateTablet for tablet 1722730c38b344ffb1ff3347e314f197 (DEFAULT_TABLE table=default.Table_e870d2f987ff44e6b91ebe0205f6fe9e [id=53b3365f07864ac1a3cd47482bb6377a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.936305 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1722730c38b344ffb1ff3347e314f197. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.938386 1166 tablet_bootstrap.cc:492] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:45.938505 1166 ts_tablet_manager.cc:1403] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.198s user 0.009s sys 0.038s
I20251211 20:52:45.897346 889 tablet_service.cc:1505] Processing CreateTablet for tablet 21cb988566a247d0b9ac9b337b1188fb (DEFAULT_TABLE table=default.Table_11a7f002cdb8441eabe140368c795751 [id=d25542cf6b174800a1c0d46f07397bfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.938606 872 rpcz_store.cc:276] 1211 20:52:44.654930 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:44.662040 (+ 7110us) service_pool.cc:225] Handling call
1211 20:52:44.737956 (+ 75916us) spinlock_profiling.cc:243] Waited 69.7 ms on lock 0x616000128e54. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.026939 (+288983us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x6160001e1054. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.170566 (+143627us) spinlock_profiling.cc:243] Waited 5.31 ms on lock 0x616000105a54. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.713433 (+542867us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x6160001c0f54. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.725739 (+ 12306us) spinlock_profiling.cc:243] Waited 4.1 ms on lock 0x616000156154. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.751703 (+ 25964us) spinlock_profiling.cc:243] Waited 6.21 ms on lock 0x6160000c7f54. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.896250 (+144547us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1977504256}
I20251211 20:52:45.938906 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21cb988566a247d0b9ac9b337b1188fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.939467 1166 raft_consensus.cc:359] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.939894 1029 tablet_service.cc:1505] Processing CreateTablet for tablet b03a2f87c2c4415b98f461a559d4e960 (DEFAULT_TABLE table=default.Table_51015ac894894382b4c0d1931b24f4d4 [id=2118ffd1623f491e95ecf3719fa44130]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.939956 1166 raft_consensus.cc:385] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.940053 1166 raft_consensus.cc:740] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.940243 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b03a2f87c2c4415b98f461a559d4e960. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.940181 885 tablet_service.cc:1505] Processing CreateTablet for tablet 13269a20421240d1a98215129443248b (DEFAULT_TABLE table=default.Table_d887c953026046dc883ede774a58fd8e [id=1d8a79f716a0447cb0393f4759178598]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.940253 1166 consensus_queue.cc:260] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.940587 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13269a20421240d1a98215129443248b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.940747 1166 ts_tablet_manager.cc:1434] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:45.941006 1166 tablet_bootstrap.cc:492] T 98d575ac680d49d0990877666638f013 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:45.942512 1166 tablet_bootstrap.cc:654] T 98d575ac680d49d0990877666638f013 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.897763 884 tablet_service.cc:1505] Processing CreateTablet for tablet 3c865c99e9314f5f91658b9126da4d39 (DEFAULT_TABLE table=default.Table_0d7df3fba38849a49b65d8290812b95c [id=7f966d5cdad3423bb770414802a3cf7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.943107 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c865c99e9314f5f91658b9126da4d39. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.897831 881 tablet_service.cc:1505] Processing CreateTablet for tablet 9527db28329245e7b8343e69d3881123 (DEFAULT_TABLE table=default.Table_b9de49d730424e2b863b64535f2ba732 [id=fd5a2aedcd31426394ad1ffdd0936893]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.898062 880 tablet_service.cc:1505] Processing CreateTablet for tablet b65b142f801e4111ad153164dfe9b211 (DEFAULT_TABLE table=default.Table_0910ec0ec3e64c139587947350a39ae5 [id=4ce959831e384a14854ce5570dededaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.957664 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65b142f801e4111ad153164dfe9b211. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.958477 889 tablet_service.cc:1505] Processing CreateTablet for tablet b03a2f87c2c4415b98f461a559d4e960 (DEFAULT_TABLE table=default.Table_51015ac894894382b4c0d1931b24f4d4 [id=2118ffd1623f491e95ecf3719fa44130]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.958968 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b03a2f87c2c4415b98f461a559d4e960. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.897346 888 tablet_service.cc:1505] Processing CreateTablet for tablet acc952c98f414a05aa863372de1ab397 (DEFAULT_TABLE table=default.Table_ee37a9ef24e548ebb2b6b55b54fa6f91 [id=162bfc57131c46c9bbf360299e331097]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.963629 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acc952c98f414a05aa863372de1ab397. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.965358 2670 raft_consensus.cc:493] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.965550 2670 raft_consensus.cc:515] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.966286 2670 leader_election.cc:290] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.967120 907 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "95866de91f9d437d80f40b7962253610" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:45.898548 883 tablet_service.cc:1505] Processing CreateTablet for tablet 4acbc679266b4efd815e00a855751ace (DEFAULT_TABLE table=default.Table_d44ac0044b77421f88b8a7b00811c8d3 [id=9800f52437ab4fb6b5a46e4861154750]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.967499 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3621735fcb294e3a8c0180ba9ef86442" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:45.967733 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4acbc679266b4efd815e00a855751ace. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.968495 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "373d3f03d7e0408cbe06955f8f60b16c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:45.904315 874 rpcz_store.cc:276] 1211 20:52:44.323372 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:44.323432 (+ 60us) service_pool.cc:225] Handling call
1211 20:52:44.735459 (+412027us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x616000128e54. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.019677 (+284218us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x6160001e1054. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.274061 (+254384us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x616000105a54. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.714659 (+440598us) spinlock_profiling.cc:243] Waited 418 ms on lock 0x6160001c0f54. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.726335 (+ 11676us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x616000156154. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.759694 (+ 33359us) spinlock_profiling.cc:243] Waited 4.77 ms on lock 0x6160000c7f54. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.822319 (+ 62625us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2525036800}
I20251211 20:52:45.968977 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a7979798e92e45eb969197c8c7ac2580" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:45.914533 1160 raft_consensus.cc:359] T f3855ffb51a74101b26fd28429e65bc2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.914917 2692 leader_election.cc:290] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:45.969118 1160 raft_consensus.cc:385] T f3855ffb51a74101b26fd28429e65bc2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:45.914777 847 leader_election.cc:304] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41; no voters: e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:45.919714 873 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:43876 (request call id 166) took 1206 ms. Trace:
I20251211 20:52:45.969237 1160 raft_consensus.cc:740] T f3855ffb51a74101b26fd28429e65bc2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:45.969331 873 rpcz_store.cc:276] 1211 20:52:44.596349 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:44.596683 (+ 334us) service_pool.cc:225] Handling call
1211 20:52:44.734123 (+137440us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x616000128e54. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.017405 (+283282us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x6160001e1054. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.719218 (+701813us) spinlock_profiling.cc:243] Waited 675 ms on lock 0x6160001c0f54. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.720568 (+ 1350us) spinlock_profiling.cc:243] Waited 907 us on lock 0x616000128854. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.763683 (+ 43115us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x6160000c7f54. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.802741 (+ 39058us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2463951744}
I20251211 20:52:45.969523 1160 consensus_queue.cc:260] T f3855ffb51a74101b26fd28429e65bc2 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:45.969954 847 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.1:42177, user_credentials={real_user=slave}} blocked reactor thread for 131570us
I20251211 20:52:45.970088 1160 ts_tablet_manager.cc:1434] T f3855ffb51a74101b26fd28429e65bc2 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.137s user 0.002s sys 0.000s
I20251211 20:52:45.970412 1160 tablet_bootstrap.cc:492] T 6a61cf8f5de14adaa4a1de5bddde9a2f P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:45.972203 1160 tablet_bootstrap.cc:654] T 6a61cf8f5de14adaa4a1de5bddde9a2f P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:45.898217 891 tablet_service.cc:1505] Processing CreateTablet for tablet 4c0455a5b4f44412803475d1c4852b48 (DEFAULT_TABLE table=default.Table_e00092673b0e4f09b8d7eec63e1ee034 [id=18497e06d7254441a4655ce1cdc5a9bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.973088 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9527db28329245e7b8343e69d3881123. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.974179 1043 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2e7e158809b9471fbd6e6ee9fb9ac569" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:45.976488 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c0455a5b4f44412803475d1c4852b48. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.978134 2692 raft_consensus.cc:2804] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:45.978335 2692 raft_consensus.cc:697] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 LEADER]: Becoming Leader. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Running, Role: LEADER
I20251211 20:52:45.978621 2692 consensus_queue.cc:237] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.967183 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d43210ec61af423ebde624397dea9f2e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:45.984097 2759 raft_consensus.cc:493] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:45.984314 2759 raft_consensus.cc:515] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:45.984987 2759 leader_election.cc:290] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:45.990280 847 leader_election.cc:343] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 14e2650f8ec1482ea9511b989fdc963c
W20251211 20:52:45.990610 847 leader_election.cc:343] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: ccc3ddd4b7f74f11a3835997cb7ae8d0
W20251211 20:52:45.990868 847 leader_election.cc:343] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: e28d66ce4711430cb09a9650e99fbe9e
W20251211 20:52:45.991245 847 leader_election.cc:343] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 0b267c7feaac43f6aacb7229759c7ec3
W20251211 20:52:45.991503 847 consensus_peers.cc:597] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 260bdeb73439437cbf234d20f7264ab3. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:45.991925 847 leader_election.cc:343] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 25454e67437541d9ac2bfab29bad55f9
I20251211 20:52:45.829756 870 rpcz_store.cc:276] 1211 20:52:44.319761 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:44.319833 (+ 72us) service_pool.cc:225] Handling call
1211 20:52:44.729473 (+409640us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x616000128e54. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.023903 (+294430us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x6160001e1054. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.147790 (+123887us) spinlock_profiling.cc:243] Waited 8.43 ms on lock 0x616000105a54. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.714045 (+566255us) spinlock_profiling.cc:243] Waited 553 ms on lock 0x6160001c0f54. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.726032 (+ 11987us) spinlock_profiling.cc:243] Waited 3.53 ms on lock 0x616000156154. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.749066 (+ 23034us) spinlock_profiling.cc:243] Waited 6.2 ms on lock 0x6160000c7f54. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.789424 (+ 40358us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2837241984}
I20251211 20:52:45.914984 871 rpcz_store.cc:276] 1211 20:52:44.316060 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:44.316117 (+ 57us) service_pool.cc:225] Handling call
1211 20:52:44.731830 (+415713us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x616000128e54. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.021762 (+289932us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x6160001e1054. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.169305 (+147543us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x616000105a54. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.712677 (+543372us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x6160001c0f54. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.726190 (+ 13513us) spinlock_profiling.cc:243] Waited 4.68 ms on lock 0x616000156154. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.756511 (+ 30321us) spinlock_profiling.cc:243] Waited 5.94 ms on lock 0x6160000c7f54. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f9004e9f2c0 00007f9008d4fb2f 00007f9003174c36 00007f9003174965 00007f8ffffa556e 00007f8ffffa444e 00007f8ffffaa685 00007f8ffffaafaf 00007f8ffffaadd8 0000000000a8970c
1211 20:52:45.801099 (+ 44588us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2800788224}
I20251211 20:52:46.015226 883 tablet_service.cc:1505] Processing CreateTablet for tablet 62afbd04479f4dff87aa18bef705c6e9 (DEFAULT_TABLE table=default.Table_4b87bd2fa22f4d59b01baeca62ce98f8 [id=5a1c2ebabb5c49d5afe8f668d2bfa3e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.015394 888 tablet_service.cc:1505] Processing CreateTablet for tablet 7e89288decd14ed4b59af6d17900ec6d (DEFAULT_TABLE table=default.Table_ff22c4d4d5d34ac4bf51e3e4c97b2472 [id=925885faf3a6449faef52cc2033bfbf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.015676 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62afbd04479f4dff87aa18bef705c6e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.015987 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e89288decd14ed4b59af6d17900ec6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.018539 887 tablet_service.cc:1505] Processing CreateTablet for tablet d1391fcd70ab420bbfcb30738a871b6f (DEFAULT_TABLE table=default.Table_d15e45922b4340a59fce3b9dc10faf71 [id=8e1b3fd5c43a446ab4cd6a8a4b4ee8e1]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:45.925513 711 leader_election.cc:343] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.019253 711 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 93740.5us
I20251211 20:52:45.926062 756 tablet_service.cc:1505] Processing CreateTablet for tablet 13269a20421240d1a98215129443248b (DEFAULT_TABLE table=default.Table_d887c953026046dc883ede774a58fd8e [id=1d8a79f716a0447cb0393f4759178598]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.927938 757 tablet_service.cc:1505] Processing CreateTablet for tablet 1722730c38b344ffb1ff3347e314f197 (DEFAULT_TABLE table=default.Table_e870d2f987ff44e6b91ebe0205f6fe9e [id=53b3365f07864ac1a3cd47482bb6377a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:45.928395 710 leader_election.cc:304] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
W20251211 20:52:46.020318 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 91946us
I20251211 20:52:46.020488 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1722730c38b344ffb1ff3347e314f197. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:45.929232 747 tablet_service.cc:1505] Processing CreateTablet for tablet b03a2f87c2c4415b98f461a559d4e960 (DEFAULT_TABLE table=default.Table_51015ac894894382b4c0d1931b24f4d4 [id=2118ffd1623f491e95ecf3719fa44130]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.021270 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b03a2f87c2c4415b98f461a559d4e960. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.021576 2286 raft_consensus.cc:2804] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:46.023761 2286 raft_consensus.cc:697] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:46.024039 2764 raft_consensus.cc:493] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.024030 2286 consensus_queue.cc:237] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.024192 2764 raft_consensus.cc:515] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.024791 2764 leader_election.cc:290] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
W20251211 20:52:46.027974 711 leader_election.cc:343] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.028301 711 leader_election.cc:343] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.028503 711 leader_election.cc:343] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.028702 711 leader_election.cc:343] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.028990 711 leader_election.cc:343] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.029058 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2e7e158809b9471fbd6e6ee9fb9ac569" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:46.029620 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13269a20421240d1a98215129443248b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.032246 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5453f68789fd4e89b69e98860050865b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:46.032433 768 raft_consensus.cc:2468] T 5453f68789fd4e89b69e98860050865b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 0.
I20251211 20:52:46.033433 982 leader_election.cc:304] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:46.033902 2775 raft_consensus.cc:2804] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:46.033975 2775 raft_consensus.cc:493] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:46.034030 2775 raft_consensus.cc:3060] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.036674 2775 raft_consensus.cc:515] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.037812 2775 leader_election.cc:290] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:45.937224 2736 raft_consensus.cc:493] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.038664 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5453f68789fd4e89b69e98860050865b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:46.038822 768 raft_consensus.cc:3060] T 5453f68789fd4e89b69e98860050865b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:45.996506 741 tablet_service.cc:1505] Processing CreateTablet for tablet d1391fcd70ab420bbfcb30738a871b6f (DEFAULT_TABLE table=default.Table_d15e45922b4340a59fce3b9dc10faf71 [id=8e1b3fd5c43a446ab4cd6a8a4b4ee8e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.012563 754 tablet_service.cc:1505] Processing CreateTablet for tablet 62afbd04479f4dff87aa18bef705c6e9 (DEFAULT_TABLE table=default.Table_4b87bd2fa22f4d59b01baeca62ce98f8 [id=5a1c2ebabb5c49d5afe8f668d2bfa3e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.014665 739 tablet_service.cc:1505] Processing CreateTablet for tablet 7e89288decd14ed4b59af6d17900ec6d (DEFAULT_TABLE table=default.Table_ff22c4d4d5d34ac4bf51e3e4c97b2472 [id=925885faf3a6449faef52cc2033bfbf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.038805 2736 raft_consensus.cc:515] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.039734 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1391fcd70ab420bbfcb30738a871b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.039907 2736 leader_election.cc:290] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.041455 768 raft_consensus.cc:2468] T 5453f68789fd4e89b69e98860050865b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 1.
I20251211 20:52:46.042101 982 leader_election.cc:304] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:46.042459 2764 raft_consensus.cc:2804] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:46.042564 2764 raft_consensus.cc:697] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 LEADER]: Becoming Leader. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Running, Role: LEADER
I20251211 20:52:46.042814 2764 consensus_queue.cc:237] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.045719 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e89288decd14ed4b59af6d17900ec6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.051898 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62afbd04479f4dff87aa18bef705c6e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.056432 1168 tablet_bootstrap.cc:492] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:46.056658 1168 ts_tablet_manager.cc:1403] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.173s user 0.006s sys 0.005s
I20251211 20:52:45.844442 2429 leader_election.cc:290] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.057545 1168 raft_consensus.cc:359] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.057822 1168 raft_consensus.cc:385] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.057922 1168 raft_consensus.cc:740] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.058140 1168 consensus_queue.cc:260] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.058693 1168 ts_tablet_manager.cc:1434] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:46.059021 1168 tablet_bootstrap.cc:492] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:46.062584 2692 raft_consensus.cc:493] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.064568 2692 raft_consensus.cc:515] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.065116 2692 leader_election.cc:290] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.064170 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "58e35de0792a464a818170886ca8d8c9" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.065996 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1391fcd70ab420bbfcb30738a871b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.076297 886 tablet_service.cc:1505] Processing CreateTablet for tablet 1722730c38b344ffb1ff3347e314f197 (DEFAULT_TABLE table=default.Table_e870d2f987ff44e6b91ebe0205f6fe9e [id=53b3365f07864ac1a3cd47482bb6377a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.076895 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1722730c38b344ffb1ff3347e314f197. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.077011 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 62afbd04479f4dff87aa18bef705c6e9 (DEFAULT_TABLE table=default.Table_4b87bd2fa22f4d59b01baeca62ce98f8 [id=5a1c2ebabb5c49d5afe8f668d2bfa3e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.077565 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62afbd04479f4dff87aa18bef705c6e9. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:46.080078 847 leader_election.cc:343] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 52ef1707f6f2425aafd454f303af0333
I20251211 20:52:46.086021 1015 tablet_service.cc:1505] Processing CreateTablet for tablet 7e89288decd14ed4b59af6d17900ec6d (DEFAULT_TABLE table=default.Table_ff22c4d4d5d34ac4bf51e3e4c97b2472 [id=925885faf3a6449faef52cc2033bfbf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.086566 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e89288decd14ed4b59af6d17900ec6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.093613 1168 tablet_bootstrap.cc:654] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.100131 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 171d40a8a9124f9bb1b38797fef8e5d5 (DEFAULT_TABLE table=default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.100544 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 171d40a8a9124f9bb1b38797fef8e5d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.100701 747 tablet_service.cc:1505] Processing CreateTablet for tablet 2c65926700954af7ab539f3b78805d8d (DEFAULT_TABLE table=default.Table_beb19ebcfc3b426e9aa9f70a68445f72 [id=67d5a2d2ef5a4e329c5ffd80756ef8f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.101061 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c65926700954af7ab539f3b78805d8d. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:46.084395 711 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 55013.4us
W20251211 20:52:46.087622 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 66673.9us
I20251211 20:52:46.107236 1166 tablet_bootstrap.cc:492] T 98d575ac680d49d0990877666638f013 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:46.107409 1166 ts_tablet_manager.cc:1403] T 98d575ac680d49d0990877666638f013 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.166s user 0.020s sys 0.023s
I20251211 20:52:46.107944 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 95af3eeefd084777a70736c59d487322 (DEFAULT_TABLE table=default.Table_b8e180cc27234500a1c4c7bdd2b93ac8 [id=5dfca04de96e404a802046007f6a2afb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.108282 1166 raft_consensus.cc:359] T 98d575ac680d49d0990877666638f013 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.108588 1166 raft_consensus.cc:385] T 98d575ac680d49d0990877666638f013 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.108683 1166 raft_consensus.cc:740] T 98d575ac680d49d0990877666638f013 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.108755 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 2c65926700954af7ab539f3b78805d8d (DEFAULT_TABLE table=default.Table_beb19ebcfc3b426e9aa9f70a68445f72 [id=67d5a2d2ef5a4e329c5ffd80756ef8f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.109174 756 tablet_service.cc:1505] Processing CreateTablet for tablet 03184dcc56dd4e92adc7d0ad22aa8d8b (DEFAULT_TABLE table=default.Table_68261841754847ad8d22e86c29886144 [id=697f23a617f64762bf406b386cde7473]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.109158 1166 consensus_queue.cc:260] T 98d575ac680d49d0990877666638f013 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.109571 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03184dcc56dd4e92adc7d0ad22aa8d8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.110067 1166 ts_tablet_manager.cc:1434] T 98d575ac680d49d0990877666638f013 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251211 20:52:46.110961 2813 consensus_queue.cc:1048] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:46.111867 1013 tablet_service.cc:1505] Processing CreateTablet for tablet 03184dcc56dd4e92adc7d0ad22aa8d8b (DEFAULT_TABLE table=default.Table_68261841754847ad8d22e86c29886144 [id=697f23a617f64762bf406b386cde7473]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.112582 2806 consensus_queue.cc:1048] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:46.113605 2500 raft_consensus.cc:493] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.113718 2500 raft_consensus.cc:515] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.114226 2500 leader_election.cc:290] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.115082 2764 raft_consensus.cc:493] T 94658fcf512a48d0a992a7e2e0f7c176 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.115206 2764 raft_consensus.cc:515] T 94658fcf512a48d0a992a7e2e0f7c176 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.115777 2764 leader_election.cc:290] T 94658fcf512a48d0a992a7e2e0f7c176 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
W20251211 20:52:46.116899 711 leader_election.cc:343] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.117235 711 leader_election.cc:343] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.117691 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95af3eeefd084777a70736c59d487322. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:46.118206 711 leader_election.cc:343] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.118947 711 leader_election.cc:343] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.120101 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "94658fcf512a48d0a992a7e2e0f7c176" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:46.122920 711 consensus_peers.cc:597] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:46.123183 711 consensus_peers.cc:597] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:46.124099 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1578887b2f5748cd83a6fbdec2f3fc99" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:46.124253 1048 raft_consensus.cc:2468] T 1578887b2f5748cd83a6fbdec2f3fc99 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:46.121096 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "41bf5b5111f64dfa91ed76ed4ab4f6f7" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:46.126010 1044 raft_consensus.cc:2468] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:46.129532 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c65926700954af7ab539f3b78805d8d. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:46.120784 982 leader_election.cc:343] T 94658fcf512a48d0a992a7e2e0f7c176 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.129860 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03184dcc56dd4e92adc7d0ad22aa8d8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.132361 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "06b8f8e46e524545aeab6c931efaf116" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.134791 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd43a27db69044b68826cda87e16a086" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.139041 754 tablet_service.cc:1505] Processing CreateTablet for tablet 171d40a8a9124f9bb1b38797fef8e5d5 (DEFAULT_TABLE table=default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.139531 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 171d40a8a9124f9bb1b38797fef8e5d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.143973 741 tablet_service.cc:1505] Processing CreateTablet for tablet 95af3eeefd084777a70736c59d487322 (DEFAULT_TABLE table=default.Table_b8e180cc27234500a1c4c7bdd2b93ac8 [id=5dfca04de96e404a802046007f6a2afb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.144517 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7be44df39a8f4d6da793479d425fa684" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.144774 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95af3eeefd084777a70736c59d487322. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.145957 881 tablet_service.cc:1505] Processing CreateTablet for tablet 2c65926700954af7ab539f3b78805d8d (DEFAULT_TABLE table=default.Table_beb19ebcfc3b426e9aa9f70a68445f72 [id=67d5a2d2ef5a4e329c5ffd80756ef8f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.146359 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c65926700954af7ab539f3b78805d8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.149530 2813 consensus_queue.cc:1048] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:46.150485 710 leader_election.cc:304] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:46.150867 2813 raft_consensus.cc:2804] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:46.150952 2813 raft_consensus.cc:493] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:46.151013 2813 raft_consensus.cc:3060] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.151921 889 tablet_service.cc:1505] Processing CreateTablet for tablet 171d40a8a9124f9bb1b38797fef8e5d5 (DEFAULT_TABLE table=default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.152388 888 tablet_service.cc:1505] Processing CreateTablet for tablet 95af3eeefd084777a70736c59d487322 (DEFAULT_TABLE table=default.Table_b8e180cc27234500a1c4c7bdd2b93ac8 [id=5dfca04de96e404a802046007f6a2afb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.152575 747 tablet_service.cc:1505] Processing CreateTablet for tablet fdbb1787e1bd41a4a7f86beb1fe36b5a (DEFAULT_TABLE table=default.Table_eeb5507eb6a74588905b9e236e414324 [id=c4432f9a16b84796ae788c7a3cde370b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.152771 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95af3eeefd084777a70736c59d487322. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.152918 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdbb1787e1bd41a4a7f86beb1fe36b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.153395 2813 raft_consensus.cc:515] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.154003 2813 leader_election.cc:290] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.154714 710 leader_election.cc:304] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:46.155524 2813 raft_consensus.cc:2804] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:46.155687 2813 raft_consensus.cc:493] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:46.155817 2813 raft_consensus.cc:3060] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.158414 2813 raft_consensus.cc:515] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.159366 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "845b5ac4a5984c0cb4ab188925103dc9" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.159978 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1578887b2f5748cd83a6fbdec2f3fc99" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:46.160148 1048 raft_consensus.cc:3060] T 1578887b2f5748cd83a6fbdec2f3fc99 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.160099 2692 consensus_queue.cc:1048] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:46.160701 2813 leader_election.cc:290] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.162577 1048 raft_consensus.cc:2468] T 1578887b2f5748cd83a6fbdec2f3fc99 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:46.162774 747 tablet_service.cc:1505] Processing CreateTablet for tablet 7a299ee03d0e48aba7c4f71e3e0f381a (DEFAULT_TABLE table=default.Table_6402a801d48e4a53827b1b62b6cdaba1 [id=6876565d3a9c48ce841b830cf75139ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.162878 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "41bf5b5111f64dfa91ed76ed4ab4f6f7" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:46.163017 1045 raft_consensus.cc:3060] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.163152 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a299ee03d0e48aba7c4f71e3e0f381a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.163754 880 tablet_service.cc:1505] Processing CreateTablet for tablet 03184dcc56dd4e92adc7d0ad22aa8d8b (DEFAULT_TABLE table=default.Table_68261841754847ad8d22e86c29886144 [id=697f23a617f64762bf406b386cde7473]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.164170 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03184dcc56dd4e92adc7d0ad22aa8d8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.165493 1045 raft_consensus.cc:2468] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:46.166224 710 leader_election.cc:304] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:46.166775 2500 raft_consensus.cc:2804] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:46.166914 2500 raft_consensus.cc:697] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:46.167150 2500 consensus_queue.cc:237] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.167897 1160 tablet_bootstrap.cc:492] T 6a61cf8f5de14adaa4a1de5bddde9a2f P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:46.168066 1160 ts_tablet_manager.cc:1403] T 6a61cf8f5de14adaa4a1de5bddde9a2f P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.198s user 0.010s sys 0.017s
I20251211 20:52:46.168403 847 leader_election.cc:304] T 2e7e158809b9471fbd6e6ee9fb9ac569 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:46.168927 1160 raft_consensus.cc:359] T 6a61cf8f5de14adaa4a1de5bddde9a2f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.169121 1160 raft_consensus.cc:385] T 6a61cf8f5de14adaa4a1de5bddde9a2f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.169174 1160 raft_consensus.cc:740] T 6a61cf8f5de14adaa4a1de5bddde9a2f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.169385 1160 consensus_queue.cc:260] T 6a61cf8f5de14adaa4a1de5bddde9a2f P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.169907 1160 ts_tablet_manager.cc:1434] T 6a61cf8f5de14adaa4a1de5bddde9a2f P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:46.170257 1160 tablet_bootstrap.cc:492] T 41c30bded6504bbc9f0af8f34dc81e8d P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:46.171248 710 leader_election.cc:304] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:46.171489 2678 raft_consensus.cc:2804] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:46.171633 2678 raft_consensus.cc:697] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:46.171833 2678 consensus_queue.cc:237] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.175987 2806 consensus_queue.cc:1048] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251211 20:52:46.178038 847 leader_election.cc:343] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: e28d66ce4711430cb09a9650e99fbe9e
I20251211 20:52:46.178166 847 leader_election.cc:304] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:46.178411 1166 tablet_bootstrap.cc:492] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:46.178521 847 leader_election.cc:343] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: ccc3ddd4b7f74f11a3835997cb7ae8d0
I20251211 20:52:46.178622 847 leader_election.cc:304] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
W20251211 20:52:46.179035 847 leader_election.cc:343] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 14e2650f8ec1482ea9511b989fdc963c
I20251211 20:52:46.179126 847 leader_election.cc:304] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
W20251211 20:52:46.179487 847 leader_election.cc:343] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 0b267c7feaac43f6aacb7229759c7ec3
I20251211 20:52:46.179605 847 leader_election.cc:304] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:46.180887 1166 tablet_bootstrap.cc:654] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.182307 2647 raft_consensus.cc:2749] T 2e7e158809b9471fbd6e6ee9fb9ac569 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:52:46.182499 2759 raft_consensus.cc:2749] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.183254 2460 consensus_queue.cc:1048] T 260bdeb73439437cbf234d20f7264ab3 P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20251211 20:52:46.183743 2764 raft_consensus.cc:493] T 79cc81e91e1846038621af7a0a9f7a2e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.183885 2764 raft_consensus.cc:515] T 79cc81e91e1846038621af7a0a9f7a2e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.184444 2764 leader_election.cc:290] T 79cc81e91e1846038621af7a0a9f7a2e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:46.184916 2696 raft_consensus.cc:2749] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:52:46.185096 2670 raft_consensus.cc:2749] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.185060 769 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79cc81e91e1846038621af7a0a9f7a2e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:46.185545 982 leader_election.cc:343] T 79cc81e91e1846038621af7a0a9f7a2e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.185784 2692 raft_consensus.cc:2749] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.185973 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 7a299ee03d0e48aba7c4f71e3e0f381a (DEFAULT_TABLE table=default.Table_6402a801d48e4a53827b1b62b6cdaba1 [id=6876565d3a9c48ce841b830cf75139ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.187122 2759 raft_consensus.cc:493] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6f8d784fd1cd492bbe3809403ec34919)
I20251211 20:52:46.187286 2759 raft_consensus.cc:515] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.191965 894 tablet_service.cc:1505] Processing CreateTablet for tablet 7a299ee03d0e48aba7c4f71e3e0f381a (DEFAULT_TABLE table=default.Table_6402a801d48e4a53827b1b62b6cdaba1 [id=6876565d3a9c48ce841b830cf75139ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.196266 2759 leader_election.cc:290] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.200094 1160 tablet_bootstrap.cc:654] T 41c30bded6504bbc9f0af8f34dc81e8d P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.204124 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a299ee03d0e48aba7c4f71e3e0f381a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.204373 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 171d40a8a9124f9bb1b38797fef8e5d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.208662 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a299ee03d0e48aba7c4f71e3e0f381a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.189703 1013 tablet_service.cc:1505] Processing CreateTablet for tablet fdbb1787e1bd41a4a7f86beb1fe36b5a (DEFAULT_TABLE table=default.Table_eeb5507eb6a74588905b9e236e414324 [id=c4432f9a16b84796ae788c7a3cde370b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.220042 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdbb1787e1bd41a4a7f86beb1fe36b5a. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:46.222532 421 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 85570.9us
I20251211 20:52:46.224676 1168 tablet_bootstrap.cc:492] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:46.224893 1166 tablet_bootstrap.cc:492] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:46.224968 1168 ts_tablet_manager.cc:1403] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.166s user 0.002s sys 0.001s
I20251211 20:52:46.225207 1166 ts_tablet_manager.cc:1403] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.047s user 0.006s sys 0.003s
I20251211 20:52:46.226594 1166 raft_consensus.cc:359] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.227114 1166 raft_consensus.cc:385] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.227232 1166 raft_consensus.cc:740] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.227553 1166 consensus_queue.cc:260] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.228971 1166 ts_tablet_manager.cc:1434] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:46.229733 1166 tablet_bootstrap.cc:492] T 1e13965b6361488eaeb7dd13b467ca65 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:46.232843 1166 tablet_bootstrap.cc:654] T 1e13965b6361488eaeb7dd13b467ca65 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.236640 756 tablet_service.cc:1505] Processing CreateTablet for tablet f262dfbc01824f6093785a21bcd27247 (DEFAULT_TABLE table=default.Table_fadb0061961d4706af19ddfb2f79d521 [id=8062ca5194e445a88f7dca2684519d16]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.237205 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f262dfbc01824f6093785a21bcd27247. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.239611 883 tablet_service.cc:1505] Processing CreateTablet for tablet fdbb1787e1bd41a4a7f86beb1fe36b5a (DEFAULT_TABLE table=default.Table_eeb5507eb6a74588905b9e236e414324 [id=c4432f9a16b84796ae788c7a3cde370b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.241480 880 tablet_service.cc:1505] Processing CreateTablet for tablet f262dfbc01824f6093785a21bcd27247 (DEFAULT_TABLE table=default.Table_fadb0061961d4706af19ddfb2f79d521 [id=8062ca5194e445a88f7dca2684519d16]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.241755 2500 raft_consensus.cc:493] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.241955 2500 raft_consensus.cc:515] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.242695 2500 leader_election.cc:290] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.243270 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdbb1787e1bd41a4a7f86beb1fe36b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.243868 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f262dfbc01824f6093785a21bcd27247. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.244443 907 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5453f68789fd4e89b69e98860050865b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:46.246263 907 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1799e6604f4ed7875ddedecab6f327" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.246416 905 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "94658fcf512a48d0a992a7e2e0f7c176" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.246644 1168 raft_consensus.cc:359] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.246732 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79cc81e91e1846038621af7a0a9f7a2e" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.247816 1168 raft_consensus.cc:385] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.248183 2500 consensus_queue.cc:1048] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:46.248180 1168 raft_consensus.cc:740] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.248780 1168 consensus_queue.cc:260] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.249042 2813 consensus_queue.cc:1048] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:46.249693 2494 raft_consensus.cc:493] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.251557 2494 raft_consensus.cc:515] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.251749 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0aac6fb3972e4aba8950828c9e49d507" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.252107 2494 leader_election.cc:290] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.254490 1168 ts_tablet_manager.cc:1434] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.029s user 0.001s sys 0.001s
I20251211 20:52:46.254463 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0aac6fb3972e4aba8950828c9e49d507" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:46.254825 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5453f68789fd4e89b69e98860050865b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.255955 887 tablet_service.cc:1505] Processing CreateTablet for tablet ea4483b0917045348eab2d05dd2a446e (DEFAULT_TABLE table=default.Table_31a95f7c09a34c4585eef1f7914a97a7 [id=8151557842dc4ca8ab0d1b3d663862ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.256132 2759 raft_consensus.cc:493] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.256309 2759 raft_consensus.cc:515] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.256415 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea4483b0917045348eab2d05dd2a446e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.256904 2759 leader_election.cc:290] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.257213 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "44e2d2bdecf247baaac26e06bccc94de" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.247048 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "47779034ad89466e9e1801987128b4dd" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
W20251211 20:52:46.247191 983 leader_election.cc:343] T 95866de91f9d437d80f40b7962253610 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.257809 983 leader_election.cc:304] T 95866de91f9d437d80f40b7962253610 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:46.258280 983 leader_election.cc:343] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.258383 983 leader_election.cc:304] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:46.258877 983 leader_election.cc:343] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.258975 983 leader_election.cc:304] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.259495 2775 raft_consensus.cc:2749] T 95866de91f9d437d80f40b7962253610 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.259675 2775 raft_consensus.cc:2749] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.259790 2775 raft_consensus.cc:2749] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.260545 1048 raft_consensus.cc:2468] T 47779034ad89466e9e1801987128b4dd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
W20251211 20:52:46.263284 983 leader_election.cc:343] T a7979798e92e45eb969197c8c7ac2580 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.263422 983 leader_election.cc:304] T a7979798e92e45eb969197c8c7ac2580 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.263813 2835 raft_consensus.cc:2749] T a7979798e92e45eb969197c8c7ac2580 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:46.263945 983 leader_election.cc:343] T d43210ec61af423ebde624397dea9f2e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.264035 983 leader_election.cc:304] T d43210ec61af423ebde624397dea9f2e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.264299 2835 raft_consensus.cc:2749] T d43210ec61af423ebde624397dea9f2e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:46.264698 983 leader_election.cc:343] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.264995 1168 tablet_bootstrap.cc:492] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:46.265002 983 leader_election.cc:304] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.265188 2764 raft_consensus.cc:2749] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:46.265657 983 leader_election.cc:343] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.265803 983 leader_election.cc:304] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.266021 2764 raft_consensus.cc:2749] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:46.266314 983 leader_election.cc:343] T fd43a27db69044b68826cda87e16a086 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.266515 983 leader_election.cc:304] T fd43a27db69044b68826cda87e16a086 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.266741 2764 raft_consensus.cc:2749] T fd43a27db69044b68826cda87e16a086 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:46.267045 983 leader_election.cc:343] T 7be44df39a8f4d6da793479d425fa684 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.267171 983 leader_election.cc:304] T 7be44df39a8f4d6da793479d425fa684 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.267371 2764 raft_consensus.cc:2749] T 7be44df39a8f4d6da793479d425fa684 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:46.267699 983 leader_election.cc:343] T 845b5ac4a5984c0cb4ab188925103dc9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.267835 983 leader_election.cc:304] T 845b5ac4a5984c0cb4ab188925103dc9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.268090 2764 raft_consensus.cc:2749] T 845b5ac4a5984c0cb4ab188925103dc9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.268913 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1799e6604f4ed7875ddedecab6f327" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:46.274441 1168 tablet_bootstrap.cc:654] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:46.264585 711 leader_election.cc:343] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.275673 711 leader_election.cc:343] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.270201 710 leader_election.cc:304] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:46.276470 1166 tablet_bootstrap.cc:492] T 1e13965b6361488eaeb7dd13b467ca65 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:46.276767 2678 raft_consensus.cc:2804] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:46.276865 2678 raft_consensus.cc:493] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:46.276932 2678 raft_consensus.cc:3060] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.276661 2807 consensus_queue.cc:1048] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251211 20:52:46.278143 1166 ts_tablet_manager.cc:1403] T 1e13965b6361488eaeb7dd13b467ca65 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.049s user 0.005s sys 0.000s
I20251211 20:52:46.279021 1166 raft_consensus.cc:359] T 1e13965b6361488eaeb7dd13b467ca65 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.279338 2678 raft_consensus.cc:515] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.279515 1166 raft_consensus.cc:385] T 1e13965b6361488eaeb7dd13b467ca65 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.279767 1166 raft_consensus.cc:740] T 1e13965b6361488eaeb7dd13b467ca65 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.280064 2678 leader_election.cc:290] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.280750 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "41bf5b5111f64dfa91ed76ed4ab4f6f7" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.280772 741 tablet_service.cc:1505] Processing CreateTablet for tablet ea4483b0917045348eab2d05dd2a446e (DEFAULT_TABLE table=default.Table_31a95f7c09a34c4585eef1f7914a97a7 [id=8151557842dc4ca8ab0d1b3d663862ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.281287 1166 consensus_queue.cc:260] T 1e13965b6361488eaeb7dd13b467ca65 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.282064 1166 ts_tablet_manager.cc:1434] T 1e13965b6361488eaeb7dd13b467ca65 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:46.282421 1166 tablet_bootstrap.cc:492] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:46.282505 913 raft_consensus.cc:3060] T f3855ffb51a74101b26fd28429e65bc2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.283633 2764 raft_consensus.cc:493] T 497c10742f01476db885bd70c4d2c491 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.283754 2764 raft_consensus.cc:515] T 497c10742f01476db885bd70c4d2c491 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.284276 2764 leader_election.cc:290] T 497c10742f01476db885bd70c4d2c491 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:46.287353 1166 tablet_bootstrap.cc:654] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.289031 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1578887b2f5748cd83a6fbdec2f3fc99" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.290427 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1578887b2f5748cd83a6fbdec2f3fc99" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:46.290797 886 tablet_service.cc:1505] Processing CreateTablet for tablet a249906ca3204563a485aa4f5abe91f2 (DEFAULT_TABLE table=default.Table_8e6bffee74fd4ea4a95ec5e51b0154c1 [id=c345c786f9a949dea34cc93bb72ae4cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.291278 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a249906ca3204563a485aa4f5abe91f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.295167 2678 raft_consensus.cc:493] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.295333 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "41bf5b5111f64dfa91ed76ed4ab4f6f7" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:46.295959 2678 raft_consensus.cc:515] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.297391 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea4483b0917045348eab2d05dd2a446e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.299507 1041 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5bcc94ec9630484cb7a3e8bf6d4f378a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:46.300318 1041 raft_consensus.cc:2468] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:46.300633 2806 raft_consensus.cc:493] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.300828 2806 raft_consensus.cc:515] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.300076 1015 tablet_service.cc:1505] Processing CreateTablet for tablet f262dfbc01824f6093785a21bcd27247 (DEFAULT_TABLE table=default.Table_fadb0061961d4706af19ddfb2f79d521 [id=8062ca5194e445a88f7dca2684519d16]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.301393 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f262dfbc01824f6093785a21bcd27247. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.301762 2806 leader_election.cc:290] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.302296 888 tablet_service.cc:1505] Processing CreateTablet for tablet e71c5eed203546fea69be5026da29fbb (DEFAULT_TABLE table=default.Table_2198fd77ec454943a0c0a392964c1fd6 [id=88c89157d3404a558c1d0ca2ddbcf3bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.302760 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e71c5eed203546fea69be5026da29fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.304827 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet aab26df748724fe5b2bb7fad9e3bfc0c (table default.Table_22bed4b4e9e24a90818b19c6f5c91e1f [id=df6255d66f334ef2b4406f4b845cf980]). Expected version 1 got 0
I20251211 20:52:46.307772 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "47779034ad89466e9e1801987128b4dd" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:46.308077 1048 raft_consensus.cc:3060] T 47779034ad89466e9e1801987128b4dd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.309228 747 tablet_service.cc:1505] Processing CreateTablet for tablet a249906ca3204563a485aa4f5abe91f2 (DEFAULT_TABLE table=default.Table_8e6bffee74fd4ea4a95ec5e51b0154c1 [id=c345c786f9a949dea34cc93bb72ae4cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.309664 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a249906ca3204563a485aa4f5abe91f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.310655 1048 raft_consensus.cc:2468] T 47779034ad89466e9e1801987128b4dd P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:46.311458 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7452dd736b0440fcb4626858e794676f" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:46.311700 1048 raft_consensus.cc:2468] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:46.297096 754 tablet_service.cc:1505] Processing CreateTablet for tablet e71c5eed203546fea69be5026da29fbb (DEFAULT_TABLE table=default.Table_2198fd77ec454943a0c0a392964c1fd6 [id=88c89157d3404a558c1d0ca2ddbcf3bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.312350 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e71c5eed203546fea69be5026da29fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.313586 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f25bde14792248b188660036f07ae902" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:46.313758 1048 raft_consensus.cc:2468] T f25bde14792248b188660036f07ae902 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:46.315150 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "497c10742f01476db885bd70c4d2c491" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:46.316365 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "47779034ad89466e9e1801987128b4dd" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.317437 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5bcc94ec9630484cb7a3e8bf6d4f378a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.318562 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "47779034ad89466e9e1801987128b4dd" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:46.320137 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f25bde14792248b188660036f07ae902" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:46.320166 711 leader_election.cc:343] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.320840 711 leader_election.cc:343] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.321285 711 consensus_peers.cc:597] T c91ff29847d54e55b19321652a857e9f P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:46.321530 711 leader_election.cc:343] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.322008 711 consensus_peers.cc:597] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:46.322521 711 consensus_peers.cc:597] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:46.322770 711 consensus_peers.cc:597] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:46.323208 711 consensus_peers.cc:597] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:46.323253 824 heartbeater.cc:499] Master 127.0.104.61:40749 was elected leader, sending a full tablet report...
W20251211 20:52:46.323469 711 consensus_peers.cc:597] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:46.323727 907 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7452dd736b0440fcb4626858e794676f" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:46.324054 983 leader_election.cc:343] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.324082 710 leader_election.cc:304] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
W20251211 20:52:46.324311 847 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.1:42177, user_credentials={real_user=slave}} blocked reactor thread for 144817us
I20251211 20:52:46.324390 2500 raft_consensus.cc:2804] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:46.324453 2500 raft_consensus.cc:493] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20251211 20:52:46.324422 983 leader_election.cc:343] T 94658fcf512a48d0a992a7e2e0f7c176 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.324508 2500 raft_consensus.cc:3060] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:46.324515 847 leader_election.cc:343] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 25454e67437541d9ac2bfab29bad55f9
I20251211 20:52:46.324537 983 leader_election.cc:304] T 94658fcf512a48d0a992a7e2e0f7c176 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.324612 847 leader_election.cc:304] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:46.324766 2764 raft_consensus.cc:2749] T 94658fcf512a48d0a992a7e2e0f7c176 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:46.325172 983 leader_election.cc:343] T 79cc81e91e1846038621af7a0a9f7a2e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.325286 983 leader_election.cc:304] T 79cc81e91e1846038621af7a0a9f7a2e P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.325526 2764 raft_consensus.cc:2749] T 79cc81e91e1846038621af7a0a9f7a2e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:46.325906 983 leader_election.cc:343] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.326226 983 leader_election.cc:343] T 44e2d2bdecf247baaac26e06bccc94de P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.326349 983 leader_election.cc:304] T 44e2d2bdecf247baaac26e06bccc94de P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.326581 2764 raft_consensus.cc:2749] T 44e2d2bdecf247baaac26e06bccc94de P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.326994 2500 raft_consensus.cc:515] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.327881 2429 consensus_queue.cc:1048] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:46.328580 2500 leader_election.cc:290] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.328599 710 leader_election.cc:304] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:46.329150 2500 raft_consensus.cc:2804] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:46.329303 2500 raft_consensus.cc:697] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:46.329572 2500 consensus_queue.cc:237] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.334784 907 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "497c10742f01476db885bd70c4d2c491" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:46.336048 711 leader_election.cc:343] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.338225 710 leader_election.cc:304] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:46.338531 2813 raft_consensus.cc:2804] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:46.338614 2813 raft_consensus.cc:493] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:46.338672 2813 raft_consensus.cc:3060] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.339228 2759 raft_consensus.cc:2749] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:46.339211 711 leader_election.cc:343] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.339533 711 consensus_peers.cc:597] T 4a39870c7eea4a379b284ea2e5ddf760 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:46.340196 711 consensus_peers.cc:597] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:46.340183 847 leader_election.cc:343] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 52ef1707f6f2425aafd454f303af0333
I20251211 20:52:46.340284 847 leader_election.cc:304] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:46.340521 2759 raft_consensus.cc:2749] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.340631 1168 tablet_bootstrap.cc:492] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
W20251211 20:52:46.340749 711 consensus_peers.cc:597] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:46.340978 711 leader_election.cc:343] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.341361 983 leader_election.cc:343] T 497c10742f01476db885bd70c4d2c491 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.341507 2813 raft_consensus.cc:515] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:46.341867 711 leader_election.cc:343] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.341899 2759 consensus_queue.cc:1048] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:46.342129 2813 leader_election.cc:290] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.343645 2670 raft_consensus.cc:493] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.343784 2670 raft_consensus.cc:515] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.343818 710 leader_election.cc:304] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:46.344090 2813 raft_consensus.cc:2804] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:46.344161 2759 raft_consensus.cc:493] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.344177 2813 raft_consensus.cc:493] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:46.344244 2813 raft_consensus.cc:3060] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:46.342135 711 leader_election.cc:343] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.345146 711 consensus_peers.cc:597] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:46.345353 711 consensus_peers.cc:597] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:46.345548 711 consensus_peers.cc:597] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:46.345775 711 leader_election.cc:343] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.347242 2813 raft_consensus.cc:515] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.297084 2678 leader_election.cc:290] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.348223 2813 leader_election.cc:290] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:46.348627 711 leader_election.cc:343] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.340773 1168 ts_tablet_manager.cc:1403] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.076s user 0.002s sys 0.002s
I20251211 20:52:46.350003 1168 raft_consensus.cc:359] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.353463 1168 raft_consensus.cc:385] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.353711 1168 raft_consensus.cc:740] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.351368 756 tablet_service.cc:1505] Processing CreateTablet for tablet f1bca08aba654a05880eb3b45e669be4 (DEFAULT_TABLE table=default.Table_4e6562227887405aae36515729c90a28 [id=9c44b27aece14760b4c94bc8d9f82218]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.354141 1168 consensus_queue.cc:260] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.354882 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1bca08aba654a05880eb3b45e669be4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.354952 1166 tablet_bootstrap.cc:492] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:46.355150 1166 ts_tablet_manager.cc:1403] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.073s user 0.011s sys 0.000s
I20251211 20:52:46.356278 1020 tablet_service.cc:1505] Processing CreateTablet for tablet e71c5eed203546fea69be5026da29fbb (DEFAULT_TABLE table=default.Table_2198fd77ec454943a0c0a392964c1fd6 [id=88c89157d3404a558c1d0ca2ddbcf3bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.356462 1013 tablet_service.cc:1505] Processing CreateTablet for tablet ea4483b0917045348eab2d05dd2a446e (DEFAULT_TABLE table=default.Table_31a95f7c09a34c4585eef1f7914a97a7 [id=8151557842dc4ca8ab0d1b3d663862ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.356684 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e71c5eed203546fea69be5026da29fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.356922 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea4483b0917045348eab2d05dd2a446e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.356873 1029 tablet_service.cc:1505] Processing CreateTablet for tablet d3167c876d9d4c0d976ff21cbba653ab (DEFAULT_TABLE table=default.Table_dde47886ad4d4ea8a332c68e13085fe0 [id=8cc29ee550ed480c9e261d842fe08e3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.357231 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3167c876d9d4c0d976ff21cbba653ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.356231 1019 tablet_service.cc:1505] Processing CreateTablet for tablet a249906ca3204563a485aa4f5abe91f2 (DEFAULT_TABLE table=default.Table_8e6bffee74fd4ea4a95ec5e51b0154c1 [id=c345c786f9a949dea34cc93bb72ae4cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.358915 1022 tablet_service.cc:1505] Processing CreateTablet for tablet f1bca08aba654a05880eb3b45e669be4 (DEFAULT_TABLE table=default.Table_4e6562227887405aae36515729c90a28 [id=9c44b27aece14760b4c94bc8d9f82218]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.359867 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a249906ca3204563a485aa4f5abe91f2. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:46.360383 982 leader_election.cc:343] T 497c10742f01476db885bd70c4d2c491 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.362247 2764 raft_consensus.cc:493] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251211 20:52:46.362707 1784 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet ee7b93b0b87541d9af6eb9e68b4cea41 (table default.Table_b6e6aea68c88426fb01353146c06b6ac [id=a4f984b4263a4bc38f6248628f643ae3], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet ee7b93b0b87541d9af6eb9e68b4cea41 consensus state has no leader
I20251211 20:52:46.352362 747 tablet_service.cc:1505] Processing CreateTablet for tablet d3167c876d9d4c0d976ff21cbba653ab (DEFAULT_TABLE table=default.Table_dde47886ad4d4ea8a332c68e13085fe0 [id=8cc29ee550ed480c9e261d842fe08e3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.352037 739 tablet_service.cc:1505] Processing CreateTablet for tablet bb358f9722234ed7959560c6dfb13dc6 (DEFAULT_TABLE table=default.Table_06dc3002fa354d78ab4d5a90fd0fb3ab [id=13db407cd0aa40889e554244cbcc3616]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.363198 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb358f9722234ed7959560c6dfb13dc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.363772 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1bca08aba654a05880eb3b45e669be4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.344264 2759 raft_consensus.cc:515] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.367693 2764 raft_consensus.cc:515] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.368427 2764 leader_election.cc:290] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:46.369468 881 tablet_service.cc:1505] Processing CreateTablet for tablet d3167c876d9d4c0d976ff21cbba653ab (DEFAULT_TABLE table=default.Table_dde47886ad4d4ea8a332c68e13085fe0 [id=8cc29ee550ed480c9e261d842fe08e3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.369993 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3167c876d9d4c0d976ff21cbba653ab. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:46.350756 711 leader_election.cc:343] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.371141 711 leader_election.cc:343] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.371518 711 leader_election.cc:343] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.372133 711 consensus_peers.cc:597] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:46.373026 2848 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 11f5f94732f049d0be4c97cebfffcb05)
I20251211 20:52:46.376611 2848 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:46.378909 1829 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1830b0c58cfe495989f3d911dda0409a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1256 } } ignore_live_leader: false dest_uuid: "11f5f94732f049d0be4c97cebfffcb05" is_pre_election: true
I20251211 20:52:46.363260 982 leader_election.cc:304] T 497c10742f01476db885bd70c4d2c491 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.381186 2764 raft_consensus.cc:2749] T 497c10742f01476db885bd70c4d2c491 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.382071 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3167c876d9d4c0d976ff21cbba653ab. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:46.388890 847 consensus_peers.cc:597] T 4ce495233a5e4f3792faf44680df54ba P eb0f1933d9a74fccaa4618047fc05c41 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4ce495233a5e4f3792faf44680df54ba. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:46.389325 847 leader_election.cc:343] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: e6ba12f94f5f4d688f4222a4c2fb5436
W20251211 20:52:46.389775 847 consensus_peers.cc:597] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2207af105bc3471ca930d35a91d38e70. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:46.390986 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e9ead34065e240eeb1f946319537d32a" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:46.391955 2807 consensus_queue.cc:1048] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251211 20:52:46.392791 982 leader_election.cc:343] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.394796 1166 raft_consensus.cc:359] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.395018 1166 raft_consensus.cc:385] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.395805 1038 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f25bde14792248b188660036f07ae902" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:46.397473 2848 leader_election.cc:290] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11f5f94732f049d0be4c97cebfffcb05 (127.0.104.61:40749), 9230f94a6a1a49cead51aeea9f069087 (127.0.104.60:42087)
I20251211 20:52:46.397858 1036 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5bcc94ec9630484cb7a3e8bf6d4f378a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:46.399376 1038 raft_consensus.cc:3060] T f25bde14792248b188660036f07ae902 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.399660 1166 raft_consensus.cc:740] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.399932 1166 consensus_queue.cc:260] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.400898 1166 ts_tablet_manager.cc:1434] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.045s user 0.001s sys 0.001s
I20251211 20:52:46.401067 1036 raft_consensus.cc:3060] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.415725 2670 leader_election.cc:290] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.420198 1166 tablet_bootstrap.cc:492] T 84b2a67c836844ceb3628cc3b7d8ba10 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:46.355104 1168 ts_tablet_manager.cc:1434] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20251211 20:52:46.420668 2500 consensus_queue.cc:1048] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:46.420991 1168 tablet_bootstrap.cc:492] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:46.423627 1168 tablet_bootstrap.cc:654] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.425510 1036 raft_consensus.cc:2468] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:46.426102 710 leader_election.cc:304] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:46.426326 2429 raft_consensus.cc:2804] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:46.426541 2429 raft_consensus.cc:697] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:46.427196 2429 consensus_queue.cc:237] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:46.391167 847 leader_election.cc:343] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 793f543d6b614ad7b3ab8e717db2ceaf
W20251211 20:52:46.428683 847 leader_election.cc:343] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: e6ba12f94f5f4d688f4222a4c2fb5436
I20251211 20:52:46.428808 847 leader_election.cc:304] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
W20251211 20:52:46.429176 847 leader_election.cc:343] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 793f543d6b614ad7b3ab8e717db2ceaf
I20251211 20:52:46.429261 847 leader_election.cc:304] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:46.429766 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5bcc94ec9630484cb7a3e8bf6d4f378a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:46.430219 2825 raft_consensus.cc:2749] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:46.431320 2842 log.cc:927] Time spent T 2e7e158809b9471fbd6e6ee9fb9ac569 P eb0f1933d9a74fccaa4618047fc05c41: Append to log took a long time: real 0.086s user 0.000s sys 0.000s
I20251211 20:52:46.432096 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7452dd736b0440fcb4626858e794676f" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
W20251211 20:52:46.435638 847 leader_election.cc:343] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 4201da4241aa48dfb670a713e4453ca5
I20251211 20:52:46.435981 2825 consensus_queue.cc:1048] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251211 20:52:46.437773 847 leader_election.cc:343] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: bb0fc179a337469eae056c5bb3815e6b
W20251211 20:52:46.440765 1295 leader_election.cc:336] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9230f94a6a1a49cead51aeea9f069087 (127.0.104.60:42087): Network error: Client connection negotiation failed: client connection to 127.0.104.60:42087: connect: Connection refused (error 111)
I20251211 20:52:46.441020 1295 leader_election.cc:304] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1830b0c58cfe495989f3d911dda0409a; no voters: 11f5f94732f049d0be4c97cebfffcb05, 9230f94a6a1a49cead51aeea9f069087
I20251211 20:52:46.441349 2848 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20251211 20:52:46.441659 2500 consensus_queue.cc:1048] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251211 20:52:46.444180 2833 log.cc:927] Time spent T 5262c2241e524395a73c0eb368917169 P eb0f1933d9a74fccaa4618047fc05c41: Append to log took a long time: real 0.076s user 0.000s sys 0.000s
I20251211 20:52:46.444545 2759 leader_election.cc:290] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.447266 1168 tablet_bootstrap.cc:492] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:46.447605 1044 raft_consensus.cc:3060] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.448990 1168 ts_tablet_manager.cc:1403] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.001s
I20251211 20:52:46.450094 1168 raft_consensus.cc:359] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:46.451177 847 leader_election.cc:343] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 4201da4241aa48dfb670a713e4453ca5
I20251211 20:52:46.451297 847 leader_election.cc:304] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:46.451828 2825 raft_consensus.cc:2749] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.451972 1168 raft_consensus.cc:385] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.452283 1168 raft_consensus.cc:740] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.452832 2696 raft_consensus.cc:2749] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:52:46.453414 2696 raft_consensus.cc:493] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.453760 886 tablet_service.cc:1505] Processing CreateTablet for tablet f1bca08aba654a05880eb3b45e669be4 (DEFAULT_TABLE table=default.Table_4e6562227887405aae36515729c90a28 [id=9c44b27aece14760b4c94bc8d9f82218]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:46.454749 847 leader_election.cc:343] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: bb0fc179a337469eae056c5bb3815e6b
I20251211 20:52:46.454879 847 leader_election.cc:304] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:46.454882 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1bca08aba654a05880eb3b45e669be4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.452775 1168 consensus_queue.cc:260] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.455945 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b52377c8ffd741a598e5328a00d876fe (table default.Table_686a26915b42421380037144900dd97c [id=8a1b0c9d641b4a3c83d1ca48576d110d]). Expected version 1 got 0
I20251211 20:52:46.456691 883 tablet_service.cc:1505] Processing CreateTablet for tablet bb358f9722234ed7959560c6dfb13dc6 (DEFAULT_TABLE table=default.Table_06dc3002fa354d78ab4d5a90fd0fb3ab [id=13db407cd0aa40889e554244cbcc3616]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.457052 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb358f9722234ed7959560c6dfb13dc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.459282 2500 consensus_queue.cc:1048] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:46.460208 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7452dd736b0440fcb4626858e794676f" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:46.461992 1168 ts_tablet_manager.cc:1434] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.013s user 0.001s sys 0.000s
I20251211 20:52:46.462813 905 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e9ead34065e240eeb1f946319537d32a" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.453529 2696 raft_consensus.cc:515] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.463629 1168 tablet_bootstrap.cc:492] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:46.465312 1038 raft_consensus.cc:2468] T f25bde14792248b188660036f07ae902 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:46.465799 710 leader_election.cc:304] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:46.466058 2500 raft_consensus.cc:2804] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:46.466151 2500 raft_consensus.cc:697] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:46.466362 2500 consensus_queue.cc:237] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.466526 2825 raft_consensus.cc:2749] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.467262 1168 tablet_bootstrap.cc:654] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.469262 1044 raft_consensus.cc:2468] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:46.471136 710 leader_election.cc:304] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:46.471455 2813 raft_consensus.cc:2804] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:46.471570 2813 raft_consensus.cc:697] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:46.471810 2813 consensus_queue.cc:237] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:46.472189 983 leader_election.cc:343] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.472309 983 leader_election.cc:304] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.472345 904 raft_consensus.cc:3060] T 6a61cf8f5de14adaa4a1de5bddde9a2f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.472677 2764 raft_consensus.cc:2749] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.476579 2696 leader_election.cc:290] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.480635 2678 consensus_queue.cc:1048] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251211 20:52:46.480804 847 leader_election.cc:343] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 2dde7f28e61d41f48015ba6e4f3757f2
I20251211 20:52:46.481003 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet c7a2610f20df4c1ca81eae2fc7dba6bb (table default.Table_59072235d93b41c0b5cd5e3debcb8ddf [id=9a15a8a075af4969ae2c8b3be1bfed78]). Expected version 1 got 0
I20251211 20:52:46.481500 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3e6f62c2deff4c11b137fc0cba5ee234 (table default.Table_51db250005ed4b3d947f32acd1d62563 [id=fdfaa0b24d2f4736aee695ed228212cf]). Expected version 1 got 0
I20251211 20:52:46.481855 1796 catalog_manager.cc:5654] T 2f5916a4d9b24d7b8db3fcf028bbe3c9 P e715a811a82446ce8d5bb4aafd5b81c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } }
I20251211 20:52:46.482424 1796 catalog_manager.cc:5654] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3). New cstate: current_term: 1 leader_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: HEALTHY } } }
I20251211 20:52:46.483032 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet f2d6c9781b4d4c37bc08703347963d5d (table default.Table_2aa7ff9186a04249abf8cd971cc0cb17 [id=b50ab5b032ba4c98938324460512468f]). Expected version 1 got 0
I20251211 20:52:46.483326 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 6b7dd8232d974a43b99ec9b28075dc8d (table default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]). Expected version 2 got 0
I20251211 20:52:46.483448 905 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f25bde14792248b188660036f07ae902" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:46.484069 1166 tablet_bootstrap.cc:654] T 84b2a67c836844ceb3628cc3b7d8ba10 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:46.485787 847 consensus_peers.cc:597] T 7ec261837cf04bf88a48ff8add1b0d66 P eb0f1933d9a74fccaa4618047fc05c41 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7ec261837cf04bf88a48ff8add1b0d66. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:46.493148 711 leader_election.cc:343] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.493988 711 leader_election.cc:343] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.494385 711 consensus_peers.cc:597] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:46.494756 711 leader_election.cc:343] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.496263 847 leader_election.cc:343] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 2dde7f28e61d41f48015ba6e4f3757f2
I20251211 20:52:46.496447 847 leader_election.cc:304] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:46.496925 2696 raft_consensus.cc:2749] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.498893 2696 consensus_queue.cc:1048] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251211 20:52:46.499352 711 consensus_peers.cc:597] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:46.505187 1019 tablet_service.cc:1505] Processing CreateTablet for tablet bb358f9722234ed7959560c6dfb13dc6 (DEFAULT_TABLE table=default.Table_06dc3002fa354d78ab4d5a90fd0fb3ab [id=13db407cd0aa40889e554244cbcc3616]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.505810 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb358f9722234ed7959560c6dfb13dc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.506299 881 tablet_service.cc:1505] Processing CreateTablet for tablet e9b9f24dcdd54b34acb435091f093333 (DEFAULT_TABLE table=default.Table_e54af82590454fdf941b0462c30738ad [id=97e5719ea39a4ebe85ff613750c6863a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.506855 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9b9f24dcdd54b34acb435091f093333. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.511711 1160 tablet_bootstrap.cc:492] T 41c30bded6504bbc9f0af8f34dc81e8d P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:46.511878 1160 ts_tablet_manager.cc:1403] T 41c30bded6504bbc9f0af8f34dc81e8d P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.342s user 0.020s sys 0.011s
I20251211 20:52:46.511754 754 tablet_service.cc:1505] Processing CreateTablet for tablet e9b9f24dcdd54b34acb435091f093333 (DEFAULT_TABLE table=default.Table_e54af82590454fdf941b0462c30738ad [id=97e5719ea39a4ebe85ff613750c6863a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.512357 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9b9f24dcdd54b34acb435091f093333. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.512589 1160 raft_consensus.cc:359] T 41c30bded6504bbc9f0af8f34dc81e8d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.512750 1160 raft_consensus.cc:385] T 41c30bded6504bbc9f0af8f34dc81e8d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.520529 1160 raft_consensus.cc:740] T 41c30bded6504bbc9f0af8f34dc81e8d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.520799 2429 consensus_queue.cc:1048] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:46.521255 1160 consensus_queue.cc:260] T 41c30bded6504bbc9f0af8f34dc81e8d P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.521343 2429 raft_consensus.cc:493] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.521453 2429 raft_consensus.cc:515] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.523283 2429 leader_election.cc:290] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.526490 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "18b977f463d5420186c007b38110e07e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.527834 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "18b977f463d5420186c007b38110e07e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:46.528136 1045 raft_consensus.cc:2468] T 18b977f463d5420186c007b38110e07e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:46.530295 2759 raft_consensus.cc:493] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.530444 2759 raft_consensus.cc:515] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.531039 2759 leader_election.cc:290] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.532208 2764 raft_consensus.cc:493] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.532367 2764 raft_consensus.cc:515] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.532936 2764 leader_election.cc:290] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:46.534547 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a295a24e81f14e5fa126a4a58ffd08cf" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:46.534998 983 leader_election.cc:343] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.535015 883 tablet_service.cc:1505] Processing CreateTablet for tablet 06d49ac8182143c69f494da50c050ee0 (DEFAULT_TABLE table=default.Table_3db64a0e2af24640883c6413a68001b0 [id=5ccf927de35144b9b9cc9324638fe9c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.535493 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06d49ac8182143c69f494da50c050ee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.537034 710 leader_election.cc:304] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:46.537147 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 06d49ac8182143c69f494da50c050ee0 (DEFAULT_TABLE table=default.Table_3db64a0e2af24640883c6413a68001b0 [id=5ccf927de35144b9b9cc9324638fe9c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.537369 2807 raft_consensus.cc:2804] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:46.537448 2807 raft_consensus.cc:493] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:46.537513 2807 raft_consensus.cc:3060] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.537550 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06d49ac8182143c69f494da50c050ee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.539994 1013 tablet_service.cc:1505] Processing CreateTablet for tablet e9b9f24dcdd54b34acb435091f093333 (DEFAULT_TABLE table=default.Table_e54af82590454fdf941b0462c30738ad [id=97e5719ea39a4ebe85ff613750c6863a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.540376 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9b9f24dcdd54b34acb435091f093333. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.541263 2807 raft_consensus.cc:515] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.541886 2807 leader_election.cc:290] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.542586 756 tablet_service.cc:1505] Processing CreateTablet for tablet 41627cb27be8443fa37e514f9b3825a6 (DEFAULT_TABLE table=default.Table_c73d4009f4bc41b38aee5612313b30c8 [id=bd5290958c7249f39a6cc51f631f42a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.544028 886 tablet_service.cc:1505] Processing CreateTablet for tablet 41627cb27be8443fa37e514f9b3825a6 (DEFAULT_TABLE table=default.Table_c73d4009f4bc41b38aee5612313b30c8 [id=bd5290958c7249f39a6cc51f631f42a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.544451 747 tablet_service.cc:1505] Processing CreateTablet for tablet 06d49ac8182143c69f494da50c050ee0 (DEFAULT_TABLE table=default.Table_3db64a0e2af24640883c6413a68001b0 [id=5ccf927de35144b9b9cc9324638fe9c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.545183 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41627cb27be8443fa37e514f9b3825a6. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:46.546094 847 leader_election.cc:343] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: e4b5a7553110424d901788c7a2ecdce6
W20251211 20:52:46.537519 711 leader_election.cc:343] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.547969 2696 raft_consensus.cc:493] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251211 20:52:46.548035 847 consensus_peers.cc:597] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 68792ccef54944a09af9b5456e9ff1fe. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:46.548102 2696 raft_consensus.cc:515] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.548636 2696 leader_election.cc:290] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:46.549202 711 consensus_peers.cc:597] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:46.550640 847 leader_election.cc:343] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 92a71dff4ea8445d8484fc2747471148
I20251211 20:52:46.554687 1166 tablet_bootstrap.cc:492] T 84b2a67c836844ceb3628cc3b7d8ba10 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:46.554879 1166 ts_tablet_manager.cc:1403] T 84b2a67c836844ceb3628cc3b7d8ba10 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.135s user 0.006s sys 0.002s
I20251211 20:52:46.555734 1166 raft_consensus.cc:359] T 84b2a67c836844ceb3628cc3b7d8ba10 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.555941 1166 raft_consensus.cc:385] T 84b2a67c836844ceb3628cc3b7d8ba10 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.556020 1166 raft_consensus.cc:740] T 84b2a67c836844ceb3628cc3b7d8ba10 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.556268 1166 consensus_queue.cc:260] T 84b2a67c836844ceb3628cc3b7d8ba10 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.556774 1166 ts_tablet_manager.cc:1434] T 84b2a67c836844ceb3628cc3b7d8ba10 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:46.558955 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "18b977f463d5420186c007b38110e07e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:46.561403 711 leader_election.cc:343] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.565232 774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a295a24e81f14e5fa126a4a58ffd08cf" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:46.566305 2764 raft_consensus.cc:493] T 98d575ac680d49d0990877666638f013 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.566468 2764 raft_consensus.cc:515] T 98d575ac680d49d0990877666638f013 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.567044 2764 leader_election.cc:290] T 98d575ac680d49d0990877666638f013 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:46.567739 776 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "98d575ac680d49d0990877666638f013" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:46.568045 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "98d575ac680d49d0990877666638f013" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:46.568271 982 leader_election.cc:343] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.568397 982 leader_election.cc:304] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.568658 2764 raft_consensus.cc:2749] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:46.569082 982 leader_election.cc:343] T 98d575ac680d49d0990877666638f013 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.569381 2429 raft_consensus.cc:493] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.569859 2429 raft_consensus.cc:515] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.570175 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41627cb27be8443fa37e514f9b3825a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.570816 2429 leader_election.cc:290] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.572381 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "92efda2bd0c84d07a0af6b537c0cd882" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:46.572628 1045 raft_consensus.cc:2468] T 92efda2bd0c84d07a0af6b537c0cd882 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:46.573006 710 leader_election.cc:304] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:46.573578 2429 raft_consensus.cc:2804] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:46.573709 2429 raft_consensus.cc:493] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20251211 20:52:46.573930 983 leader_election.cc:343] T 98d575ac680d49d0990877666638f013 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.574059 983 leader_election.cc:304] T 98d575ac680d49d0990877666638f013 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.574476 2764 raft_consensus.cc:2749] T 98d575ac680d49d0990877666638f013 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:46.575013 847 leader_election.cc:343] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 92a71dff4ea8445d8484fc2747471148
I20251211 20:52:46.575129 847 leader_election.cc:304] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:46.575048 2429 raft_consensus.cc:3060] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.575745 2675 raft_consensus.cc:493] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.575887 2675 raft_consensus.cc:515] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.576005 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06d49ac8182143c69f494da50c050ee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.576483 2675 leader_election.cc:290] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.576802 2670 raft_consensus.cc:2749] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.577936 2429 raft_consensus.cc:515] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.578410 2429 leader_election.cc:290] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.578629 1160 ts_tablet_manager.cc:1434] T 41c30bded6504bbc9f0af8f34dc81e8d P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.067s user 0.002s sys 0.000s
I20251211 20:52:46.579788 1038 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "92efda2bd0c84d07a0af6b537c0cd882" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:46.579967 1038 raft_consensus.cc:3060] T 92efda2bd0c84d07a0af6b537c0cd882 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.581182 2670 consensus_queue.cc:1048] T 2207af105bc3471ca930d35a91d38e70 P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:46.582595 1038 raft_consensus.cc:2468] T 92efda2bd0c84d07a0af6b537c0cd882 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:46.582898 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d69fb1f6d9594afcbfb844860b41a377" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.583294 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "92efda2bd0c84d07a0af6b537c0cd882" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:46.583353 710 leader_election.cc:304] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:46.583559 2500 raft_consensus.cc:2804] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:46.583747 2500 raft_consensus.cc:697] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:46.584014 2500 consensus_queue.cc:237] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.584547 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "92efda2bd0c84d07a0af6b537c0cd882" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.584734 2736 raft_consensus.cc:493] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.584856 2736 raft_consensus.cc:515] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.585106 1160 tablet_bootstrap.cc:492] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:46.585330 2736 leader_election.cc:290] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.586488 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "18b977f463d5420186c007b38110e07e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:46.586690 1047 raft_consensus.cc:3060] T 18b977f463d5420186c007b38110e07e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.587086 1160 tablet_bootstrap.cc:654] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.587970 1041 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1aeae8c18b6499e9dc1accabdc3f633" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:46.588171 1041 raft_consensus.cc:2468] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:46.589454 1047 raft_consensus.cc:2468] T 18b977f463d5420186c007b38110e07e P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
W20251211 20:52:46.589533 711 leader_election.cc:343] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.589707 711 leader_election.cc:343] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.589713 710 leader_election.cc:304] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
W20251211 20:52:46.590009 711 leader_election.cc:343] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.590140 2500 raft_consensus.cc:2804] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:46.590236 2500 raft_consensus.cc:493] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:46.590330 2500 raft_consensus.cc:3060] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:46.590363 847 leader_election.cc:343] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: e4b5a7553110424d901788c7a2ecdce6
I20251211 20:52:46.590476 847 leader_election.cc:304] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:46.590827 2825 raft_consensus.cc:2749] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.593045 2500 raft_consensus.cc:515] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.593873 2500 leader_election.cc:290] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.594530 2500 consensus_queue.cc:1048] T 75dd170c5828409fa69bd68fdf28a72b P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:46.483574 1796 catalog_manager.cc:5654] T 676faec7f3d1488f99b220f8e968e9a0 P e715a811a82446ce8d5bb4aafd5b81c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } }
I20251211 20:52:46.595916 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 676faec7f3d1488f99b220f8e968e9a0 (table default.Table_ab826a3eec8e4c4aaa8d4021d87dd382 [id=a2bbfccfe3974c378521d5ef2333df86]). Expected version 2 got 0
I20251211 20:52:46.596376 1796 catalog_manager.cc:5654] T a3b1aadf17ea4bf0a896d23cd543fca7 P e715a811a82446ce8d5bb4aafd5b81c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } }
I20251211 20:52:46.596865 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet a3b1aadf17ea4bf0a896d23cd543fca7 (table default.Table_d18401707c88493abdf90d2c713432ae [id=a07662f8dc3f4178be5dfcc1cdc85837]). Expected version 1 got 0
I20251211 20:52:46.597239 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 9f8aa28c1c9b4fc58dfc128952c180ba (table default.Table_bf7d402a36be44cfb6b79041906d26a1 [id=0327e686e7ed4475a3e8cda8721a3a3a]). Expected version 1 got 0
I20251211 20:52:46.597725 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 834b84a557b04e78b5d2c8c84520a644 (table default.Table_909bbe6da1a648d68199fb27424e4342 [id=cc99459a5039424dbc192ff748b6708b]). Expected version 2 got 0
I20251211 20:52:46.597795 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1aeae8c18b6499e9dc1accabdc3f633" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.598222 1796 catalog_manager.cc:5654] T 8b793241ee3b42b1a38e9f2f05e1173b P e715a811a82446ce8d5bb4aafd5b81c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } }
I20251211 20:52:46.598829 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1aeae8c18b6499e9dc1accabdc3f633" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:46.600013 711 leader_election.cc:343] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.600291 711 leader_election.cc:343] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.607383 1160 tablet_bootstrap.cc:492] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:46.607544 1160 ts_tablet_manager.cc:1403] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.001s
I20251211 20:52:46.608357 1160 raft_consensus.cc:359] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.608544 1160 raft_consensus.cc:385] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.608620 1160 raft_consensus.cc:740] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.608805 1160 consensus_queue.cc:260] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.609181 1160 ts_tablet_manager.cc:1434] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:46.609447 1160 tablet_bootstrap.cc:492] T 940dfd8781de444a9caa78dd363c7451 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:46.610756 1160 tablet_bootstrap.cc:654] T 940dfd8781de444a9caa78dd363c7451 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.621821 1160 tablet_bootstrap.cc:492] T 940dfd8781de444a9caa78dd363c7451 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:46.622081 1160 ts_tablet_manager.cc:1403] T 940dfd8781de444a9caa78dd363c7451 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20251211 20:52:46.622928 1160 raft_consensus.cc:359] T 940dfd8781de444a9caa78dd363c7451 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.623185 1160 raft_consensus.cc:385] T 940dfd8781de444a9caa78dd363c7451 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.623286 1160 raft_consensus.cc:740] T 940dfd8781de444a9caa78dd363c7451 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.623318 2692 raft_consensus.cc:493] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.624302 2808 raft_consensus.cc:493] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.624379 2501 consensus_queue.cc:1048] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:46.624429 2808 raft_consensus.cc:515] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.624945 2808 leader_election.cc:290] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.627528 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "95866de91f9d437d80f40b7962253610" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.628572 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 8b793241ee3b42b1a38e9f2f05e1173b (table default.Table_7506ffecc2b64648a9ba736e5298ddf4 [id=279f39d718ba45318f52f95e8aa13652]). Expected version 2 got 0
I20251211 20:52:46.629060 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet bfb52935296443c58190d4d06e2e6e9c (table default.Table_3721b5725e8c4297a392d3986a081128 [id=383392d835da4c50848068d83261cecc]). Expected version 7 got 0
I20251211 20:52:46.629489 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b2974f16034c44cf85ff85b5095cf6a0 (table default.Table_031de0f4cc7744bc9e320261f8b46696 [id=be36dabaa6d44325b4855b9f13bda847]). Expected version 1 got 0
I20251211 20:52:46.629849 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet e4647105ab57454d883ebfcde7bde891 (table default.Table_c817dec9a7674f699b3366fc302ead57 [id=d555d1fc6aef4a21b4d3f2804264d030]). Expected version 1 got 0
W20251211 20:52:46.630002 711 leader_election.cc:343] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.630388 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet ab4e7a9062134e489d6f9abe6ff26fb4 (table default.Table_4cbc8841b276406cb4434629653a5ce3 [id=b958532650374d609199cab9c07bb64b]). Expected version 2 got 0
I20251211 20:52:46.630729 2692 raft_consensus.cc:515] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.630671 1796 catalog_manager.cc:5654] T 645b08b33676420c96e880de3d460829 P e715a811a82446ce8d5bb4aafd5b81c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } }
I20251211 20:52:46.631338 2692 leader_election.cc:290] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.631348 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 645b08b33676420c96e880de3d460829 (table default.Table_e02933dd91ce4fe7a40d1c3b2f9b2d4b [id=6d42f260246e420d8818b3841926302b]). Expected version 2 got 0
I20251211 20:52:46.631613 1160 consensus_queue.cc:260] T 940dfd8781de444a9caa78dd363c7451 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.632108 1168 tablet_bootstrap.cc:492] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:46.632287 1168 ts_tablet_manager.cc:1403] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.169s user 0.013s sys 0.000s
I20251211 20:52:46.632297 1160 ts_tablet_manager.cc:1434] T 940dfd8781de444a9caa78dd363c7451 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20251211 20:52:46.634596 1160 tablet_bootstrap.cc:492] T b30d2054316b49fa8096345527a45d87 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:46.635872 710 leader_election.cc:304] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.636308 2806 raft_consensus.cc:2804] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:46.636426 2806 raft_consensus.cc:697] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
W20251211 20:52:46.636479 847 leader_election.cc:343] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: b631e6b3048e428f901af1b5b0b1ab3c
I20251211 20:52:46.636641 2806 consensus_queue.cc:237] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.637033 1160 tablet_bootstrap.cc:654] T b30d2054316b49fa8096345527a45d87 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.591069 1166 tablet_bootstrap.cc:492] T bd2bf1113dfe4b67b263b56f3d3db07d P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:46.598681 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d69fb1f6d9594afcbfb844860b41a377" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:46.638132 1045 raft_consensus.cc:2468] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:46.598912 1037 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e1aeae8c18b6499e9dc1accabdc3f633" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:46.611707 2764 raft_consensus.cc:493] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.638582 710 leader_election.cc:304] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.638533 2764 raft_consensus.cc:515] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.638818 2807 raft_consensus.cc:2804] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:46.638888 2807 raft_consensus.cc:493] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:46.638955 2807 raft_consensus.cc:3060] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.639111 2764 leader_election.cc:290] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:46.620867 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 41627cb27be8443fa37e514f9b3825a6 (DEFAULT_TABLE table=default.Table_c73d4009f4bc41b38aee5612313b30c8 [id=bd5290958c7249f39a6cc51f631f42a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:46.639473 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41627cb27be8443fa37e514f9b3825a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:46.641463 2807 raft_consensus.cc:515] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.641983 2807 leader_election.cc:290] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.644610 770 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6029c1fd563d4023a8cf7cfb8ffa0876" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:46.646186 982 leader_election.cc:343] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.626320 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "95866de91f9d437d80f40b7962253610" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:46.647030 1166 tablet_bootstrap.cc:654] T bd2bf1113dfe4b67b263b56f3d3db07d P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.647080 1044 raft_consensus.cc:2468] T 95866de91f9d437d80f40b7962253610 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:46.647439 710 leader_election.cc:304] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.647712 2807 raft_consensus.cc:2804] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:46.647848 2807 raft_consensus.cc:493] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:46.647944 1037 raft_consensus.cc:3060] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.647984 2807 raft_consensus.cc:3060] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.650470 1037 raft_consensus.cc:2468] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:46.650426 2807 raft_consensus.cc:515] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.650887 710 leader_election.cc:304] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.651175 2429 raft_consensus.cc:2804] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:46.651291 2807 leader_election.cc:290] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.651314 2429 raft_consensus.cc:697] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:46.651558 2429 consensus_queue.cc:237] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.651758 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d69fb1f6d9594afcbfb844860b41a377" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:46.651917 1045 raft_consensus.cc:3060] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.652978 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6029c1fd563d4023a8cf7cfb8ffa0876" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:46.654194 983 leader_election.cc:343] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.654304 983 leader_election.cc:304] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.654691 2764 raft_consensus.cc:2749] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.655088 1045 raft_consensus.cc:2468] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:46.655457 710 leader_election.cc:304] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:46.655764 2500 raft_consensus.cc:2804] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:46.655864 2500 raft_consensus.cc:697] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:46.656090 2500 consensus_queue.cc:237] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.660801 1037 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "95866de91f9d437d80f40b7962253610" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:46.661041 1037 raft_consensus.cc:3060] T 95866de91f9d437d80f40b7962253610 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.664130 1037 raft_consensus.cc:2468] T 95866de91f9d437d80f40b7962253610 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:46.665072 1168 raft_consensus.cc:359] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.665319 1168 raft_consensus.cc:385] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.665424 1168 raft_consensus.cc:740] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.665680 1168 consensus_queue.cc:260] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.665807 2692 raft_consensus.cc:493] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.665910 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d69fb1f6d9594afcbfb844860b41a377" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:46.665954 2692 raft_consensus.cc:515] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.666134 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "95866de91f9d437d80f40b7962253610" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:46.666260 1168 ts_tablet_manager.cc:1434] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.034s user 0.001s sys 0.001s
I20251211 20:52:46.666530 1168 tablet_bootstrap.cc:492] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:46.666515 2692 leader_election.cc:290] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.668399 1168 tablet_bootstrap.cc:654] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.668488 908 raft_consensus.cc:3060] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:46.669862 847 leader_election.cc:343] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 2197822e0f11484f9679c4858626ac2c
W20251211 20:52:46.678972 711 leader_election.cc:343] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.679397 711 leader_election.cc:343] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.679708 711 consensus_peers.cc:597] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:46.679962 711 consensus_peers.cc:597] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:46.680284 711 consensus_peers.cc:597] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:46.682906 2808 raft_consensus.cc:493] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.683033 2808 raft_consensus.cc:515] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.683200 2429 raft_consensus.cc:493] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.683282 2429 raft_consensus.cc:515] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.683687 2429 leader_election.cc:290] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.683737 2808 leader_election.cc:290] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.684276 2692 raft_consensus.cc:493] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.684439 2692 raft_consensus.cc:515] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.685057 2692 leader_election.cc:290] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.688114 1168 tablet_bootstrap.cc:492] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:46.688264 1168 ts_tablet_manager.cc:1403] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.003s
I20251211 20:52:46.689083 1168 raft_consensus.cc:359] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.689260 1168 raft_consensus.cc:385] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.689306 1168 raft_consensus.cc:740] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.689484 1168 consensus_queue.cc:260] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:46.689944 847 leader_election.cc:343] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: fb030a62796248869d46b682656eaeb0
I20251211 20:52:46.691324 1168 ts_tablet_manager.cc:1434] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:46.692312 1168 tablet_bootstrap.cc:492] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:46.694187 1168 tablet_bootstrap.cc:654] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.698709 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 5453f68789fd4e89b69e98860050865b (table default.Table_b7bf8c5ad5304175a1f2f6a03887f725 [id=50d88b663a4d4891b113aee9c5906699]). Expected version 1 got 0
I20251211 20:52:46.699055 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet ee7b93b0b87541d9af6eb9e68b4cea41 (table default.Table_b6e6aea68c88426fb01353146c06b6ac [id=a4f984b4263a4bc38f6248628f643ae3]). Expected version 3 got 0
I20251211 20:52:46.699290 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet df457783b2534b3ea23bb1fd529ce268 (table default.Table_6e9f3ca164384cc8ac4501deb3f0c6aa [id=342c2469d6e74e3e9217236e3aa863d6]). Expected version 1 got 0
I20251211 20:52:46.699666 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 18f125dd9efc401181f0644dad7aeffe (table default.Table_fa6592c2e2624d81ad21abb13ed36045 [id=9b48fc2b39db4fd5a4e5da19e34427dc]). Expected version 1 got 0
I20251211 20:52:46.699916 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b115d737d7fc483d9f5a1f040b511bae (table default.Table_6d94053661f142b79292e19752df02df [id=c5be59a2c3934b3789641ceeb2d4d6e2]). Expected version 2 got 0
I20251211 20:52:46.700135 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet da81bf8854a24954ba4548185812295d (table default.Table_18262e22c26d4d66ba4c6134fbde0368 [id=da8cf2b99b42409091484a9513623d6f]). Expected version 1 got 0
I20251211 20:52:46.700356 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet f00128ba84c245d9a0e35f26bc842b3e (table default.Table_6f8c133c7a364b93af2c83d8911a0ac3 [id=6a21c38ee7224a02b937df088494f5d8]). Expected version 1 got 0
I20251211 20:52:46.700928 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet aa50ddb2789b4be3bfa9aedef9287109 (table default.Table_44f971096d1e41f480e594985ffd5c17 [id=d4e1349374a346a5b8be180f9bdc329a]). Expected version 1 got 0
I20251211 20:52:46.701263 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet f94b656e3c1844509fdae36e33b263e8 (table default.Table_ed03ad475bed4804a65abe20efef0625 [id=e23f4305669a474aafa18928f130de3a]). Expected version 2 got 0
I20251211 20:52:46.701644 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 4c7b74f370b042ee8575f821c07dc085 (table default.Table_731826f26f9940b8afe425d694b642d9 [id=17dd9cea9ac844128ff9f82adb3f919f]). Expected version 1 got 0
I20251211 20:52:46.702044 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b85af79c2574443998e847bc19c53b69 (table default.Table_fa3f2f115cd043a9aa6be2b070d320cb [id=15fad5e1cf8d4ea08dae8275a29a11dd]). Expected version 1 got 0
I20251211 20:52:46.702483 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 8dc4e730951144d78b77280f6cce6d0e (table default.Table_890c9027415a44ccaf95c0adede51992 [id=7bc8a2b91fe54b91b6594b371fea7bd3]). Expected version 2 got 0
I20251211 20:52:46.704032 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 969a8587019c4202b6e705d126f83d44 (table default.Table_6460313002684bba93b30ad8631f0f49 [id=22347b15537d451d9927a17e8e5a202a]). Expected version 1 got 0
I20251211 20:52:46.704447 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3dfea28c44d84c68850db65a30d52ee6 (table default.Table_c504f8f3796f43b697488bd604268543 [id=2af76b9d32094a4ca656060a4478e08a]). Expected version 1 got 0
I20251211 20:52:46.680812 1166 tablet_bootstrap.cc:492] T bd2bf1113dfe4b67b263b56f3d3db07d P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:46.704917 1166 ts_tablet_manager.cc:1403] T bd2bf1113dfe4b67b263b56f3d3db07d P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.148s user 0.003s sys 0.001s
I20251211 20:52:46.705080 710 leader_election.cc:304] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.705389 2500 raft_consensus.cc:2804] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:46.705518 2500 raft_consensus.cc:697] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:46.705766 2500 consensus_queue.cc:237] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.705842 1166 raft_consensus.cc:359] T bd2bf1113dfe4b67b263b56f3d3db07d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.705989 1166 raft_consensus.cc:385] T bd2bf1113dfe4b67b263b56f3d3db07d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.706032 1166 raft_consensus.cc:740] T bd2bf1113dfe4b67b263b56f3d3db07d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.684257 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d43210ec61af423ebde624397dea9f2e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:46.706225 1048 raft_consensus.cc:2468] T d43210ec61af423ebde624397dea9f2e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:46.706475 1166 consensus_queue.cc:260] T bd2bf1113dfe4b67b263b56f3d3db07d P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.706624 710 leader_election.cc:304] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:46.706919 2807 raft_consensus.cc:2804] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:46.706982 2807 raft_consensus.cc:493] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:46.707032 2807 raft_consensus.cc:3060] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.708014 1166 ts_tablet_manager.cc:1434] T bd2bf1113dfe4b67b263b56f3d3db07d P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:46.684475 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7be44df39a8f4d6da793479d425fa684" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:46.708159 1047 raft_consensus.cc:2468] T 7be44df39a8f4d6da793479d425fa684 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:46.708667 710 leader_election.cc:304] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:46.710326 2807 raft_consensus.cc:515] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.710772 1166 tablet_bootstrap.cc:492] T de508d79f30f489cb745401f53115165 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:46.710875 2807 leader_election.cc:290] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.711043 2807 raft_consensus.cc:2804] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:46.711103 2807 raft_consensus.cc:493] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:46.711156 2807 raft_consensus.cc:3060] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.711321 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d43210ec61af423ebde624397dea9f2e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:46.711468 1047 raft_consensus.cc:3060] T d43210ec61af423ebde624397dea9f2e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.714002 1047 raft_consensus.cc:2468] T d43210ec61af423ebde624397dea9f2e P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:46.714452 710 leader_election.cc:304] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:46.714705 2808 raft_consensus.cc:2804] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:46.714820 2808 raft_consensus.cc:697] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:46.715060 2808 consensus_queue.cc:237] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:46.717124 847 leader_election.cc:343] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: b631e6b3048e428f901af1b5b0b1ab3c
I20251211 20:52:46.717242 847 leader_election.cc:304] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:46.717607 2692 raft_consensus.cc:2749] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:46.718043 847 leader_election.cc:343] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 2197822e0f11484f9679c4858626ac2c
I20251211 20:52:46.718138 847 leader_election.cc:304] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:46.718792 2692 raft_consensus.cc:2749] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.723316 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7be44df39a8f4d6da793479d425fa684" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.724462 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d43210ec61af423ebde624397dea9f2e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.725687 2807 raft_consensus.cc:515] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.727790 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d43210ec61af423ebde624397dea9f2e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:46.728623 2807 leader_election.cc:290] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.728750 1166 tablet_bootstrap.cc:654] T de508d79f30f489cb745401f53115165 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.729573 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7be44df39a8f4d6da793479d425fa684" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:46.729758 1047 raft_consensus.cc:3060] T 7be44df39a8f4d6da793479d425fa684 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.731808 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 59acb8474e114bf7ae7f1f043d611459 (table default.Table_c7ed8c0c309c40a3a371ffa130b47cd2 [id=ed1d0670426945aba888b818ad657225]). Expected version 1 got 0
I20251211 20:52:46.732121 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 16d2ff8925e342729a71c93db378d031 (table default.Table_34e6ed4f13da437ea50d349b92e83bd6 [id=4a74b9a60cbb4198b55cee630b2f4bef]). Expected version 2 got 0
I20251211 20:52:46.732467 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 5773c13034714a008c8d7dbe777b4828 (table default.Table_66eb1b021eed464f82f9cd223c88297b [id=94ff09564cd649c7a2d0e94b0f3c7b3a]). Expected version 1 got 0
I20251211 20:52:46.732929 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 78f77d8916d84edabbe95ce17d8df2c1 (table default.Table_e2af0e2fbd3d46d595136c7a7bf24e4c [id=ddc12ae6d8ed4587b5ef4da3ef81a055]). Expected version 1 got 0
I20251211 20:52:46.733389 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 46cb1d16a0e94b3a833f32c635213fd2 (table default.Table_3482fbfc5b144a7198a7d01aa003cb6b [id=1bdaf5ce0f6b4b99a038f9f9efb828f6]). Expected version 1 got 0
I20251211 20:52:46.733843 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet e3f711f02af843adb4f5c3c750924e33 (table default.Table_34bd5155aa3b4ef7818022ba9346c074 [id=83a61d23e80946439a14096ab3ab1bd9]). Expected version 2 got 0
I20251211 20:52:46.734058 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b6cc819332e1445c96ce3ffac4861454 (table default.Table_db4c0f706e0643b39d39ffa84352826d [id=4d459c42bf584068ac3851227d80331a]). Expected version 1 got 0
I20251211 20:52:46.734275 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet d37a7d0d662e45079b0cfc02765d2e01 (table default.Table_b26cc06e1b0745e08181124e1cbce934 [id=8a12299efe714617a970fc05c3bb6232]). Expected version 1 got 0
I20251211 20:52:46.734601 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 5e47af616e1b4d2db3780c1d67cdcb47 (table default.Table_296bdf30a11541c389afe3df0b0f667a [id=052edcbde60c41449d468c8a449fc3f1]). Expected version 1 got 0
I20251211 20:52:46.734884 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet e89d4d76bf64453892a4d0b55fcc3454 (table default.Table_fba99384e9f14160b3e3504f8d1d0b07 [id=01bba75f854d4c338d6c09a446dd7499]). Expected version 1 got 0
I20251211 20:52:46.735209 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 7ca98b43863a47c091baae0a6ce65630 (table default.Table_f0a70914acc94231a4e84155c3e4d13f [id=56c070ad80624c698ebe2e01b4929b6e]). Expected version 1 got 0
I20251211 20:52:46.735780 1796 catalog_manager.cc:5654] T 3de5b34b96404cdc9c191fed7f055638 P e715a811a82446ce8d5bb4aafd5b81c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } }
W20251211 20:52:46.729728 711 leader_election.cc:343] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.741072 711 leader_election.cc:343] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.744501 1168 tablet_bootstrap.cc:492] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:46.744675 1168 ts_tablet_manager.cc:1403] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.053s user 0.000s sys 0.016s
W20251211 20:52:46.745549 711 leader_election.cc:343] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.745543 1168 raft_consensus.cc:359] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.745748 1168 raft_consensus.cc:385] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.745802 1168 raft_consensus.cc:740] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.745975 1168 consensus_queue.cc:260] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.746455 1168 ts_tablet_manager.cc:1434] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:46.746399 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7be44df39a8f4d6da793479d425fa684" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:46.746752 1168 tablet_bootstrap.cc:492] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:46.748466 1168 tablet_bootstrap.cc:654] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.748709 2670 raft_consensus.cc:493] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.748844 2670 raft_consensus.cc:515] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.749354 2670 leader_election.cc:290] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:46.750279 847 leader_election.cc:343] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: bb112b6431af42a6a7b328dfdefbc2a2
I20251211 20:52:46.751943 1047 raft_consensus.cc:2468] T 7be44df39a8f4d6da793479d425fa684 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
W20251211 20:52:46.752640 847 leader_election.cc:343] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: fb030a62796248869d46b682656eaeb0
I20251211 20:52:46.752772 847 leader_election.cc:304] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:46.753026 2670 raft_consensus.cc:2749] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.753885 908 raft_consensus.cc:3060] T 41c30bded6504bbc9f0af8f34dc81e8d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:46.754453 711 leader_election.cc:343] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.755045 847 leader_election.cc:343] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: bb112b6431af42a6a7b328dfdefbc2a2
I20251211 20:52:46.755146 847 leader_election.cc:304] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:46.755674 2670 raft_consensus.cc:2749] T bb112b6431af42a6a7b328dfdefbc2a2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.756258 710 leader_election.cc:304] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.756497 2807 raft_consensus.cc:2804] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:46.756611 2807 raft_consensus.cc:697] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:46.756851 2807 consensus_queue.cc:237] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.757866 2808 consensus_queue.cc:1048] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:46.758710 1160 tablet_bootstrap.cc:492] T b30d2054316b49fa8096345527a45d87 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:46.758832 1160 ts_tablet_manager.cc:1403] T b30d2054316b49fa8096345527a45d87 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.124s user 0.029s sys 0.030s
I20251211 20:52:46.758865 1168 tablet_bootstrap.cc:492] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:46.758996 1168 ts_tablet_manager.cc:1403] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.005s
I20251211 20:52:46.759768 1168 raft_consensus.cc:359] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.759928 1168 raft_consensus.cc:385] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.759977 1168 raft_consensus.cc:740] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.760183 1168 consensus_queue.cc:260] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.760691 1168 ts_tablet_manager.cc:1434] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:46.761134 1160 raft_consensus.cc:359] T b30d2054316b49fa8096345527a45d87 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.761348 1160 raft_consensus.cc:385] T b30d2054316b49fa8096345527a45d87 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.762130 1160 raft_consensus.cc:740] T b30d2054316b49fa8096345527a45d87 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.762430 1160 consensus_queue.cc:260] T b30d2054316b49fa8096345527a45d87 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.763080 1168 tablet_bootstrap.cc:492] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:46.763329 1166 tablet_bootstrap.cc:492] T de508d79f30f489cb745401f53115165 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:46.763459 1166 ts_tablet_manager.cc:1403] T de508d79f30f489cb745401f53115165 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.053s user 0.001s sys 0.001s
I20251211 20:52:46.764505 1166 raft_consensus.cc:359] T de508d79f30f489cb745401f53115165 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.764683 1166 raft_consensus.cc:385] T de508d79f30f489cb745401f53115165 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.764734 1166 raft_consensus.cc:740] T de508d79f30f489cb745401f53115165 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.764945 1166 consensus_queue.cc:260] T de508d79f30f489cb745401f53115165 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.765151 1168 tablet_bootstrap.cc:654] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.765506 1166 ts_tablet_manager.cc:1434] T de508d79f30f489cb745401f53115165 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:46.765784 1166 tablet_bootstrap.cc:492] T 04679dfddf564d6f8c16d1c257e29b41 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:46.765776 1160 ts_tablet_manager.cc:1434] T b30d2054316b49fa8096345527a45d87 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20251211 20:52:46.767135 1166 tablet_bootstrap.cc:654] T 04679dfddf564d6f8c16d1c257e29b41 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.767270 1160 tablet_bootstrap.cc:492] T 50f9f29c064c4f938a0f20f302ab5e8a P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:46.770278 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet d6923cfd826d495291d9ac8e0b3b44c6 (table default.Table_68ccb2052d824ccf92f7162d2961fcb1 [id=9ac9591d4f04482d926710fdf6e797c6]). Expected version 1 got 0
I20251211 20:52:46.775367 1160 tablet_bootstrap.cc:654] T 50f9f29c064c4f938a0f20f302ab5e8a P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.775741 1168 tablet_bootstrap.cc:492] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:46.775892 1168 ts_tablet_manager.cc:1403] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20251211 20:52:46.776723 1168 raft_consensus.cc:359] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.776927 1168 raft_consensus.cc:385] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.776988 1168 raft_consensus.cc:740] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.777222 1168 consensus_queue.cc:260] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.778774 1168 ts_tablet_manager.cc:1434] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:46.779796 1168 tablet_bootstrap.cc:492] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:46.781414 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 33fd09ea06ee47908b85911e4ee79708 (table default.Table_3c3cd48c8fe5411e96cdf811d492925d [id=30620a01c27f456bac05fa7c02a2cb4d]). Expected version 2 got 0
I20251211 20:52:46.781744 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 6212b3b4425543168837946a54d60834 (table default.Table_8c30c4151d484db0bb9e2b9ca262248d [id=2d489cd7d73e403d84bf10960e6a2dfb]). Expected version 1 got 0
I20251211 20:52:46.781978 1168 tablet_bootstrap.cc:654] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.785365 908 raft_consensus.cc:3060] T b30d2054316b49fa8096345527a45d87 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.786726 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 7be44df39a8f4d6da793479d425fa684 (table default.Table_f6a10556ff104db5bd5ecc9fc380e228 [id=f75813add54e46189ff1bbed82d83ba7]). Expected version 1 got 0
I20251211 20:52:46.789639 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 497c10742f01476db885bd70c4d2c491 (table default.Table_18b21232dfa74378b27defe672bb2574 [id=602d7eee703948d99a0edcb7cac0651b]). Expected version 1 got 0
I20251211 20:52:46.789933 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3003ef71a3554c71a3b24cdfb9c293bc (table default.Table_92efd469dd324f7d886e2cbc90ece61c [id=6e7fb3271e0c48c0bb9d56fe4fccee3a]). Expected version 1 got 0
I20251211 20:52:46.790213 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet fd43a27db69044b68826cda87e16a086 (table default.Table_a8ebfd7d7a7f4f6cae8b6b02b99b80b0 [id=f1d32002c612403a82bdff5baf209b9e]). Expected version 1 got 0
I20251211 20:52:46.790582 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 94658fcf512a48d0a992a7e2e0f7c176 (table default.Table_c9d86e82aebb4ef0bc83789720352e38 [id=97377a8ef56c4cdebe95d15955207404]). Expected version 1 got 0
I20251211 20:52:46.790784 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet fb29f983c4524a49bdc5c0c1144ad708 (table default.Table_7809ac85fdfe4c1d9c58afb54edecb27 [id=c4b0bfff99f64888beb70d9154d2f156]). Expected version 1 got 0
I20251211 20:52:46.791275 1796 catalog_manager.cc:5654] T a9bb22dce13f4c9da76ca446715cd99a P e715a811a82446ce8d5bb4aafd5b81c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } }
I20251211 20:52:46.796162 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 4fe5c534b572491c911aae01eaa79bae (table default.Table_c1f9dca5c2144d50b67681dcc6800e77 [id=14fa8debfe5443ce81587f02dfa15847]). Expected version 2 got 0
I20251211 20:52:46.796499 1168 tablet_bootstrap.cc:492] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:46.796634 1168 ts_tablet_manager.cc:1403] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.000s
I20251211 20:52:46.797336 1168 raft_consensus.cc:359] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.797516 1168 raft_consensus.cc:385] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.797564 1168 raft_consensus.cc:740] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.797760 1168 consensus_queue.cc:260] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.798266 1168 ts_tablet_manager.cc:1434] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:46.798528 1168 tablet_bootstrap.cc:492] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:46.800299 1168 tablet_bootstrap.cc:654] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.801618 2501 consensus_queue.cc:1048] T bbe3529c301a4a65b58b634a443e1fb6 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:46.805315 2670 consensus_queue.cc:1048] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:46.806912 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 51d12723f5194afb97d2c05fcbaf3776 (table default.Table_823505200e3142dbbe8cd56ac78daf55 [id=ec0bb10933fa4529bef3580bee6409e7]). Expected version 2 got 0
I20251211 20:52:46.807250 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 2e800d8cc5ea4535a065d7d9243326b4 (table default.Table_083c478bedec45a5bf5203e4ed333860 [id=8baf0ea87d734d58896adbd0d6ef99c2]). Expected version 1 got 0
I20251211 20:52:46.807415 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet d43210ec61af423ebde624397dea9f2e (table default.Table_af0df4d1ee144455a6041e5a6a0b9b80 [id=7c5b2e33fa1c4b12b3941577360908a8]). Expected version 3 got 0
I20251211 20:52:46.807544 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 8675e2a18f0f4a0e95a87005cf82d8b4 (table default.Table_bd9633c55bd242c18aac3a7f2c18a329 [id=767f50bee5464316a18f73e64bab817a]). Expected version 2 got 0
I20251211 20:52:46.807952 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 861d2d70a24946699dccd65cab05195e (table default.Table_819c4206e91e41358ac71e82185b4140 [id=28be836b71ed4107af7adc3fcb948b5e]). Expected version 1 got 0
I20251211 20:52:46.808182 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet f26ba8f4bd3a4a1f8fa2ba6e80d543f8 (table default.Table_71e9c158dc22435a8fbb02aecca8919d [id=b966d0c0b49d4023894950c5dc06f678]). Expected version 2 got 0
I20251211 20:52:46.812039 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 1c4b2c22bd4c456eb77af45576de466f (table default.Table_e3a99b043e4f420a93ebfee6aa87f2ec [id=3e7d6cefcb7d412191c6540e9ad77c2d]). Expected version 2 got 0
I20251211 20:52:46.813640 1160 tablet_bootstrap.cc:492] T 50f9f29c064c4f938a0f20f302ab5e8a P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:46.813848 2764 raft_consensus.cc:493] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.813970 2764 raft_consensus.cc:515] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.814496 2764 leader_election.cc:290] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:46.815641 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01104975fd8b470aa026578b97ff6b8d" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.815654 770 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01104975fd8b470aa026578b97ff6b8d" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:46.816221 982 leader_election.cc:343] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.816536 983 leader_election.cc:343] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.816633 983 leader_election.cc:304] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.817060 2764 raft_consensus.cc:2749] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.820443 1160 ts_tablet_manager.cc:1403] T 50f9f29c064c4f938a0f20f302ab5e8a P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.054s user 0.003s sys 0.003s
I20251211 20:52:46.821352 1160 raft_consensus.cc:359] T 50f9f29c064c4f938a0f20f302ab5e8a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.821611 1160 raft_consensus.cc:385] T 50f9f29c064c4f938a0f20f302ab5e8a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.821719 1160 raft_consensus.cc:740] T 50f9f29c064c4f938a0f20f302ab5e8a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.822012 1160 consensus_queue.cc:260] T 50f9f29c064c4f938a0f20f302ab5e8a P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.823482 1160 ts_tablet_manager.cc:1434] T 50f9f29c064c4f938a0f20f302ab5e8a P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:46.824074 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet dfd194cf00f549c8937cb6ec40987240 (table default.Table_8db9acb9d35149fdabcb9a1d24061b46 [id=d919c6132b3c466e8a1404f981b003b8]). Expected version 1 got 0
I20251211 20:52:46.828068 1160 tablet_bootstrap.cc:492] T 3acfef6f082641a199b80eefcfc772ac P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:46.829780 1160 tablet_bootstrap.cc:654] T 3acfef6f082641a199b80eefcfc772ac P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.830371 1166 tablet_bootstrap.cc:492] T 04679dfddf564d6f8c16d1c257e29b41 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:46.830530 1166 ts_tablet_manager.cc:1403] T 04679dfddf564d6f8c16d1c257e29b41 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.065s user 0.002s sys 0.000s
I20251211 20:52:46.830715 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet bc4e7362e6d045da81bf39f4135f500e (table default.Table_328b7dd76e9e407c83b47a0f51cfcce5 [id=9e7c08f833e14a90b17a9b9f60b702de]). Expected version 1 got 0
I20251211 20:52:46.830993 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet ce1799e6604f4ed7875ddedecab6f327 (table default.Table_5db1ac98e36d45b1b01889ee275ad83d [id=a6849726434d41d29dce12f47066a685]). Expected version 1 got 0
I20251211 20:52:46.831487 2429 raft_consensus.cc:493] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.831651 2429 raft_consensus.cc:515] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.832257 2429 leader_election.cc:290] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.833039 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "58e35de0792a464a818170886ca8d8c9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:46.833303 1047 raft_consensus.cc:2468] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:46.833495 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "58e35de0792a464a818170886ca8d8c9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:46.833985 711 leader_election.cc:343] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.834201 710 leader_election.cc:304] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.833926 1796 catalog_manager.cc:5654] T fed300d31c414f4a9b27c66e78097688 P e715a811a82446ce8d5bb4aafd5b81c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } }
I20251211 20:52:46.834614 2429 raft_consensus.cc:2804] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:46.834684 2429 raft_consensus.cc:493] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:46.834740 2429 raft_consensus.cc:3060] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.837502 2429 raft_consensus.cc:515] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.838822 2429 leader_election.cc:290] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.838899 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "58e35de0792a464a818170886ca8d8c9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:46.839071 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "58e35de0792a464a818170886ca8d8c9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:46.839218 1047 raft_consensus.cc:3060] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:46.839855 711 leader_election.cc:343] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.841176 1168 tablet_bootstrap.cc:492] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:46.841318 1168 ts_tablet_manager.cc:1403] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.043s user 0.009s sys 0.013s
I20251211 20:52:46.842046 1047 raft_consensus.cc:2468] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:46.842298 2429 raft_consensus.cc:493] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.842252 1168 raft_consensus.cc:359] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.842450 1168 raft_consensus.cc:385] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.842502 1168 raft_consensus.cc:740] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.842514 2806 raft_consensus.cc:493] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.842649 2806 raft_consensus.cc:515] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.842710 1168 consensus_queue.cc:260] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.842772 710 leader_election.cc:304] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.843001 2808 raft_consensus.cc:2804] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:46.843104 2808 raft_consensus.cc:697] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:46.835021 1796 catalog_manager.cc:5654] T 3b2c6064c5eb4c0f8979c485f5d85a75 P e715a811a82446ce8d5bb4aafd5b81c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } }
I20251211 20:52:46.843238 2806 leader_election.cc:290] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.843302 2808 consensus_queue.cc:237] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.843405 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3b2c6064c5eb4c0f8979c485f5d85a75 (table default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f]). Expected version 1 got 0
I20251211 20:52:46.843645 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 7b9f569554fb4836ac2b62542c3ecf7e (table default.Table_1a73d567e8914a83974e3e8f5f58be0b [id=6ef691749df24251a7a7c5919c219754]). Expected version 1 got 0
I20251211 20:52:46.844417 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d391d4e6a101462d8d1220f1e17d0daa" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:46.844671 1047 raft_consensus.cc:2468] T d391d4e6a101462d8d1220f1e17d0daa P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:46.842602 2429 raft_consensus.cc:515] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.845070 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 4cf0cde801bf44449e539d2799b895a7 (table default.Table_907c596bdddc4edca25d14cbacebb23f [id=321235280d1d4fa0bae5cc0333860f85]). Expected version 1 got 0
I20251211 20:52:46.845232 710 leader_election.cc:304] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:46.845444 2500 raft_consensus.cc:2804] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:46.845512 2500 raft_consensus.cc:493] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:46.845568 2500 raft_consensus.cc:3060] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.846096 2429 leader_election.cc:290] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.846170 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c26c3d0170f64a41ae8944c6668cc5f3" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:46.846354 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d391d4e6a101462d8d1220f1e17d0daa" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.846422 1047 raft_consensus.cc:2468] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:46.846823 710 leader_election.cc:304] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:46.847287 2429 raft_consensus.cc:2804] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:46.847379 2429 raft_consensus.cc:493] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:46.847467 2429 raft_consensus.cc:3060] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:46.847946 711 leader_election.cc:343] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.848953 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c26c3d0170f64a41ae8944c6668cc5f3" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:46.849654 711 leader_election.cc:343] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.849695 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 72af55c8045b4f8f8e116a5782794d81 (table default.Table_2cff6fab088c4d939b13b504c133cc12 [id=b6c4b72063b3428593d5650a160dc66b]). Expected version 1 got 0
I20251211 20:52:46.850579 2500 raft_consensus.cc:515] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.851209 2500 leader_election.cc:290] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.851675 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d391d4e6a101462d8d1220f1e17d0daa" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:46.851713 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d391d4e6a101462d8d1220f1e17d0daa" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:46.851851 1047 raft_consensus.cc:3060] T d391d4e6a101462d8d1220f1e17d0daa P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:46.852190 711 leader_election.cc:343] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.831251 1166 raft_consensus.cc:359] T 04679dfddf564d6f8c16d1c257e29b41 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.852305 1166 raft_consensus.cc:385] T 04679dfddf564d6f8c16d1c257e29b41 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.852387 1166 raft_consensus.cc:740] T 04679dfddf564d6f8c16d1c257e29b41 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.852615 1166 consensus_queue.cc:260] T 04679dfddf564d6f8c16d1c257e29b41 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.854638 1047 raft_consensus.cc:2468] T d391d4e6a101462d8d1220f1e17d0daa P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:46.855893 1168 ts_tablet_manager.cc:1434] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.014s user 0.001s sys 0.001s
I20251211 20:52:46.856340 1166 ts_tablet_manager.cc:1434] T 04679dfddf564d6f8c16d1c257e29b41 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.026s user 0.002s sys 0.000s
I20251211 20:52:46.856925 1166 tablet_bootstrap.cc:492] T 79d92736bb1c46d2bfa5678b2d396ec2 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:46.850632 2429 raft_consensus.cc:515] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.857470 2429 leader_election.cc:290] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.857748 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c26c3d0170f64a41ae8944c6668cc5f3" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:46.857932 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c26c3d0170f64a41ae8944c6668cc5f3" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:46.858076 1048 raft_consensus.cc:3060] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.860893 1048 raft_consensus.cc:2468] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:46.862128 1166 tablet_bootstrap.cc:654] T 79d92736bb1c46d2bfa5678b2d396ec2 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.857857 1168 tablet_bootstrap.cc:492] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:46.858266 711 leader_election.cc:343] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.862717 710 leader_election.cc:304] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.863560 2429 raft_consensus.cc:2804] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:46.863986 2429 raft_consensus.cc:697] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:46.864266 2429 consensus_queue.cc:237] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.867672 710 leader_election.cc:304] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:46.867893 847 consensus_peers.cc:597] T 4df68d1b6a91458e9f212ae81aa25438 P eb0f1933d9a74fccaa4618047fc05c41 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4df68d1b6a91458e9f212ae81aa25438. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:46.867739 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b81a4cef27164824b52695039744db15 (table default.Table_defdeab28749411590cf75b5251f604b [id=11352566d1744193a40a8bb605aa3912]). Expected version 1 got 0
I20251211 20:52:46.868279 2807 raft_consensus.cc:2804] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:46.868458 2807 raft_consensus.cc:697] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:46.868835 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 7ffa06c4a7ee48b0b337a27aff090592 (table default.Table_647731023b044ebb8954f7fdcc3faf97 [id=4b91abd6e9e94a8a941b33e62e23c083]). Expected version 1 got 0
I20251211 20:52:46.869115 2807 consensus_queue.cc:237] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.870098 1168 tablet_bootstrap.cc:654] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:46.876027 711 consensus_peers.cc:597] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:46.876958 1168 tablet_bootstrap.cc:492] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:46.878371 1160 tablet_bootstrap.cc:492] T 3acfef6f082641a199b80eefcfc772ac P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:46.878513 1160 ts_tablet_manager.cc:1403] T 3acfef6f082641a199b80eefcfc772ac P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.051s user 0.002s sys 0.002s
I20251211 20:52:46.879161 1160 raft_consensus.cc:359] T 3acfef6f082641a199b80eefcfc772ac P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.879348 1160 raft_consensus.cc:385] T 3acfef6f082641a199b80eefcfc772ac P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.879395 1160 raft_consensus.cc:740] T 3acfef6f082641a199b80eefcfc772ac P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.879616 1168 ts_tablet_manager.cc:1403] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.000s
I20251211 20:52:46.879675 1160 consensus_queue.cc:260] T 3acfef6f082641a199b80eefcfc772ac P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.880285 1160 ts_tablet_manager.cc:1434] T 3acfef6f082641a199b80eefcfc772ac P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:46.880558 1160 tablet_bootstrap.cc:492] T 2e33606930774fe4aafc6e3819618af8 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:46.882180 1160 tablet_bootstrap.cc:654] T 2e33606930774fe4aafc6e3819618af8 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.883111 1168 raft_consensus.cc:359] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.883718 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 5b3e8ec3faf445ac9b746e65dc4dbfe0 (table default.Table_7fe80ff5483a4b0292091119980f6639 [id=3c3462a0c09748cea0d2ffb544ae2f13]). Expected version 1 got 0
I20251211 20:52:46.883785 1168 raft_consensus.cc:385] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.883895 1168 raft_consensus.cc:740] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
W20251211 20:52:46.884150 983 consensus_peers.cc:597] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:46.884119 1168 consensus_queue.cc:260] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.884328 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 84bb3583857848459e0e865283de623d (table default.Table_92281c5fcfee4355a79215ffedeee588 [id=6e5fe0a78e4b48f98dac028802f1ddf4]). Expected version 4 got 0
I20251211 20:52:46.884750 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet a7afcce522494c9ca1d80e6eea913e6e (table default.Table_cb6a4bac7b694f33a37de2ee78c91ca0 [id=8940f18527c249a9baefc170ed82f45a]). Expected version 1 got 0
I20251211 20:52:46.885004 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 1f4eb1a174c648c897ae2b7612e44585 (table default.Table_e70873b71ba24d7eae548e2acfd243b3 [id=87adcc552a704d56bd2fe812fb3b791f]). Expected version 1 got 0
I20251211 20:52:46.885246 1168 ts_tablet_manager.cc:1434] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20251211 20:52:46.885396 1796 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 0302cc8613934c43a0650f081cf98797 (table default.Table_bf827b216f1b43c196923dc1828ff0d0 [id=ed371e073250439fa44330c7101e9387]). Expected version 1 got 0
I20251211 20:52:46.886080 1166 tablet_bootstrap.cc:492] T 79d92736bb1c46d2bfa5678b2d396ec2 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:46.886222 1166 ts_tablet_manager.cc:1403] T 79d92736bb1c46d2bfa5678b2d396ec2 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.003s
I20251211 20:52:46.886332 1168 tablet_bootstrap.cc:492] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:46.886786 908 raft_consensus.cc:3060] T 3acfef6f082641a199b80eefcfc772ac P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.886925 1166 raft_consensus.cc:359] T 79d92736bb1c46d2bfa5678b2d396ec2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.887086 1166 raft_consensus.cc:385] T 79d92736bb1c46d2bfa5678b2d396ec2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.887133 1166 raft_consensus.cc:740] T 79d92736bb1c46d2bfa5678b2d396ec2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.887331 1166 consensus_queue.cc:260] T 79d92736bb1c46d2bfa5678b2d396ec2 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.887979 1168 tablet_bootstrap.cc:654] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.900561 1160 tablet_bootstrap.cc:492] T 2e33606930774fe4aafc6e3819618af8 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:46.900714 1160 ts_tablet_manager.cc:1403] T 2e33606930774fe4aafc6e3819618af8 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20251211 20:52:46.901465 1160 raft_consensus.cc:359] T 2e33606930774fe4aafc6e3819618af8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.901647 1160 raft_consensus.cc:385] T 2e33606930774fe4aafc6e3819618af8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.901696 1160 raft_consensus.cc:740] T 2e33606930774fe4aafc6e3819618af8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.901890 1160 consensus_queue.cc:260] T 2e33606930774fe4aafc6e3819618af8 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.902238 2825 raft_consensus.cc:493] T 50f9f29c064c4f938a0f20f302ab5e8a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.902366 1160 ts_tablet_manager.cc:1434] T 2e33606930774fe4aafc6e3819618af8 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:46.902347 2825 raft_consensus.cc:515] T 50f9f29c064c4f938a0f20f302ab5e8a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.902871 1166 ts_tablet_manager.cc:1434] T 79d92736bb1c46d2bfa5678b2d396ec2 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.017s user 0.001s sys 0.001s
I20251211 20:52:46.902894 2825 leader_election.cc:290] T 50f9f29c064c4f938a0f20f302ab5e8a P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.903205 1166 tablet_bootstrap.cc:492] T f2ae084982f44fdf8bfecdd2bc4c01b9 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:46.904377 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "50f9f29c064c4f938a0f20f302ab5e8a" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:46.904851 1166 tablet_bootstrap.cc:654] T f2ae084982f44fdf8bfecdd2bc4c01b9 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.905978 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "50f9f29c064c4f938a0f20f302ab5e8a" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:46.907424 847 leader_election.cc:304] T 50f9f29c064c4f938a0f20f302ab5e8a P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:46.907872 2759 raft_consensus.cc:2749] T 50f9f29c064c4f938a0f20f302ab5e8a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.908448 961 heartbeater.cc:499] Master 127.0.104.61:40749 was elected leader, sending a full tablet report...
I20251211 20:52:46.913833 1166 tablet_bootstrap.cc:492] T f2ae084982f44fdf8bfecdd2bc4c01b9 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:46.914003 1166 ts_tablet_manager.cc:1403] T f2ae084982f44fdf8bfecdd2bc4c01b9 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20251211 20:52:46.914718 1166 raft_consensus.cc:359] T f2ae084982f44fdf8bfecdd2bc4c01b9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.914911 1166 raft_consensus.cc:385] T f2ae084982f44fdf8bfecdd2bc4c01b9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.914960 1166 raft_consensus.cc:740] T f2ae084982f44fdf8bfecdd2bc4c01b9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.915194 1166 consensus_queue.cc:260] T f2ae084982f44fdf8bfecdd2bc4c01b9 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.916363 1166 ts_tablet_manager.cc:1434] T f2ae084982f44fdf8bfecdd2bc4c01b9 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:46.917011 1166 tablet_bootstrap.cc:492] T 969a8587019c4202b6e705d126f83d44 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:46.918182 1160 tablet_bootstrap.cc:492] T c91ff29847d54e55b19321652a857e9f P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:46.919039 1166 tablet_bootstrap.cc:654] T 969a8587019c4202b6e705d126f83d44 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.920481 1168 tablet_bootstrap.cc:492] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:46.920665 1168 ts_tablet_manager.cc:1403] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.035s user 0.002s sys 0.021s
I20251211 20:52:46.921411 1168 raft_consensus.cc:359] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.921612 1168 raft_consensus.cc:385] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.921684 1168 raft_consensus.cc:740] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.922010 1168 consensus_queue.cc:260] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.923349 1168 ts_tablet_manager.cc:1434] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:46.924388 1160 tablet_bootstrap.cc:654] T c91ff29847d54e55b19321652a857e9f P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.924593 1168 tablet_bootstrap.cc:492] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:46.926404 1168 tablet_bootstrap.cc:654] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.928483 1166 tablet_bootstrap.cc:492] T 969a8587019c4202b6e705d126f83d44 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:46.928661 1166 ts_tablet_manager.cc:1403] T 969a8587019c4202b6e705d126f83d44 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20251211 20:52:46.929410 1166 raft_consensus.cc:359] T 969a8587019c4202b6e705d126f83d44 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.929584 1166 raft_consensus.cc:385] T 969a8587019c4202b6e705d126f83d44 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.929657 1166 raft_consensus.cc:740] T 969a8587019c4202b6e705d126f83d44 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.930011 1166 consensus_queue.cc:260] T 969a8587019c4202b6e705d126f83d44 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.930743 1166 ts_tablet_manager.cc:1434] T 969a8587019c4202b6e705d126f83d44 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:46.931979 1166 tablet_bootstrap.cc:492] T be876471f8bf453aa4467c91891cbbde P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:46.933601 2764 raft_consensus.cc:493] T 84b2a67c836844ceb3628cc3b7d8ba10 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.933707 1166 tablet_bootstrap.cc:654] T be876471f8bf453aa4467c91891cbbde P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.933725 2764 raft_consensus.cc:515] T 84b2a67c836844ceb3628cc3b7d8ba10 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.934316 2764 leader_election.cc:290] T 84b2a67c836844ceb3628cc3b7d8ba10 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:46.934938 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "84b2a67c836844ceb3628cc3b7d8ba10" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.935923 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "84b2a67c836844ceb3628cc3b7d8ba10" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:46.936117 983 leader_election.cc:343] T 84b2a67c836844ceb3628cc3b7d8ba10 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:46.936463 982 leader_election.cc:343] T 84b2a67c836844ceb3628cc3b7d8ba10 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.936570 982 leader_election.cc:304] T 84b2a67c836844ceb3628cc3b7d8ba10 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.936959 2764 raft_consensus.cc:2749] T 84b2a67c836844ceb3628cc3b7d8ba10 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:46.938834 1166 tablet_bootstrap.cc:492] T be876471f8bf453aa4467c91891cbbde P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:46.938975 1166 ts_tablet_manager.cc:1403] T be876471f8bf453aa4467c91891cbbde P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.001s
I20251211 20:52:46.939723 1166 raft_consensus.cc:359] T be876471f8bf453aa4467c91891cbbde P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.939883 1166 raft_consensus.cc:385] T be876471f8bf453aa4467c91891cbbde P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.939932 1166 raft_consensus.cc:740] T be876471f8bf453aa4467c91891cbbde P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.940167 1166 consensus_queue.cc:260] T be876471f8bf453aa4467c91891cbbde P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.940745 1166 ts_tablet_manager.cc:1434] T be876471f8bf453aa4467c91891cbbde P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:46.941018 1166 tablet_bootstrap.cc:492] T 5df87ea7d28340da84ef141c92efc710 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:46.942654 1166 tablet_bootstrap.cc:654] T 5df87ea7d28340da84ef141c92efc710 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.943859 1168 tablet_bootstrap.cc:492] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:46.945264 1168 ts_tablet_manager.cc:1403] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.006s
I20251211 20:52:46.946171 2429 consensus_queue.cc:1048] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:46.946462 1168 raft_consensus.cc:359] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.946625 1168 raft_consensus.cc:385] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.946717 1168 raft_consensus.cc:740] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.950637 1168 consensus_queue.cc:260] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:46.951941 711 consensus_peers.cc:597] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:46.951992 1168 ts_tablet_manager.cc:1434] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20251211 20:52:46.952311 1168 tablet_bootstrap.cc:492] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:46.957773 847 consensus_peers.cc:597] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb4004212441473483a209d23c0800ce. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:46.962457 1166 tablet_bootstrap.cc:492] T 5df87ea7d28340da84ef141c92efc710 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:46.962805 1166 ts_tablet_manager.cc:1403] T 5df87ea7d28340da84ef141c92efc710 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.005s
I20251211 20:52:46.963685 1166 raft_consensus.cc:359] T 5df87ea7d28340da84ef141c92efc710 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.963955 1166 raft_consensus.cc:385] T 5df87ea7d28340da84ef141c92efc710 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.964066 1166 raft_consensus.cc:740] T 5df87ea7d28340da84ef141c92efc710 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.964365 1166 consensus_queue.cc:260] T 5df87ea7d28340da84ef141c92efc710 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.965066 1166 ts_tablet_manager.cc:1434] T 5df87ea7d28340da84ef141c92efc710 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:46.965407 1166 tablet_bootstrap.cc:492] T 962b80ae747b4470abd23643e711fad9 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:46.967152 1166 tablet_bootstrap.cc:654] T 962b80ae747b4470abd23643e711fad9 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.973924 2764 consensus_queue.cc:1048] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:46.982262 1168 tablet_bootstrap.cc:654] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.985631 2807 raft_consensus.cc:493] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:46.985808 2807 raft_consensus.cc:515] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.985922 1160 tablet_bootstrap.cc:492] T c91ff29847d54e55b19321652a857e9f P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:46.986068 1160 ts_tablet_manager.cc:1403] T c91ff29847d54e55b19321652a857e9f P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.068s user 0.001s sys 0.002s
I20251211 20:52:46.986444 2807 leader_election.cc:290] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:46.986876 1160 raft_consensus.cc:359] T c91ff29847d54e55b19321652a857e9f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.987035 1160 raft_consensus.cc:385] T c91ff29847d54e55b19321652a857e9f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:46.987082 1160 raft_consensus.cc:740] T c91ff29847d54e55b19321652a857e9f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:46.987088 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a7979798e92e45eb969197c8c7ac2580" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:46.987372 1160 consensus_queue.cc:260] T c91ff29847d54e55b19321652a857e9f P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:46.987998 711 leader_election.cc:343] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:46.989094 1160 ts_tablet_manager.cc:1434] T c91ff29847d54e55b19321652a857e9f P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:46.990128 1160 tablet_bootstrap.cc:492] T efe4d6bb07374294b8ab50a9fe605f87 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:46.991832 1160 tablet_bootstrap.cc:654] T efe4d6bb07374294b8ab50a9fe605f87 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:46.992223 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a7979798e92e45eb969197c8c7ac2580" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:46.992506 1047 raft_consensus.cc:2468] T a7979798e92e45eb969197c8c7ac2580 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:46.994314 710 leader_election.cc:304] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:46.994724 2807 raft_consensus.cc:2804] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:46.994818 2807 raft_consensus.cc:493] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:46.994892 2807 raft_consensus.cc:3060] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:46.997457 2807 raft_consensus.cc:515] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:46.999979 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a7979798e92e45eb969197c8c7ac2580" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:47.000627 711 leader_election.cc:343] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:47.002617 2807 leader_election.cc:290] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.003443 2807 raft_consensus.cc:493] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.003651 2807 raft_consensus.cc:515] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.004604 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6029c1fd563d4023a8cf7cfb8ffa0876" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:47.005141 711 leader_election.cc:343] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:47.005815 2807 leader_election.cc:290] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.009532 1037 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6029c1fd563d4023a8cf7cfb8ffa0876" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:47.009760 1037 raft_consensus.cc:2468] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:47.010088 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a7979798e92e45eb969197c8c7ac2580" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:47.010232 1048 raft_consensus.cc:3060] T a7979798e92e45eb969197c8c7ac2580 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:47.010385 710 leader_election.cc:304] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:47.010771 2807 raft_consensus.cc:2804] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:47.010835 2807 raft_consensus.cc:493] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:47.010886 2807 raft_consensus.cc:3060] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:47.012909 1048 raft_consensus.cc:2468] T a7979798e92e45eb969197c8c7ac2580 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:47.013355 710 leader_election.cc:304] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:47.013754 2429 raft_consensus.cc:2804] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:47.013880 2429 raft_consensus.cc:697] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:47.014118 2429 consensus_queue.cc:237] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.017079 908 raft_consensus.cc:3060] T 940dfd8781de444a9caa78dd363c7451 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:47.021916 1160 tablet_bootstrap.cc:492] T efe4d6bb07374294b8ab50a9fe605f87 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:47.022992 1160 ts_tablet_manager.cc:1403] T efe4d6bb07374294b8ab50a9fe605f87 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.034s user 0.001s sys 0.004s
I20251211 20:52:47.030432 2807 raft_consensus.cc:515] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.033123 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6029c1fd563d4023a8cf7cfb8ffa0876" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:47.033763 711 leader_election.cc:343] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:47.034030 2501 raft_consensus.cc:493] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.034168 2501 raft_consensus.cc:515] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.034719 2501 leader_election.cc:290] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.036322 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "373d3f03d7e0408cbe06955f8f60b16c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:47.036531 1045 raft_consensus.cc:2468] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:47.037302 710 leader_election.cc:304] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:47.037653 2808 raft_consensus.cc:2804] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:47.037726 2808 raft_consensus.cc:493] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:47.037777 2808 raft_consensus.cc:3060] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:47.037806 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "373d3f03d7e0408cbe06955f8f60b16c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:47.038007 904 raft_consensus.cc:3060] T c91ff29847d54e55b19321652a857e9f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:47.038594 711 leader_election.cc:343] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:47.039366 2764 raft_consensus.cc:493] T bd2bf1113dfe4b67b263b56f3d3db07d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.039526 2764 raft_consensus.cc:515] T bd2bf1113dfe4b67b263b56f3d3db07d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.040180 2764 leader_election.cc:290] T bd2bf1113dfe4b67b263b56f3d3db07d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:47.040355 2808 raft_consensus.cc:515] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.040755 2764 raft_consensus.cc:493] T 04679dfddf564d6f8c16d1c257e29b41 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.040835 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bd2bf1113dfe4b67b263b56f3d3db07d" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:47.040941 2808 leader_election.cc:290] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.040881 2764 raft_consensus.cc:515] T 04679dfddf564d6f8c16d1c257e29b41 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.041296 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "373d3f03d7e0408cbe06955f8f60b16c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:47.041354 2764 leader_election.cc:290] T 04679dfddf564d6f8c16d1c257e29b41 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
W20251211 20:52:47.041404 983 leader_election.cc:343] T bd2bf1113dfe4b67b263b56f3d3db07d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:47.041978 711 leader_election.cc:343] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:47.041985 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "04679dfddf564d6f8c16d1c257e29b41" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:47.042042 770 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bd2bf1113dfe4b67b263b56f3d3db07d" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:47.042042 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "04679dfddf564d6f8c16d1c257e29b41" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:47.042312 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "373d3f03d7e0408cbe06955f8f60b16c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:47.042470 1045 raft_consensus.cc:3060] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:47.042822 982 leader_election.cc:343] T bd2bf1113dfe4b67b263b56f3d3db07d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:47.042965 982 leader_election.cc:304] T bd2bf1113dfe4b67b263b56f3d3db07d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:47.043159 983 leader_election.cc:343] T 04679dfddf564d6f8c16d1c257e29b41 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:47.043268 2764 raft_consensus.cc:2749] T bd2bf1113dfe4b67b263b56f3d3db07d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:47.043736 982 leader_election.cc:343] T 04679dfddf564d6f8c16d1c257e29b41 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:47.043908 982 leader_election.cc:304] T 04679dfddf564d6f8c16d1c257e29b41 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:47.044144 2764 raft_consensus.cc:2749] T 04679dfddf564d6f8c16d1c257e29b41 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:47.042747 1160 raft_consensus.cc:359] T efe4d6bb07374294b8ab50a9fe605f87 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.046407 1160 raft_consensus.cc:385] T efe4d6bb07374294b8ab50a9fe605f87 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:47.047837 2807 leader_election.cc:290] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.047916 1037 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6029c1fd563d4023a8cf7cfb8ffa0876" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:47.048085 1037 raft_consensus.cc:3060] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:47.049590 1045 raft_consensus.cc:2468] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:47.050055 710 leader_election.cc:304] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:47.050709 1037 raft_consensus.cc:2468] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:47.050755 2807 raft_consensus.cc:2804] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:47.052218 710 leader_election.cc:304] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:47.052614 2808 raft_consensus.cc:2804] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:47.052934 1160 raft_consensus.cc:740] T efe4d6bb07374294b8ab50a9fe605f87 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:47.052997 2808 raft_consensus.cc:697] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:47.053260 1160 consensus_queue.cc:260] T efe4d6bb07374294b8ab50a9fe605f87 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.053319 2808 consensus_queue.cc:237] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:47.059414 711 consensus_peers.cc:597] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:47.061555 2807 raft_consensus.cc:697] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:47.061838 1160 ts_tablet_manager.cc:1434] T efe4d6bb07374294b8ab50a9fe605f87 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.037s user 0.001s sys 0.001s
I20251211 20:52:47.062031 2807 consensus_queue.cc:237] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.065517 1160 tablet_bootstrap.cc:492] T f22e4d34cde24b37a025c1b5a1046b1b P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:47.066746 1096 heartbeater.cc:499] Master 127.0.104.61:40749 was elected leader, sending a full tablet report...
W20251211 20:52:47.070540 711 consensus_peers.cc:597] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:47.077618 1160 tablet_bootstrap.cc:654] T f22e4d34cde24b37a025c1b5a1046b1b P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:47.077659 2501 raft_consensus.cc:493] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.077809 2501 raft_consensus.cc:515] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.078460 2501 leader_election.cc:290] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.079676 1037 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a295a24e81f14e5fa126a4a58ffd08cf" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:47.079906 1037 raft_consensus.cc:2468] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:47.082062 710 leader_election.cc:304] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:47.082480 2501 raft_consensus.cc:2804] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:47.082592 2501 raft_consensus.cc:493] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:47.082659 2501 raft_consensus.cc:3060] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:47.085340 2501 raft_consensus.cc:515] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.085351 1168 tablet_bootstrap.cc:492] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:47.085937 1168 ts_tablet_manager.cc:1403] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.134s user 0.018s sys 0.002s
I20251211 20:52:47.086071 2501 leader_election.cc:290] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.086794 1168 raft_consensus.cc:359] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.086978 1168 raft_consensus.cc:385] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:47.087055 1168 raft_consensus.cc:740] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:47.087244 1168 consensus_queue.cc:260] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.088141 1168 ts_tablet_manager.cc:1434] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:47.088447 1168 tablet_bootstrap.cc:492] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:47.090998 1037 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a295a24e81f14e5fa126a4a58ffd08cf" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:47.091207 1037 raft_consensus.cc:3060] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:47.093730 2759 consensus_queue.cc:1048] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:47.093813 1037 raft_consensus.cc:2468] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:47.094587 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a295a24e81f14e5fa126a4a58ffd08cf" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:47.095783 711 leader_election.cc:343] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:47.097669 881 tablet_service.cc:1505] Processing CreateTablet for tablet 6ed4a92d5ff74c80a4672522dec15f12 (DEFAULT_TABLE table=default.Table_c78768d721bd46a99b00e5b810e654db [id=fd5d745156974870995e5dd973e909b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.098124 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ed4a92d5ff74c80a4672522dec15f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.100589 886 tablet_service.cc:1505] Processing CreateTablet for tablet 8b0b13bdaed54fc0af48f857fc4cbe4f (DEFAULT_TABLE table=default.Table_4b288130d07345eab081b9d7ccea6c2b [id=49f9dc1b1ef3459fbb2ef7182a98ece9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.101013 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b0b13bdaed54fc0af48f857fc4cbe4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.103945 883 tablet_service.cc:1505] Processing CreateTablet for tablet 61bfdb8a3db94bfa9764c7df31cdc3e6 (DEFAULT_TABLE table=default.Table_acd9171c49ff48d2a7ed21684903193b [id=9347a65a369e4bd9be199f22f7cbec2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.104444 889 tablet_service.cc:1505] Processing CreateTablet for tablet 1aa0bf9afbcf4d8c903834bd3b9d9123 (DEFAULT_TABLE table=default.Table_9386ee8bce224f50bc410ab74a565d56 [id=b84914da81a1417cbea99ae2c54f4ed3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.105054 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61bfdb8a3db94bfa9764c7df31cdc3e6. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:47.106528 2550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:52:47.107010 2550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:52:47.107125 2550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20251211 20:52:47.112383 880 tablet_service.cc:1505] Processing CreateTablet for tablet ba796f7d6669432d946c5d33a52b54c4 (DEFAULT_TABLE table=default.Table_0514969896e648faa4dc0327ecaf8993 [id=6b95f555799e4ffdac037f83c5384838]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.112836 888 tablet_service.cc:1505] Processing CreateTablet for tablet d88f9d0ce18f44669260f0f958cb039d (DEFAULT_TABLE table=default.Table_0be97b736b654803810e4c33d0b01df7 [id=6263dade24d348518ac7bfce5ddc5a79]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.113147 891 tablet_service.cc:1505] Processing CreateTablet for tablet 31921711297041ec9ef32c8ace8020aa (DEFAULT_TABLE table=default.Table_f611fe8c909d4af2af7c49dc28f4d2a9 [id=71b4acb11cdc4ce9911a6708bf072209]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.113484 894 tablet_service.cc:1505] Processing CreateTablet for tablet 4a45e8e2dfd3443e8ee7d08aae37470b (DEFAULT_TABLE table=default.Table_a205c236aef84e30921a240403104033 [id=5b7755830b2f4cc58ee621b150b81c3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.113721 887 tablet_service.cc:1505] Processing CreateTablet for tablet 07f0681426c049719b5760176f41106a (DEFAULT_TABLE table=default.Table_ff29e8c9caa441708463eebe086f5ba9 [id=56ca7846360149c3a8f8f932317349bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.118947 890 tablet_service.cc:1505] Processing CreateTablet for tablet 3c734e9d003643afb444b2b15f93fa93 (DEFAULT_TABLE table=default.Table_08dc6bc7d022495db033bda27f21c84c [id=77cfe7b6d0094c6497429960b929141c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.121485 884 tablet_service.cc:1505] Processing CreateTablet for tablet 37c12e3157c54adb8ed5621a9a27d613 (DEFAULT_TABLE table=default.Table_34c853fcb87d480fb23827277ebed594 [id=15901f3423534245ba63bf94be889977]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.121796 1168 tablet_bootstrap.cc:654] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:47.122824 756 tablet_service.cc:1505] Processing CreateTablet for tablet 8b0b13bdaed54fc0af48f857fc4cbe4f (DEFAULT_TABLE table=default.Table_4b288130d07345eab081b9d7ccea6c2b [id=49f9dc1b1ef3459fbb2ef7182a98ece9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.123270 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b0b13bdaed54fc0af48f857fc4cbe4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.126157 885 tablet_service.cc:1505] Processing CreateTablet for tablet dd1ae033dedc436fa3a3f864a7a208a2 (DEFAULT_TABLE table=default.Table_233d5484bd4e4456aed925bc271edfa5 [id=ca0c3c6978874d93a04b57cc0c425c83]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.122824 754 tablet_service.cc:1505] Processing CreateTablet for tablet 6ed4a92d5ff74c80a4672522dec15f12 (DEFAULT_TABLE table=default.Table_c78768d721bd46a99b00e5b810e654db [id=fd5d745156974870995e5dd973e909b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.128204 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ed4a92d5ff74c80a4672522dec15f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.128243 879 tablet_service.cc:1505] Processing CreateTablet for tablet 23f6e5415e14478281733c5e59c65d44 (DEFAULT_TABLE table=default.Table_50aa8aa5823a4e71b0f191ff52eefb75 [id=95fb6825923a4a679a7e5d6695863a02]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.134037 1160 tablet_bootstrap.cc:492] T f22e4d34cde24b37a025c1b5a1046b1b P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:47.134232 1160 ts_tablet_manager.cc:1403] T f22e4d34cde24b37a025c1b5a1046b1b P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.071s user 0.003s sys 0.005s
I20251211 20:52:47.135016 1160 raft_consensus.cc:359] T f22e4d34cde24b37a025c1b5a1046b1b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.135243 1160 raft_consensus.cc:385] T f22e4d34cde24b37a025c1b5a1046b1b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:47.135318 1160 raft_consensus.cc:740] T f22e4d34cde24b37a025c1b5a1046b1b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:47.135694 1160 consensus_queue.cc:260] T f22e4d34cde24b37a025c1b5a1046b1b P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.136118 877 tablet_service.cc:1505] Processing CreateTablet for tablet ebe6ca12e5e14337b317cad3621029ca (DEFAULT_TABLE table=default.Table_7b6d7c6f71514a118834aa645c8405e9 [id=8564c2055c9d420cb3228a39f66c66fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.136721 1160 ts_tablet_manager.cc:1434] T f22e4d34cde24b37a025c1b5a1046b1b P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20251211 20:52:47.137097 2550 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251211 20:52:47.137192 2550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:52:47.137271 2550 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251211 20:52:47.137333 2550 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251211 20:52:47.137667 2948 raft_consensus.cc:493] T 79d92736bb1c46d2bfa5678b2d396ec2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.137818 2948 raft_consensus.cc:515] T 79d92736bb1c46d2bfa5678b2d396ec2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.138731 2948 leader_election.cc:290] T 79d92736bb1c46d2bfa5678b2d396ec2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:47.140398 1168 tablet_bootstrap.cc:492] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:47.140648 1168 ts_tablet_manager.cc:1403] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.052s user 0.003s sys 0.003s
I20251211 20:52:47.140640 747 tablet_service.cc:1505] Processing CreateTablet for tablet 61bfdb8a3db94bfa9764c7df31cdc3e6 (DEFAULT_TABLE table=default.Table_acd9171c49ff48d2a7ed21684903193b [id=9347a65a369e4bd9be199f22f7cbec2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.141083 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61bfdb8a3db94bfa9764c7df31cdc3e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.141618 1168 raft_consensus.cc:359] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.141855 1168 raft_consensus.cc:385] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:47.141952 1168 raft_consensus.cc:740] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:47.142182 1168 consensus_queue.cc:260] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.142513 875 tablet_service.cc:1505] Processing CreateTablet for tablet 93f6f6c240aa403ea783a0c7ff0cfb8a (DEFAULT_TABLE table=default.Table_d8214c15312c49ce85f2c52b14cf4a78 [id=bdca175d3ca14dfc912d92c7f4beae89]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.143218 1168 ts_tablet_manager.cc:1434] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:47.144708 878 tablet_service.cc:1505] Processing CreateTablet for tablet 46e5c35780ad498ab4c274d311cd606f (DEFAULT_TABLE table=default.Table_d424820106284c94adee000401d8ab22 [id=92aecd5e8f9b453082fd628c9b24e3cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.145457 1168 tablet_bootstrap.cc:492] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:47.146845 893 tablet_service.cc:1505] Processing CreateTablet for tablet 087bc334ae9c4018b6b86afef4354911 (DEFAULT_TABLE table=default.Table_280921729cb94c62b206fc68ef348c61 [id=cff90a42974d452786230faab90e95ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.147652 739 tablet_service.cc:1505] Processing CreateTablet for tablet ba796f7d6669432d946c5d33a52b54c4 (DEFAULT_TABLE table=default.Table_0514969896e648faa4dc0327ecaf8993 [id=6b95f555799e4ffdac037f83c5384838]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.148054 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba796f7d6669432d946c5d33a52b54c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.149456 757 tablet_service.cc:1505] Processing CreateTablet for tablet 31921711297041ec9ef32c8ace8020aa (DEFAULT_TABLE table=default.Table_f611fe8c909d4af2af7c49dc28f4d2a9 [id=71b4acb11cdc4ce9911a6708bf072209]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.149576 882 tablet_service.cc:1505] Processing CreateTablet for tablet 1647056e32af435a86c48c1d8c8bf412 (DEFAULT_TABLE table=default.Table_cd866c64dfc7477782129c897b849771 [id=ef146ea6ee794351944112c4b0dcef58]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.149837 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31921711297041ec9ef32c8ace8020aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.151964 892 tablet_service.cc:1505] Processing CreateTablet for tablet 170ef6a9993c4b00ac3ddbec411bb5a6 (DEFAULT_TABLE table=default.Table_7f9e421192224c908c84e08e5b787787 [id=f8a9880896634b27b099dc95674eef45]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.152386 876 tablet_service.cc:1505] Processing CreateTablet for tablet 413ad8f69a4a4f5898ef6ddbb395c821 (DEFAULT_TABLE table=default.Table_89bf0fe9a0c54cae85b1b8305d2b91e8 [id=9671f96ca9fe424db31d663aeee7f725]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.147652 741 tablet_service.cc:1505] Processing CreateTablet for tablet d88f9d0ce18f44669260f0f958cb039d (DEFAULT_TABLE table=default.Table_0be97b736b654803810e4c33d0b01df7 [id=6263dade24d348518ac7bfce5ddc5a79]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.153092 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d88f9d0ce18f44669260f0f958cb039d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.156917 746 tablet_service.cc:1505] Processing CreateTablet for tablet 4a45e8e2dfd3443e8ee7d08aae37470b (DEFAULT_TABLE table=default.Table_a205c236aef84e30921a240403104033 [id=5b7755830b2f4cc58ee621b150b81c3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.157330 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a45e8e2dfd3443e8ee7d08aae37470b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.161223 1168 tablet_bootstrap.cc:654] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:47.164767 741 tablet_service.cc:1505] Processing CreateTablet for tablet 37c12e3157c54adb8ed5621a9a27d613 (DEFAULT_TABLE table=default.Table_34c853fcb87d480fb23827277ebed594 [id=15901f3423534245ba63bf94be889977]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.165302 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37c12e3157c54adb8ed5621a9a27d613. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.169544 2948 raft_consensus.cc:493] T 1e13965b6361488eaeb7dd13b467ca65 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.169718 2948 raft_consensus.cc:515] T 1e13965b6361488eaeb7dd13b467ca65 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.170320 2948 leader_election.cc:290] T 1e13965b6361488eaeb7dd13b467ca65 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:47.164767 739 tablet_service.cc:1505] Processing CreateTablet for tablet dd1ae033dedc436fa3a3f864a7a208a2 (DEFAULT_TABLE table=default.Table_233d5484bd4e4456aed925bc271edfa5 [id=ca0c3c6978874d93a04b57cc0c425c83]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.171222 1015 tablet_service.cc:1505] Processing CreateTablet for tablet 6ed4a92d5ff74c80a4672522dec15f12 (DEFAULT_TABLE table=default.Table_c78768d721bd46a99b00e5b810e654db [id=fd5d745156974870995e5dd973e909b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.171415 710 leader_election.cc:304] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:47.171381 1021 tablet_service.cc:1505] Processing CreateTablet for tablet 4a45e8e2dfd3443e8ee7d08aae37470b (DEFAULT_TABLE table=default.Table_a205c236aef84e30921a240403104033 [id=5b7755830b2f4cc58ee621b150b81c3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.171630 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ed4a92d5ff74c80a4672522dec15f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.171774 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd1ae033dedc436fa3a3f864a7a208a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.171859 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a45e8e2dfd3443e8ee7d08aae37470b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.171952 751 tablet_service.cc:1505] Processing CreateTablet for tablet 23f6e5415e14478281733c5e59c65d44 (DEFAULT_TABLE table=default.Table_50aa8aa5823a4e71b0f191ff52eefb75 [id=95fb6825923a4a679a7e5d6695863a02]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.172187 1013 tablet_service.cc:1505] Processing CreateTablet for tablet 8b0b13bdaed54fc0af48f857fc4cbe4f (DEFAULT_TABLE table=default.Table_4b288130d07345eab081b9d7ccea6c2b [id=49f9dc1b1ef3459fbb2ef7182a98ece9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.172276 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23f6e5415e14478281733c5e59c65d44. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.172519 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b0b13bdaed54fc0af48f857fc4cbe4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.175923 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 61bfdb8a3db94bfa9764c7df31cdc3e6 (DEFAULT_TABLE table=default.Table_acd9171c49ff48d2a7ed21684903193b [id=9347a65a369e4bd9be199f22f7cbec2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.176411 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61bfdb8a3db94bfa9764c7df31cdc3e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.179854 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 1aa0bf9afbcf4d8c903834bd3b9d9123 (DEFAULT_TABLE table=default.Table_9386ee8bce224f50bc410ab74a565d56 [id=b84914da81a1417cbea99ae2c54f4ed3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.179842 741 tablet_service.cc:1505] Processing CreateTablet for tablet ebe6ca12e5e14337b317cad3621029ca (DEFAULT_TABLE table=default.Table_7b6d7c6f71514a118834aa645c8405e9 [id=8564c2055c9d420cb3228a39f66c66fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.180266 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1aa0bf9afbcf4d8c903834bd3b9d9123. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.180456 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebe6ca12e5e14337b317cad3621029ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.182363 2759 raft_consensus.cc:493] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.182554 2759 raft_consensus.cc:515] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.183183 2759 leader_election.cc:290] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.183928 1029 tablet_service.cc:1505] Processing CreateTablet for tablet ba796f7d6669432d946c5d33a52b54c4 (DEFAULT_TABLE table=default.Table_0514969896e648faa4dc0327ecaf8993 [id=6b95f555799e4ffdac037f83c5384838]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:47.184202 847 leader_election.cc:343] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: c74427cb9d6f4a679e43eae796ff8c3f
I20251211 20:52:47.184358 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba796f7d6669432d946c5d33a52b54c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.184695 751 tablet_service.cc:1505] Processing CreateTablet for tablet 93f6f6c240aa403ea783a0c7ff0cfb8a (DEFAULT_TABLE table=default.Table_d8214c15312c49ce85f2c52b14cf4a78 [id=bdca175d3ca14dfc912d92c7f4beae89]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.185025 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93f6f6c240aa403ea783a0c7ff0cfb8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.186482 757 tablet_service.cc:1505] Processing CreateTablet for tablet 07f0681426c049719b5760176f41106a (DEFAULT_TABLE table=default.Table_ff29e8c9caa441708463eebe086f5ba9 [id=56ca7846360149c3a8f8f932317349bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.186823 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07f0681426c049719b5760176f41106a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.187943 1022 tablet_service.cc:1505] Processing CreateTablet for tablet d88f9d0ce18f44669260f0f958cb039d (DEFAULT_TABLE table=default.Table_0be97b736b654803810e4c33d0b01df7 [id=6263dade24d348518ac7bfce5ddc5a79]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.188333 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d88f9d0ce18f44669260f0f958cb039d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.191753 2736 raft_consensus.cc:2804] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:47.191893 2736 raft_consensus.cc:697] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:47.191879 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 31921711297041ec9ef32c8ace8020aa (DEFAULT_TABLE table=default.Table_f611fe8c909d4af2af7c49dc28f4d2a9 [id=71b4acb11cdc4ce9911a6708bf072209]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.192117 2736 consensus_queue.cc:237] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.192255 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31921711297041ec9ef32c8ace8020aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.195631 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 07f0681426c049719b5760176f41106a (DEFAULT_TABLE table=default.Table_ff29e8c9caa441708463eebe086f5ba9 [id=56ca7846360149c3a8f8f932317349bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.196054 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07f0681426c049719b5760176f41106a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.200554 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79d92736bb1c46d2bfa5678b2d396ec2" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:47.201280 2948 raft_consensus.cc:493] T de508d79f30f489cb745401f53115165 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.201376 905 raft_consensus.cc:3060] T 2e33606930774fe4aafc6e3819618af8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:47.201453 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1e13965b6361488eaeb7dd13b467ca65" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:47.201434 2948 raft_consensus.cc:515] T de508d79f30f489cb745401f53115165 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.202051 2948 leader_election.cc:290] T de508d79f30f489cb745401f53115165 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:47.211158 2494 raft_consensus.cc:493] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.211328 2494 raft_consensus.cc:515] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.211995 2494 leader_election.cc:290] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.213745 742 tablet_service.cc:1505] Processing CreateTablet for tablet 170ef6a9993c4b00ac3ddbec411bb5a6 (DEFAULT_TABLE table=default.Table_7f9e421192224c908c84e08e5b787787 [id=f8a9880896634b27b099dc95674eef45]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.214154 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 170ef6a9993c4b00ac3ddbec411bb5a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.219825 2948 raft_consensus.cc:493] T f2ae084982f44fdf8bfecdd2bc4c01b9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.219995 2948 raft_consensus.cc:515] T f2ae084982f44fdf8bfecdd2bc4c01b9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.220644 2948 leader_election.cc:290] T f2ae084982f44fdf8bfecdd2bc4c01b9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:47.220639 745 tablet_service.cc:1505] Processing CreateTablet for tablet 274f00ed89484f158a57cdc0a16eb96c (DEFAULT_TABLE table=default.Table_171c2a61012b444caf720781072205a3 [id=64212968bb94465093adb65222437d60]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.220661 738 tablet_service.cc:1505] Processing CreateTablet for tablet 3ca3d6cf418e4e718f6bb24c8582d512 (DEFAULT_TABLE table=default.Table_d8d6a529a8934b64a5a430cb54ac0954 [id=a2b9a68c9d9147b79b16fd7bcd83f7f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.221087 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 274f00ed89484f158a57cdc0a16eb96c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.221292 738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ca3d6cf418e4e718f6bb24c8582d512. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.221418 739 tablet_service.cc:1505] Processing CreateTablet for tablet 413ad8f69a4a4f5898ef6ddbb395c821 (DEFAULT_TABLE table=default.Table_89bf0fe9a0c54cae85b1b8305d2b91e8 [id=9671f96ca9fe424db31d663aeee7f725]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.221778 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 413ad8f69a4a4f5898ef6ddbb395c821. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.223912 740 tablet_service.cc:1505] Processing CreateTablet for tablet 5a16915e31b64a20ad6409c38410c978 (DEFAULT_TABLE table=default.Table_4585c080cc0f44a3af3d60f96f928eb2 [id=ce43ec33f1414838992e472b6bad7dce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.224272 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a16915e31b64a20ad6409c38410c978. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.227762 741 tablet_service.cc:1505] Processing CreateTablet for tablet 1647056e32af435a86c48c1d8c8bf412 (DEFAULT_TABLE table=default.Table_cd866c64dfc7477782129c897b849771 [id=ef146ea6ee794351944112c4b0dcef58]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.228242 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1647056e32af435a86c48c1d8c8bf412. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.228329 746 tablet_service.cc:1505] Processing CreateTablet for tablet 3c734e9d003643afb444b2b15f93fa93 (DEFAULT_TABLE table=default.Table_08dc6bc7d022495db033bda27f21c84c [id=77cfe7b6d0094c6497429960b929141c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.228664 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c734e9d003643afb444b2b15f93fa93. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.231112 756 tablet_service.cc:1505] Processing CreateTablet for tablet 087bc334ae9c4018b6b86afef4354911 (DEFAULT_TABLE table=default.Table_280921729cb94c62b206fc68ef348c61 [id=cff90a42974d452786230faab90e95ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.231460 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 087bc334ae9c4018b6b86afef4354911. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.236714 747 tablet_service.cc:1505] Processing CreateTablet for tablet 1aa0bf9afbcf4d8c903834bd3b9d9123 (DEFAULT_TABLE table=default.Table_9386ee8bce224f50bc410ab74a565d56 [id=b84914da81a1417cbea99ae2c54f4ed3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.237116 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1aa0bf9afbcf4d8c903834bd3b9d9123. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.239231 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b52377c8ffd741a598e5328a00d876fe (table default.Table_686a26915b42421380037144900dd97c [id=8a1b0c9d641b4a3c83d1ca48576d110d]). Expected version 1 got 0
I20251211 20:52:47.239957 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet c7a2610f20df4c1ca81eae2fc7dba6bb (table default.Table_59072235d93b41c0b5cd5e3debcb8ddf [id=9a15a8a075af4969ae2c8b3be1bfed78]). Expected version 1 got 0
I20251211 20:52:47.240533 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 0302cc8613934c43a0650f081cf98797 (table default.Table_bf827b216f1b43c196923dc1828ff0d0 [id=ed371e073250439fa44330c7101e9387]). Expected version 1 got 0
I20251211 20:52:47.241019 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 1f4eb1a174c648c897ae2b7612e44585 (table default.Table_e70873b71ba24d7eae548e2acfd243b3 [id=87adcc552a704d56bd2fe812fb3b791f]). Expected version 1 got 0
I20251211 20:52:47.241302 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet a7afcce522494c9ca1d80e6eea913e6e (table default.Table_cb6a4bac7b694f33a37de2ee78c91ca0 [id=8940f18527c249a9baefc170ed82f45a]). Expected version 1 got 0
I20251211 20:52:47.241586 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 84bb3583857848459e0e865283de623d (table default.Table_92281c5fcfee4355a79215ffedeee588 [id=6e5fe0a78e4b48f98dac028802f1ddf4]). Expected version 4 got 0
I20251211 20:52:47.241899 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 3e6f62c2deff4c11b137fc0cba5ee234 (table default.Table_51db250005ed4b3d947f32acd1d62563 [id=fdfaa0b24d2f4736aee695ed228212cf]). Expected version 1 got 0
I20251211 20:52:47.242254 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b115d737d7fc483d9f5a1f040b511bae (table default.Table_6d94053661f142b79292e19752df02df [id=c5be59a2c3934b3789641ceeb2d4d6e2]). Expected version 2 got 0
I20251211 20:52:47.242489 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet da81bf8854a24954ba4548185812295d (table default.Table_18262e22c26d4d66ba4c6134fbde0368 [id=da8cf2b99b42409091484a9513623d6f]). Expected version 1 got 0
I20251211 20:52:47.242914 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 6b7dd8232d974a43b99ec9b28075dc8d (table default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]). Expected version 2 got 0
I20251211 20:52:47.243345 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 8675e2a18f0f4a0e95a87005cf82d8b4 (table default.Table_bd9633c55bd242c18aac3a7f2c18a329 [id=767f50bee5464316a18f73e64bab817a]). Expected version 2 got 0
I20251211 20:52:47.244021 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 676faec7f3d1488f99b220f8e968e9a0 (table default.Table_ab826a3eec8e4c4aaa8d4021d87dd382 [id=a2bbfccfe3974c378521d5ef2333df86]). Expected version 2 got 0
I20251211 20:52:47.244361 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet f2d6c9781b4d4c37bc08703347963d5d (table default.Table_2aa7ff9186a04249abf8cd971cc0cb17 [id=b50ab5b032ba4c98938324460512468f]). Expected version 1 got 0
I20251211 20:52:47.244779 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet a3b1aadf17ea4bf0a896d23cd543fca7 (table default.Table_d18401707c88493abdf90d2c713432ae [id=a07662f8dc3f4178be5dfcc1cdc85837]). Expected version 1 got 0
I20251211 20:52:47.245287 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 9f8aa28c1c9b4fc58dfc128952c180ba (table default.Table_bf7d402a36be44cfb6b79041906d26a1 [id=0327e686e7ed4475a3e8cda8721a3a3a]). Expected version 1 got 0
I20251211 20:52:47.245993 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 33fd09ea06ee47908b85911e4ee79708 (table default.Table_3c3cd48c8fe5411e96cdf811d492925d [id=30620a01c27f456bac05fa7c02a2cb4d]). Expected version 2 got 0
I20251211 20:52:47.246243 754 tablet_service.cc:1505] Processing CreateTablet for tablet 46e5c35780ad498ab4c274d311cd606f (DEFAULT_TABLE table=default.Table_d424820106284c94adee000401d8ab22 [id=92aecd5e8f9b453082fd628c9b24e3cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.246347 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 6212b3b4425543168837946a54d60834 (table default.Table_8c30c4151d484db0bb9e2b9ca262248d [id=2d489cd7d73e403d84bf10960e6a2dfb]). Expected version 1 got 0
I20251211 20:52:47.246654 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46e5c35780ad498ab4c274d311cd606f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.247406 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b2974f16034c44cf85ff85b5095cf6a0 (table default.Table_031de0f4cc7744bc9e320261f8b46696 [id=be36dabaa6d44325b4855b9f13bda847]). Expected version 1 got 0
I20251211 20:52:47.247975 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet ab4e7a9062134e489d6f9abe6ff26fb4 (table default.Table_4cbc8841b276406cb4434629653a5ce3 [id=b958532650374d609199cab9c07bb64b]). Expected version 2 got 0
I20251211 20:52:47.248266 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet f00128ba84c245d9a0e35f26bc842b3e (table default.Table_6f8c133c7a364b93af2c83d8911a0ac3 [id=6a21c38ee7224a02b937df088494f5d8]). Expected version 1 got 0
I20251211 20:52:47.248430 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "44e2d2bdecf247baaac26e06bccc94de" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:47.248829 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet d6923cfd826d495291d9ac8e0b3b44c6 (table default.Table_68ccb2052d824ccf92f7162d2961fcb1 [id=9ac9591d4f04482d926710fdf6e797c6]). Expected version 1 got 0
I20251211 20:52:47.249122 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 94658fcf512a48d0a992a7e2e0f7c176 (table default.Table_c9d86e82aebb4ef0bc83789720352e38 [id=97377a8ef56c4cdebe95d15955207404]). Expected version 1 got 0
I20251211 20:52:47.249384 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 18f125dd9efc401181f0644dad7aeffe (table default.Table_fa6592c2e2624d81ad21abb13ed36045 [id=9b48fc2b39db4fd5a4e5da19e34427dc]). Expected version 1 got 0
I20251211 20:52:47.250042 740 tablet_service.cc:1505] Processing CreateTablet for tablet 1e41560d2d1147c49e3f3454fb2b069b (DEFAULT_TABLE table=default.Table_7759f916c14543c38f2a391980a1b38b [id=d23c560e49134034a520447a45c5a945]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.250245 739 tablet_service.cc:1505] Processing CreateTablet for tablet 00c4326b662541b8b33242b2a82235a5 (DEFAULT_TABLE table=default.Table_e133e518b951424cba3117dfdf77d480 [id=3639f4db6085489f975da70c3340d5b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.250465 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e41560d2d1147c49e3f3454fb2b069b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.250557 742 tablet_service.cc:1505] Processing CreateTablet for tablet 68fe7dfddec04ff79199661dc41a6330 (DEFAULT_TABLE table=default.Table_282dd63df9ff4557a5ab2761d63df7b1 [id=c59fde89abe64038889ab84b50d93ad2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.250042 741 tablet_service.cc:1505] Processing CreateTablet for tablet a49fbec29ff241d299dce318fce0cead (DEFAULT_TABLE table=default.Table_ff2dfead5c37456d91374a4bc62c85ea [id=14be5949ea90486d97522326490ce78b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.250902 739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00c4326b662541b8b33242b2a82235a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.251060 738 tablet_service.cc:1505] Processing CreateTablet for tablet db0504131ddd4e3c88b2b44bd2bcb920 (DEFAULT_TABLE table=default.Table_28bfcb62d3324b46af3d8e28ab649bf4 [id=9d63f6afeb224e12aaed5b9eccbe234b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.251277 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68fe7dfddec04ff79199661dc41a6330. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.251241 745 tablet_service.cc:1505] Processing CreateTablet for tablet f4f5011512dd4b31853738599384c293 (DEFAULT_TABLE table=default.Table_870517d846e84832912515fdcf0d8b64 [id=3ce9d2e49dbc4e5787f59a621f20b657]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.254562 2759 raft_consensus.cc:493] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.254738 2759 raft_consensus.cc:515] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.255363 2759 leader_election.cc:290] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.256804 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1aa0bf9afbcf4d8c903834bd3b9d9123. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:47.257958 847 leader_election.cc:343] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: a2058b67ddf1413db338bfa2f64e7a77
I20251211 20:52:47.258301 738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db0504131ddd4e3c88b2b44bd2bcb920. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.260170 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba796f7d6669432d946c5d33a52b54c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.263922 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a49fbec29ff241d299dce318fce0cead. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.263942 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d88f9d0ce18f44669260f0f958cb039d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.267825 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31921711297041ec9ef32c8ace8020aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.269975 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4f5011512dd4b31853738599384c293. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.271936 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a45e8e2dfd3443e8ee7d08aae37470b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.249749 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet aa50ddb2789b4be3bfa9aedef9287109 (table default.Table_44f971096d1e41f480e594985ffd5c17 [id=d4e1349374a346a5b8be180f9bdc329a]). Expected version 1 got 0
I20251211 20:52:47.275839 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07f0681426c049719b5760176f41106a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.276019 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet f94b656e3c1844509fdae36e33b263e8 (table default.Table_ed03ad475bed4804a65abe20efef0625 [id=e23f4305669a474aafa18928f130de3a]). Expected version 2 got 0
I20251211 20:52:47.276258 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 4c7b74f370b042ee8575f821c07dc085 (table default.Table_731826f26f9940b8afe425d694b642d9 [id=17dd9cea9ac844128ff9f82adb3f919f]). Expected version 1 got 0
I20251211 20:52:47.276422 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b85af79c2574443998e847bc19c53b69 (table default.Table_fa3f2f115cd043a9aa6be2b070d320cb [id=15fad5e1cf8d4ea08dae8275a29a11dd]). Expected version 1 got 0
I20251211 20:52:47.276696 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 16d2ff8925e342729a71c93db378d031 (table default.Table_34e6ed4f13da437ea50d349b92e83bd6 [id=4a74b9a60cbb4198b55cee630b2f4bef]). Expected version 2 got 0
I20251211 20:52:47.277097 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 8b793241ee3b42b1a38e9f2f05e1173b (table default.Table_7506ffecc2b64648a9ba736e5298ddf4 [id=279f39d718ba45318f52f95e8aa13652]). Expected version 2 got 0
I20251211 20:52:47.277565 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 7ffa06c4a7ee48b0b337a27aff090592 (table default.Table_647731023b044ebb8954f7fdcc3faf97 [id=4b91abd6e9e94a8a941b33e62e23c083]). Expected version 1 got 0
I20251211 20:52:47.277840 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 969a8587019c4202b6e705d126f83d44 (table default.Table_6460313002684bba93b30ad8631f0f49 [id=22347b15537d451d9927a17e8e5a202a]). Expected version 1 got 0
I20251211 20:52:47.278095 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 8dc4e730951144d78b77280f6cce6d0e (table default.Table_890c9027415a44ccaf95c0adede51992 [id=7bc8a2b91fe54b91b6594b371fea7bd3]). Expected version 2 got 0
I20251211 20:52:47.278846 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet fb29f983c4524a49bdc5c0c1144ad708 (table default.Table_7809ac85fdfe4c1d9c58afb54edecb27 [id=c4b0bfff99f64888beb70d9154d2f156]). Expected version 1 got 0
I20251211 20:52:47.280248 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 3dfea28c44d84c68850db65a30d52ee6 (table default.Table_c504f8f3796f43b697488bd604268543 [id=2af76b9d32094a4ca656060a4478e08a]). Expected version 1 got 0
I20251211 20:52:47.281471 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 59acb8474e114bf7ae7f1f043d611459 (table default.Table_c7ed8c0c309c40a3a371ffa130b47cd2 [id=ed1d0670426945aba888b818ad657225]). Expected version 1 got 0
I20251211 20:52:47.282060 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet aab26df748724fe5b2bb7fad9e3bfc0c (table default.Table_22bed4b4e9e24a90818b19c6f5c91e1f [id=df6255d66f334ef2b4406f4b845cf980]). Expected version 1 got 0
I20251211 20:52:47.283883 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet e3f711f02af843adb4f5c3c750924e33 (table default.Table_34bd5155aa3b4ef7818022ba9346c074 [id=83a61d23e80946439a14096ab3ab1bd9]). Expected version 2 got 0
I20251211 20:52:47.284390 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 7ca98b43863a47c091baae0a6ce65630 (table default.Table_f0a70914acc94231a4e84155c3e4d13f [id=56c070ad80624c698ebe2e01b4929b6e]). Expected version 1 got 0
I20251211 20:52:47.284857 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b6cc819332e1445c96ce3ffac4861454 (table default.Table_db4c0f706e0643b39d39ffa84352826d [id=4d459c42bf584068ac3851227d80331a]). Expected version 1 got 0
I20251211 20:52:47.285071 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet d37a7d0d662e45079b0cfc02765d2e01 (table default.Table_b26cc06e1b0745e08181124e1cbce934 [id=8a12299efe714617a970fc05c3bb6232]). Expected version 1 got 0
I20251211 20:52:47.285876 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 5773c13034714a008c8d7dbe777b4828 (table default.Table_66eb1b021eed464f82f9cd223c88297b [id=94ff09564cd649c7a2d0e94b0f3c7b3a]). Expected version 1 got 0
I20251211 20:52:47.286136 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 78f77d8916d84edabbe95ce17d8df2c1 (table default.Table_e2af0e2fbd3d46d595136c7a7bf24e4c [id=ddc12ae6d8ed4587b5ef4da3ef81a055]). Expected version 1 got 0
I20251211 20:52:47.286548 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 5e47af616e1b4d2db3780c1d67cdcb47 (table default.Table_296bdf30a11541c389afe3df0b0f667a [id=052edcbde60c41449d468c8a449fc3f1]). Expected version 1 got 0
I20251211 20:52:47.286856 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet e89d4d76bf64453892a4d0b55fcc3454 (table default.Table_fba99384e9f14160b3e3504f8d1d0b07 [id=01bba75f854d4c338d6c09a446dd7499]). Expected version 1 got 0
I20251211 20:52:47.287235 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 645b08b33676420c96e880de3d460829 (table default.Table_e02933dd91ce4fe7a40d1c3b2f9b2d4b [id=6d42f260246e420d8818b3841926302b]). Expected version 2 got 0
I20251211 20:52:47.288285 887 tablet_service.cc:1505] Processing CreateTablet for tablet 274f00ed89484f158a57cdc0a16eb96c (DEFAULT_TABLE table=default.Table_171c2a61012b444caf720781072205a3 [id=64212968bb94465093adb65222437d60]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.288892 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 274f00ed89484f158a57cdc0a16eb96c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.289855 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet e4647105ab57454d883ebfcde7bde891 (table default.Table_c817dec9a7674f699b3366fc302ead57 [id=d555d1fc6aef4a21b4d3f2804264d030]). Expected version 1 got 0
I20251211 20:52:47.290671 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet ee7b93b0b87541d9af6eb9e68b4cea41 (table default.Table_b6e6aea68c88426fb01353146c06b6ac [id=a4f984b4263a4bc38f6248628f643ae3]). Expected version 3 got 0
I20251211 20:52:47.294075 2550 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.20:41333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.62:41123,127.0.104.61:40749,127.0.104.60:42087
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.60:42087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.104.60
--webserver_port=40729
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:52:47.294782 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet df457783b2534b3ea23bb1fd529ce268 (table default.Table_6e9f3ca164384cc8ac4501deb3f0c6aa [id=342c2469d6e74e3e9217236e3aa863d6]). Expected version 1 got 0
I20251211 20:52:47.295349 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 5453f68789fd4e89b69e98860050865b (table default.Table_b7bf8c5ad5304175a1f2f6a03887f725 [id=50d88b663a4d4891b113aee9c5906699]). Expected version 1 got 0
I20251211 20:52:47.299702 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 46cb1d16a0e94b3a833f32c635213fd2 (table default.Table_3482fbfc5b144a7198a7d01aa003cb6b [id=1bdaf5ce0f6b4b99a038f9f9efb828f6]). Expected version 1 got 0
I20251211 20:52:47.300365 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 834b84a557b04e78b5d2c8c84520a644 (table default.Table_909bbe6da1a648d68199fb27424e4342 [id=cc99459a5039424dbc192ff748b6708b]). Expected version 2 got 0
I20251211 20:52:47.301333 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 7be44df39a8f4d6da793479d425fa684 (table default.Table_f6a10556ff104db5bd5ecc9fc380e228 [id=f75813add54e46189ff1bbed82d83ba7]). Expected version 1 got 0
I20251211 20:52:47.302347 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet fd43a27db69044b68826cda87e16a086 (table default.Table_a8ebfd7d7a7f4f6cae8b6b02b99b80b0 [id=f1d32002c612403a82bdff5baf209b9e]). Expected version 1 got 0
I20251211 20:52:47.303046 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 497c10742f01476db885bd70c4d2c491 (table default.Table_18b21232dfa74378b27defe672bb2574 [id=602d7eee703948d99a0edcb7cac0651b]). Expected version 1 got 0
I20251211 20:52:47.303484 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 3003ef71a3554c71a3b24cdfb9c293bc (table default.Table_92efd469dd324f7d886e2cbc90ece61c [id=6e7fb3271e0c48c0bb9d56fe4fccee3a]). Expected version 1 got 0
I20251211 20:52:47.304477 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 4fe5c534b572491c911aae01eaa79bae (table default.Table_c1f9dca5c2144d50b67681dcc6800e77 [id=14fa8debfe5443ce81587f02dfa15847]). Expected version 2 got 0
I20251211 20:52:47.306872 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd1ae033dedc436fa3a3f864a7a208a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.292284 880 tablet_service.cc:1505] Processing CreateTablet for tablet 5a16915e31b64a20ad6409c38410c978 (DEFAULT_TABLE table=default.Table_4585c080cc0f44a3af3d60f96f928eb2 [id=ce43ec33f1414838992e472b6bad7dce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.295503 894 tablet_service.cc:1505] Processing CreateTablet for tablet 3ca3d6cf418e4e718f6bb24c8582d512 (DEFAULT_TABLE table=default.Table_d8d6a529a8934b64a5a430cb54ac0954 [id=a2b9a68c9d9147b79b16fd7bcd83f7f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.307118 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37c12e3157c54adb8ed5621a9a27d613. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.298529 891 tablet_service.cc:1505] Processing CreateTablet for tablet 1e41560d2d1147c49e3f3454fb2b069b (DEFAULT_TABLE table=default.Table_7759f916c14543c38f2a391980a1b38b [id=d23c560e49134034a520447a45c5a945]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.302048 888 tablet_service.cc:1505] Processing CreateTablet for tablet a49fbec29ff241d299dce318fce0cead (DEFAULT_TABLE table=default.Table_ff2dfead5c37456d91374a4bc62c85ea [id=14be5949ea90486d97522326490ce78b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.303153 1160 tablet_bootstrap.cc:492] T 5453f68789fd4e89b69e98860050865b P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:47.309796 1160 tablet_bootstrap.cc:654] T 5453f68789fd4e89b69e98860050865b P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:47.311730 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1647056e32af435a86c48c1d8c8bf412. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.313791 2550 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:52:47.315094 877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebe6ca12e5e14337b317cad3621029ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.303277 886 tablet_service.cc:1505] Processing CreateTablet for tablet f4f5011512dd4b31853738599384c293 (DEFAULT_TABLE table=default.Table_870517d846e84832912515fdcf0d8b64 [id=3ce9d2e49dbc4e5787f59a621f20b657]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.316474 2550 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251211 20:52:47.304947 889 tablet_service.cc:1505] Processing CreateTablet for tablet 68fe7dfddec04ff79199661dc41a6330 (DEFAULT_TABLE table=default.Table_282dd63df9ff4557a5ab2761d63df7b1 [id=c59fde89abe64038889ab84b50d93ad2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.305526 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a295a24e81f14e5fa126a4a58ffd08cf" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:47.318260 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 51d12723f5194afb97d2c05fcbaf3776 (table default.Table_823505200e3142dbbe8cd56ac78daf55 [id=ec0bb10933fa4529bef3580bee6409e7]). Expected version 2 got 0
I20251211 20:52:47.318787 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 2e800d8cc5ea4535a065d7d9243326b4 (table default.Table_083c478bedec45a5bf5203e4ed333860 [id=8baf0ea87d734d58896adbd0d6ef99c2]). Expected version 1 got 0
I20251211 20:52:47.319010 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet d43210ec61af423ebde624397dea9f2e (table default.Table_af0df4d1ee144455a6041e5a6a0b9b80 [id=7c5b2e33fa1c4b12b3941577360908a8]). Expected version 3 got 0
I20251211 20:52:47.319638 1024 tablet_service.cc:1505] Processing CreateTablet for tablet 3c734e9d003643afb444b2b15f93fa93 (DEFAULT_TABLE table=default.Table_08dc6bc7d022495db033bda27f21c84c [id=77cfe7b6d0094c6497429960b929141c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.319828 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 1c4b2c22bd4c456eb77af45576de466f (table default.Table_e3a99b043e4f420a93ebfee6aa87f2ec [id=3e7d6cefcb7d412191c6540e9ad77c2d]). Expected version 2 got 0
I20251211 20:52:47.320041 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c734e9d003643afb444b2b15f93fa93. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.320227 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet f26ba8f4bd3a4a1f8fa2ba6e80d543f8 (table default.Table_71e9c158dc22435a8fbb02aecca8919d [id=b966d0c0b49d4023894950c5dc06f678]). Expected version 2 got 0
I20251211 20:52:47.320569 1809 catalog_manager.cc:5654] T 2e7e158809b9471fbd6e6ee9fb9ac569 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:47.321110 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet dfd194cf00f549c8937cb6ec40987240 (table default.Table_8db9acb9d35149fdabcb9a1d24061b46 [id=d919c6132b3c466e8a1404f981b003b8]). Expected version 1 got 0
I20251211 20:52:47.321318 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet bc4e7362e6d045da81bf39f4135f500e (table default.Table_328b7dd76e9e407c83b47a0f51cfcce5 [id=9e7c08f833e14a90b17a9b9f60b702de]). Expected version 1 got 0
I20251211 20:52:47.321604 875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93f6f6c240aa403ea783a0c7ff0cfb8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.321739 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet ce1799e6604f4ed7875ddedecab6f327 (table default.Table_5db1ac98e36d45b1b01889ee275ad83d [id=a6849726434d41d29dce12f47066a685]). Expected version 1 got 0
I20251211 20:52:47.322409 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 3b2c6064c5eb4c0f8979c485f5d85a75 (table default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f]). Expected version 1 got 0
I20251211 20:52:47.322582 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 7b9f569554fb4836ac2b62542c3ecf7e (table default.Table_1a73d567e8914a83974e3e8f5f58be0b [id=6ef691749df24251a7a7c5919c219754]). Expected version 1 got 0
I20251211 20:52:47.322721 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 4cf0cde801bf44449e539d2799b895a7 (table default.Table_907c596bdddc4edca25d14cbacebb23f [id=321235280d1d4fa0bae5cc0333860f85]). Expected version 1 got 0
I20251211 20:52:47.323160 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 72af55c8045b4f8f8e116a5782794d81 (table default.Table_2cff6fab088c4d939b13b504c133cc12 [id=b6c4b72063b3428593d5650a160dc66b]). Expected version 1 got 0
I20251211 20:52:47.323379 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b81a4cef27164824b52695039744db15 (table default.Table_defdeab28749411590cf75b5251f604b [id=11352566d1744193a40a8bb605aa3912]). Expected version 1 got 0
I20251211 20:52:47.326198 879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23f6e5415e14478281733c5e59c65d44. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.327060 1019 tablet_service.cc:1505] Processing CreateTablet for tablet dd1ae033dedc436fa3a3f864a7a208a2 (DEFAULT_TABLE table=default.Table_233d5484bd4e4456aed925bc271edfa5 [id=ca0c3c6978874d93a04b57cc0c425c83]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.327473 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd1ae033dedc436fa3a3f864a7a208a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.327862 1024 tablet_service.cc:1505] Processing CreateTablet for tablet 37c12e3157c54adb8ed5621a9a27d613 (DEFAULT_TABLE table=default.Table_34c853fcb87d480fb23827277ebed594 [id=15901f3423534245ba63bf94be889977]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.327967 1021 tablet_service.cc:1505] Processing CreateTablet for tablet 413ad8f69a4a4f5898ef6ddbb395c821 (DEFAULT_TABLE table=default.Table_89bf0fe9a0c54cae85b1b8305d2b91e8 [id=9671f96ca9fe424db31d663aeee7f725]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.328197 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37c12e3157c54adb8ed5621a9a27d613. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.328261 878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46e5c35780ad498ab4c274d311cd606f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.329584 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 413ad8f69a4a4f5898ef6ddbb395c821. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.329595 2759 raft_consensus.cc:493] T efe4d6bb07374294b8ab50a9fe605f87 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251211 20:52:47.330071 712 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.Ping -> {remote=127.0.104.60:42087, user_credentials={real_user=slave}} blocked reactor thread for 79913.7us
I20251211 20:52:47.331902 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 23f6e5415e14478281733c5e59c65d44 (DEFAULT_TABLE table=default.Table_50aa8aa5823a4e71b0f191ff52eefb75 [id=95fb6825923a4a679a7e5d6695863a02]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.332255 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23f6e5415e14478281733c5e59c65d44. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.335708 1010 tablet_service.cc:1505] Processing CreateTablet for tablet ebe6ca12e5e14337b317cad3621029ca (DEFAULT_TABLE table=default.Table_7b6d7c6f71514a118834aa645c8405e9 [id=8564c2055c9d420cb3228a39f66c66fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.336055 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebe6ca12e5e14337b317cad3621029ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.338882 1809 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 5b3e8ec3faf445ac9b746e65dc4dbfe0 (table default.Table_7fe80ff5483a4b0292091119980f6639 [id=3c3462a0c09748cea0d2ffb544ae2f13]). Expected version 1 got 0
I20251211 20:52:47.339447 1808 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Sending DeleteTablet for 3 replicas of tablet 9c5c937cc7db4ed18fe03f4a8a99a66a
I20251211 20:52:47.340010 1808 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46568 (request call id 60) took 2395 ms. Trace:
I20251211 20:52:47.339972 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 93f6f6c240aa403ea783a0c7ff0cfb8a (DEFAULT_TABLE table=default.Table_d8214c15312c49ce85f2c52b14cf4a78 [id=bdca175d3ca14dfc912d92c7f4beae89]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.340385 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93f6f6c240aa403ea783a0c7ff0cfb8a. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:47.340440 1810 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d37a7d0d662e45079b0cfc02765d2e01 consensus state has no leader
I20251211 20:52:47.340612 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 170ef6a9993c4b00ac3ddbec411bb5a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.340642 1810 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46568 (request call id 58) took 2427 ms. Trace:
I20251211 20:52:47.341032 876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 413ad8f69a4a4f5898ef6ddbb395c821. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.341207 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c734e9d003643afb444b2b15f93fa93. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.340739 1810 rpcz_store.cc:276] 1211 20:52:44.913175 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:44.913260 (+ 85us) service_pool.cc:225] Handling call
1211 20:52:45.079092 (+165832us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
1211 20:52:45.079282 (+ 190us) catalog_manager.cc:3708] Apply alter schema
1211 20:52:45.079488 (+ 206us) catalog_manager.cc:3740] Acquired catalog manager lock
1211 20:52:45.079547 (+ 59us) catalog_manager.cc:3898] Updating metadata on disk
1211 20:52:45.241338 (+161791us) catalog_manager.cc:3929] Committing alterations to in-memory state
1211 20:52:47.340576 (+2099238us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:45.090593 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:45.090689 (+ 96us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:45.090694 (+ 5us) write_op.cc:435] Acquired schema lock
1211 20:52:45.090696 (+ 2us) tablet.cc:660] Decoding operations
1211 20:52:45.090769 (+ 73us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:45.090778 (+ 9us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:45.090782 (+ 4us) tablet.cc:683] Acquiring locks for 1 operations
1211 20:52:45.090830 (+ 48us) tablet.cc:699] Row locks acquired
1211 20:52:45.090832 (+ 2us) write_op.cc:260] PREPARE: finished
1211 20:52:45.090861 (+ 29us) write_op.cc:270] Start()
1211 20:52:45.090887 (+ 26us) write_op.cc:276] Timestamp: P: 1765486365090826 usec, L: 0
1211 20:52:45.090891 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:45.091015 (+ 124us) log.cc:844] Serialized 405 byte log entry
1211 20:52:45.239819 (+148804us) op_driver.cc:464] REPLICATION: finished
1211 20:52:45.240625 (+ 806us) write_op.cc:301] APPLY: starting
1211 20:52:45.240636 (+ 11us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:45.240663 (+ 27us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:45.240708 (+ 45us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:45.240746 (+ 38us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:45.240770 (+ 24us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1211 20:52:45.240776 (+ 6us) write_op.cc:312] APPLY: finished
1211 20:52:45.240869 (+ 93us) log.cc:844] Serialized 23 byte log entry
1211 20:52:45.240913 (+ 44us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:45.240927 (+ 14us) write_op.cc:454] Released schema lock
1211 20:52:45.240932 (+ 5us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:45.240996 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:45.240999 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":777,"apply.run_cpu_time_us":362,"apply.run_wall_time_us":361,"num_ops":1,"prepare.queue_time_us":9222,"prepare.run_cpu_time_us":501,"prepare.run_wall_time_us":501,"raft.queue_time_us":236,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":6,"replication_time_us":148912}]]}
W20251211 20:52:47.341673 1810 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.2:53985 (request call id 170) took 1425 ms (client timeout 1000 ms). Trace:
W20251211 20:52:47.341758 1810 rpcz_store.cc:269] 1211 20:52:45.916007 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:47.341429 (+1425422us) service_pool.cc:212] Skipping call since client already timed out
1211 20:52:47.341627 (+ 198us) inbound_call.cc:173] Queueing failure response
Metrics: {}
W20251211 20:52:47.342031 1810 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.3:60211 (request call id 157) took 1286 ms (client timeout 1000 ms). Trace:
W20251211 20:52:47.342115 1810 rpcz_store.cc:269] 1211 20:52:46.055304 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:47.341961 (+1286657us) service_pool.cc:212] Skipping call since client already timed out
1211 20:52:47.341989 (+ 28us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20251211 20:52:47.342451 877 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c5c937cc7db4ed18fe03f4a8a99a66a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:47.342561 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 087bc334ae9c4018b6b86afef4354911. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.343909 1013 tablet_service.cc:1505] Processing CreateTablet for tablet 46e5c35780ad498ab4c274d311cd606f (DEFAULT_TABLE table=default.Table_d424820106284c94adee000401d8ab22 [id=92aecd5e8f9b453082fd628c9b24e3cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.344249 1013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46e5c35780ad498ab4c274d311cd606f. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:47.340654 1798 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7023afec769c4ca3b2242a225004d482 consensus state has no leader
I20251211 20:52:47.345664 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a49fbec29ff241d299dce318fce0cead. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.347926 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 087bc334ae9c4018b6b86afef4354911 (DEFAULT_TABLE table=default.Table_280921729cb94c62b206fc68ef348c61 [id=cff90a42974d452786230faab90e95ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.348268 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 087bc334ae9c4018b6b86afef4354911. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.350015 1798 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46568 (request call id 67) took 2316 ms. Trace:
I20251211 20:52:47.350121 1798 rpcz_store.cc:276] 1211 20:52:45.033145 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:45.241668 (+208523us) service_pool.cc:225] Handling call
1211 20:52:45.423243 (+181575us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x616000040f54. stack: 00007ff715c18ee6 00007ff715c18ca0 00007ff714bf43ad 0000000000984f94 0000000000984f5d 00007ff724f1f6bd 00007ff724f1f7b3 00007ff721ae6521 00007ff7297a596d 00007ff729934456 00007ff720440146 00007ff72043fe75 00007ff71cbb856e 00007ff71cbb744e 00007ff71cbbd685 00007ff71cbbdfaf
1211 20:52:45.423630 (+ 387us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
1211 20:52:45.423870 (+ 240us) catalog_manager.cc:3708] Apply alter schema
1211 20:52:45.424103 (+ 233us) catalog_manager.cc:3740] Acquired catalog manager lock
1211 20:52:45.424191 (+ 88us) catalog_manager.cc:3898] Updating metadata on disk
1211 20:52:45.457055 (+ 32864us) catalog_manager.cc:3929] Committing alterations to in-memory state
1211 20:52:47.349189 (+1892134us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:45.424842 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:45.424952 (+ 110us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:45.424959 (+ 7us) write_op.cc:435] Acquired schema lock
1211 20:52:45.424962 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:45.425071 (+ 109us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:45.425082 (+ 11us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:45.425087 (+ 5us) tablet.cc:683] Acquiring locks for 1 operations
1211 20:52:45.425157 (+ 70us) tablet.cc:699] Row locks acquired
1211 20:52:45.425160 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:45.425198 (+ 38us) write_op.cc:270] Start()
1211 20:52:45.425236 (+ 38us) write_op.cc:276] Timestamp: P: 1765486365425177 usec, L: 0
1211 20:52:45.425240 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:45.425438 (+ 198us) log.cc:844] Serialized 405 byte log entry
1211 20:52:45.456149 (+ 30711us) op_driver.cc:464] REPLICATION: finished
1211 20:52:45.456284 (+ 135us) write_op.cc:301] APPLY: starting
1211 20:52:45.456303 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:45.456354 (+ 51us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:45.456390 (+ 36us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:45.456467 (+ 77us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:45.456509 (+ 42us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1211 20:52:45.456517 (+ 8us) write_op.cc:312] APPLY: finished
1211 20:52:45.456627 (+ 110us) log.cc:844] Serialized 23 byte log entry
1211 20:52:45.456938 (+ 311us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:45.456961 (+ 23us) write_op.cc:454] Released schema lock
1211 20:52:45.456969 (+ 8us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:45.457181 (+ 212us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:45.457185 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":416939392,"child_traces":[["op",{"apply.queue_time_us":81,"apply.run_cpu_time_us":529,"apply.run_wall_time_us":883,"num_ops":1,"prepare.queue_time_us":59,"prepare.run_cpu_time_us":752,"prepare.run_wall_time_us":752,"raft.queue_time_us":3943,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":459,"replication_time_us":30863}]]}
I20251211 20:52:47.340134 1808 rpcz_store.cc:276] 1211 20:52:44.944317 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:45.068749 (+124432us) service_pool.cc:225] Handling call
1211 20:52:45.069281 (+ 532us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
1211 20:52:45.069426 (+ 145us) catalog_manager.cc:2815] Modifying in-memory table state
1211 20:52:45.069496 (+ 70us) catalog_manager.cc:2824] Locking tablets
1211 20:52:45.069741 (+ 245us) catalog_manager.cc:2838] Removing table and tablets from system table
1211 20:52:45.090287 (+ 20546us) catalog_manager.cc:2856] Removing table from by-name map
1211 20:52:45.090369 (+ 82us) catalog_manager.cc:2872] Committing in-memory state
1211 20:52:47.339348 (+2248979us) catalog_manager.cc:2876] Aborting table tasks
1211 20:52:47.339973 (+ 625us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:45.070529 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:45.070665 (+ 136us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:45.070672 (+ 7us) write_op.cc:435] Acquired schema lock
1211 20:52:45.070674 (+ 2us) tablet.cc:660] Decoding operations
1211 20:52:45.070791 (+ 117us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:45.070805 (+ 14us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:45.070810 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:45.070919 (+ 109us) tablet.cc:699] Row locks acquired
1211 20:52:45.070921 (+ 2us) write_op.cc:260] PREPARE: finished
1211 20:52:45.070959 (+ 38us) write_op.cc:270] Start()
1211 20:52:45.070995 (+ 36us) write_op.cc:276] Timestamp: P: 1765486365070923 usec, L: 0
1211 20:52:45.070998 (+ 3us) op_driver.cc:348] REPLICATION: starting
1211 20:52:45.071184 (+ 186us) log.cc:844] Serialized 756 byte log entry
1211 20:52:45.088711 (+ 17527us) op_driver.cc:464] REPLICATION: finished
1211 20:52:45.088872 (+ 161us) write_op.cc:301] APPLY: starting
1211 20:52:45.088891 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:45.088939 (+ 48us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:45.088972 (+ 33us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:45.089043 (+ 71us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:45.089078 (+ 35us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
1211 20:52:45.089106 (+ 28us) write_op.cc:312] APPLY: finished
1211 20:52:45.089189 (+ 83us) log.cc:844] Serialized 29 byte log entry
1211 20:52:45.089256 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:45.089275 (+ 19us) write_op.cc:454] Released schema lock
1211 20:52:45.089282 (+ 7us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:45.089353 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:45.089356 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":105,"apply.run_cpu_time_us":474,"apply.run_wall_time_us":474,"mutex_wait_us":10138,"num_ops":2,"prepare.queue_time_us":56,"prepare.run_cpu_time_us":820,"prepare.run_wall_time_us":10949,"raft.queue_time_us":10219,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":305,"replication_time_us":17682,"spinlock_wait_cycles":10624}]]}
I20251211 20:52:47.351801 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a16915e31b64a20ad6409c38410c978. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.351989 1806 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Sending DeleteTablet for 3 replicas of tablet 87c4ec07e82d4bf79b4eeefe732674ff
I20251211 20:52:47.351975 1020 tablet_service.cc:1505] Processing CreateTablet for tablet 170ef6a9993c4b00ac3ddbec411bb5a6 (DEFAULT_TABLE table=default.Table_7f9e421192224c908c84e08e5b787787 [id=f8a9880896634b27b099dc95674eef45]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.352334 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 170ef6a9993c4b00ac3ddbec411bb5a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.352511 1806 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46568 (request call id 64) took 2353 ms. Trace:
I20251211 20:52:47.352635 1806 rpcz_store.cc:276] 1211 20:52:44.998841 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:45.098981 (+100140us) service_pool.cc:225] Handling call
1211 20:52:45.191868 (+ 92887us) spinlock_profiling.cc:243] Waited 92.6 ms on lock 0x616000040f54. stack: 00007ff715c18ee6 00007ff715c18ca0 00007ff714bf43ad 0000000000984f94 0000000000984f5d 00007ff724f1f6bd 00007ff724f1f7b3 00007ff721ae6521 00007ff7297a596d 00007ff7299331b6 00007ff72043e486 00007ff72043e1b5 00007ff71cbb856e 00007ff71cbb744e 00007ff71cbbd685 00007ff71cbbdfaf
1211 20:52:45.192220 (+ 352us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
1211 20:52:45.192344 (+ 124us) catalog_manager.cc:2815] Modifying in-memory table state
1211 20:52:45.192459 (+ 115us) catalog_manager.cc:2824] Locking tablets
1211 20:52:45.192591 (+ 132us) catalog_manager.cc:2838] Removing table and tablets from system table
1211 20:52:45.429935 (+237344us) catalog_manager.cc:2856] Removing table from by-name map
1211 20:52:45.430023 (+ 88us) catalog_manager.cc:2872] Committing in-memory state
1211 20:52:47.351920 (+1921897us) catalog_manager.cc:2876] Aborting table tasks
1211 20:52:47.352479 (+ 559us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:45.193372 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:45.193495 (+ 123us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:45.193500 (+ 5us) write_op.cc:435] Acquired schema lock
1211 20:52:45.193502 (+ 2us) tablet.cc:660] Decoding operations
1211 20:52:45.193624 (+ 122us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:45.193634 (+ 10us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:45.193639 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:45.193727 (+ 88us) tablet.cc:699] Row locks acquired
1211 20:52:45.193730 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:45.193766 (+ 36us) write_op.cc:270] Start()
1211 20:52:45.193801 (+ 35us) write_op.cc:276] Timestamp: P: 1765486365193718 usec, L: 0
1211 20:52:45.193805 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:45.193994 (+ 189us) log.cc:844] Serialized 756 byte log entry
1211 20:52:45.429107 (+235113us) op_driver.cc:464] REPLICATION: finished
1211 20:52:45.429253 (+ 146us) write_op.cc:301] APPLY: starting
1211 20:52:45.429269 (+ 16us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:45.429318 (+ 49us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:45.429353 (+ 35us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:45.429465 (+ 112us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:45.429502 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
1211 20:52:45.429510 (+ 8us) write_op.cc:312] APPLY: finished
1211 20:52:45.429588 (+ 78us) log.cc:844] Serialized 29 byte log entry
1211 20:52:45.429822 (+ 234us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:45.429843 (+ 21us) write_op.cc:454] Released schema lock
1211 20:52:45.429849 (+ 6us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:45.430088 (+ 239us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:45.430092 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":213005056,"child_traces":[["op",{"apply.queue_time_us":86,"apply.run_cpu_time_us":512,"apply.run_wall_time_us":823,"num_ops":2,"prepare.queue_time_us":52,"prepare.run_cpu_time_us":711,"prepare.run_wall_time_us":712,"raft.queue_time_us":1312,"raft.run_cpu_time_us":7,"raft.run_wall_time_us":7,"replication_time_us":235273}]]}
I20251211 20:52:47.353422 1806 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_4d6139c7976d414fbdcbdb96787d186b" } new_table_name: "default.Table_5ca4ab25a21d456c95a0ff68c9d73330" modify_external_catalogs: true
I20251211 20:52:47.355288 1021 tablet_service.cc:1505] Processing CreateTablet for tablet 274f00ed89484f158a57cdc0a16eb96c (DEFAULT_TABLE table=default.Table_171c2a61012b444caf720781072205a3 [id=64212968bb94465093adb65222437d60]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.355455 1016 tablet_service.cc:1505] Processing CreateTablet for tablet 3ca3d6cf418e4e718f6bb24c8582d512 (DEFAULT_TABLE table=default.Table_d8d6a529a8934b64a5a430cb54ac0954 [id=a2b9a68c9d9147b79b16fd7bcd83f7f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.355660 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 274f00ed89484f158a57cdc0a16eb96c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.355827 1016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ca3d6cf418e4e718f6bb24c8582d512. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.355753 1018 tablet_service.cc:1505] Processing CreateTablet for tablet 5a16915e31b64a20ad6409c38410c978 (DEFAULT_TABLE table=default.Table_4585c080cc0f44a3af3d60f96f928eb2 [id=ce43ec33f1414838992e472b6bad7dce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.355832 1015 tablet_service.cc:1505] Processing CreateTablet for tablet 1647056e32af435a86c48c1d8c8bf412 (DEFAULT_TABLE table=default.Table_cd866c64dfc7477782129c897b849771 [id=ef146ea6ee794351944112c4b0dcef58]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.356088 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a16915e31b64a20ad6409c38410c978. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.359014 1802 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Sending DeleteTablet for 3 replicas of tablet 5f2c14c7a972414e8328e0b388bde0a4
I20251211 20:52:47.359599 1802 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46568 (request call id 68) took 2316 ms. Trace:
I20251211 20:52:47.359697 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1647056e32af435a86c48c1d8c8bf412. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.359707 1802 rpcz_store.cc:276] 1211 20:52:45.042869 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:45.458473 (+415604us) service_pool.cc:225] Handling call
1211 20:52:45.466145 (+ 7672us) spinlock_profiling.cc:243] Waited 7.37 ms on lock 0x616000040f54. stack: 00007ff715c18ee6 00007ff715c18ca0 00007ff714bf43ad 0000000000984f94 0000000000984f5d 00007ff724f1f6bd 00007ff724f1f7b3 00007ff721ae6521 00007ff7297a596d 00007ff7299331b6 00007ff72043e486 00007ff72043e1b5 00007ff71cbb856e 00007ff71cbb744e 00007ff71cbbd685 00007ff71cbbdfaf
1211 20:52:45.466602 (+ 457us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
1211 20:52:45.466718 (+ 116us) catalog_manager.cc:2815] Modifying in-memory table state
1211 20:52:45.466795 (+ 77us) catalog_manager.cc:2824] Locking tablets
1211 20:52:45.466958 (+ 163us) catalog_manager.cc:2838] Removing table and tablets from system table
1211 20:52:45.478699 (+ 11741us) catalog_manager.cc:2856] Removing table from by-name map
1211 20:52:45.711666 (+232967us) catalog_manager.cc:2872] Committing in-memory state
1211 20:52:47.358962 (+1647296us) catalog_manager.cc:2876] Aborting table tasks
1211 20:52:47.359506 (+ 544us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:45.467858 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:45.468004 (+ 146us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:45.468012 (+ 8us) write_op.cc:435] Acquired schema lock
1211 20:52:45.468015 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:45.468133 (+ 118us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:45.468148 (+ 15us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:45.468152 (+ 4us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:45.468267 (+ 115us) tablet.cc:699] Row locks acquired
1211 20:52:45.468269 (+ 2us) write_op.cc:260] PREPARE: finished
1211 20:52:45.468312 (+ 43us) write_op.cc:270] Start()
1211 20:52:45.468353 (+ 41us) write_op.cc:276] Timestamp: P: 1765486365468286 usec, L: 0
1211 20:52:45.468357 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:45.468563 (+ 206us) log.cc:844] Serialized 725 byte log entry
1211 20:52:45.477350 (+ 8787us) op_driver.cc:464] REPLICATION: finished
1211 20:52:45.477550 (+ 200us) write_op.cc:301] APPLY: starting
1211 20:52:45.477572 (+ 22us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:45.477622 (+ 50us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:45.477661 (+ 39us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:45.478089 (+ 428us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:45.478120 (+ 31us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
1211 20:52:45.478127 (+ 7us) write_op.cc:312] APPLY: finished
1211 20:52:45.478323 (+ 196us) log.cc:844] Serialized 29 byte log entry
1211 20:52:45.478577 (+ 254us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:45.478600 (+ 23us) write_op.cc:454] Released schema lock
1211 20:52:45.478609 (+ 9us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:45.478791 (+ 182us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:45.478795 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":16959232,"child_traces":[["op",{"apply.queue_time_us":119,"apply.run_cpu_time_us":559,"apply.run_wall_time_us":1208,"num_ops":2,"prepare.queue_time_us":66,"prepare.run_cpu_time_us":855,"prepare.run_wall_time_us":855,"raft.queue_time_us":8214,"raft.run_cpu_time_us":318,"raft.run_wall_time_us":777,"replication_time_us":8956}]]}
I20251211 20:52:47.360662 1802 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_3c308b240c3b490d9022c7d292cdb743"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:47.361232 1802 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3c308b240c3b490d9022c7d292cdb743 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:47.388738 1802 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46568 (request call id 70) took 2119 ms. Trace:
I20251211 20:52:47.388892 1802 rpcz_store.cc:276] 1211 20:52:45.269600 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:47.360442 (+2090842us) service_pool.cc:225] Handling call
1211 20:52:47.361339 (+ 897us) catalog_manager.cc:2101] Acquired catalog manager lock
1211 20:52:47.361806 (+ 467us) catalog_manager.cc:2147] Created new table and tablet info
1211 20:52:47.388347 (+ 26541us) catalog_manager.cc:2207] Wrote table and tablets to system table
1211 20:52:47.388563 (+ 216us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
1211 20:52:47.388673 (+ 110us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:47.380016 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:47.380188 (+ 172us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:47.380195 (+ 7us) write_op.cc:435] Acquired schema lock
1211 20:52:47.380198 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:47.380313 (+ 115us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:47.380329 (+ 16us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:47.380334 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:47.380424 (+ 90us) tablet.cc:699] Row locks acquired
1211 20:52:47.380427 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:47.380468 (+ 41us) write_op.cc:270] Start()
1211 20:52:47.380507 (+ 39us) write_op.cc:276] Timestamp: P: 1765486367380403 usec, L: 0
1211 20:52:47.380511 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:47.380751 (+ 240us) log.cc:844] Serialized 433 byte log entry
1211 20:52:47.385047 (+ 4296us) op_driver.cc:464] REPLICATION: finished
1211 20:52:47.385535 (+ 488us) write_op.cc:301] APPLY: starting
1211 20:52:47.385558 (+ 23us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:47.385615 (+ 57us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:47.385661 (+ 46us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:47.385775 (+ 114us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:47.385816 (+ 41us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1211 20:52:47.385825 (+ 9us) write_op.cc:312] APPLY: finished
1211 20:52:47.385907 (+ 82us) log.cc:844] Serialized 29 byte log entry
1211 20:52:47.385976 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.386005 (+ 29us) write_op.cc:454] Released schema lock
1211 20:52:47.386012 (+ 7us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:47.386087 (+ 75us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.386090 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":423,"apply.run_cpu_time_us":547,"apply.run_wall_time_us":547,"num_ops":2,"prepare.queue_time_us":58,"prepare.run_cpu_time_us":875,"prepare.run_wall_time_us":875,"raft.queue_time_us":254,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":282,"replication_time_us":4482}]]}
I20251211 20:52:47.390506 1802 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_5b3f562b5129421a9d27a06bebc88fe4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:47.395008 1017 tablet_service.cc:1505] Processing CreateTablet for tablet a49fbec29ff241d299dce318fce0cead (DEFAULT_TABLE table=default.Table_ff2dfead5c37456d91374a4bc62c85ea [id=14be5949ea90486d97522326490ce78b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.395144 1015 tablet_service.cc:1505] Processing CreateTablet for tablet 00c4326b662541b8b33242b2a82235a5 (DEFAULT_TABLE table=default.Table_e133e518b951424cba3117dfdf77d480 [id=3639f4db6085489f975da70c3340d5b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.395443 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a49fbec29ff241d299dce318fce0cead. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.395838 1021 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c5c937cc7db4ed18fe03f4a8a99a66a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:47.396199 1018 tablet_service.cc:1505] Processing CreateTablet for tablet 68fe7dfddec04ff79199661dc41a6330 (DEFAULT_TABLE table=default.Table_282dd63df9ff4557a5ab2761d63df7b1 [id=c59fde89abe64038889ab84b50d93ad2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.396553 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68fe7dfddec04ff79199661dc41a6330. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:47.396641 1784 consensus_peers.cc:597] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 -> Peer 9230f94a6a1a49cead51aeea9f069087 (127.0.104.60:42087): Couldn't send request to peer 9230f94a6a1a49cead51aeea9f069087. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.60:42087: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20251211 20:52:47.397013 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 1e41560d2d1147c49e3f3454fb2b069b (DEFAULT_TABLE table=default.Table_7759f916c14543c38f2a391980a1b38b [id=d23c560e49134034a520447a45c5a945]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.397363 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e41560d2d1147c49e3f3454fb2b069b. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:47.398152 1783 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 7023afec769c4ca3b2242a225004d482 (table default.Table_d5276e3cc46d4331b36a8878680fc55e [id=3811f8235b2f44cca283401302d613d4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 7023afec769c4ca3b2242a225004d482 consensus state has no leader
I20251211 20:52:47.399720 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00c4326b662541b8b33242b2a82235a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.399880 1010 tablet_service.cc:1505] Processing CreateTablet for tablet f4f5011512dd4b31853738599384c293 (DEFAULT_TABLE table=default.Table_870517d846e84832912515fdcf0d8b64 [id=3ce9d2e49dbc4e5787f59a621f20b657]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.400216 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4f5011512dd4b31853738599384c293. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.401422 1029 tablet_service.cc:1505] Processing CreateTablet for tablet db0504131ddd4e3c88b2b44bd2bcb920 (DEFAULT_TABLE table=default.Table_28bfcb62d3324b46af3d8e28ab649bf4 [id=9d63f6afeb224e12aaed5b9eccbe234b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.402199 1016 tablet_service.cc:1552] Processing DeleteTablet for tablet 5f2c14c7a972414e8328e0b388bde0a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:36220
W20251211 20:52:47.402971 1802 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5b3f562b5129421a9d27a06bebc88fe4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:47.404047 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 87c4ec07e82d4bf79b4eeefe732674ff with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:47.404500 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db0504131ddd4e3c88b2b44bd2bcb920. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.431401 1166 tablet_bootstrap.cc:492] T 962b80ae747b4470abd23643e711fad9 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:47.431600 1166 ts_tablet_manager.cc:1403] T 962b80ae747b4470abd23643e711fad9 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.466s user 0.005s sys 0.004s
I20251211 20:52:47.432633 1166 raft_consensus.cc:359] T 962b80ae747b4470abd23643e711fad9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.432916 1166 raft_consensus.cc:385] T 962b80ae747b4470abd23643e711fad9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:47.432973 1166 raft_consensus.cc:740] T 962b80ae747b4470abd23643e711fad9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:47.433152 1166 consensus_queue.cc:260] T 962b80ae747b4470abd23643e711fad9 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.435755 1166 ts_tablet_manager.cc:1434] T 962b80ae747b4470abd23643e711fad9 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20251211 20:52:47.456857 2948 raft_consensus.cc:493] T 969a8587019c4202b6e705d126f83d44 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.457038 2948 raft_consensus.cc:515] T 969a8587019c4202b6e705d126f83d44 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.457628 2948 leader_election.cc:290] T 969a8587019c4202b6e705d126f83d44 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:47.459178 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "44e2d2bdecf247baaac26e06bccc94de" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:47.459362 1045 raft_consensus.cc:2468] T 44e2d2bdecf247baaac26e06bccc94de P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:47.465605 2807 raft_consensus.cc:493] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.465798 2807 raft_consensus.cc:515] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.466590 2807 leader_election.cc:290] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.475302 2948 raft_consensus.cc:493] T 5df87ea7d28340da84ef141c92efc710 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.475479 2948 raft_consensus.cc:515] T 5df87ea7d28340da84ef141c92efc710 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.476086 2948 leader_election.cc:290] T 5df87ea7d28340da84ef141c92efc710 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
W20251211 20:52:47.476591 771 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s user 0.008s sys 0.004s
W20251211 20:52:47.476697 771 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s user 0.008s sys 0.004s
W20251211 20:52:47.477919 847 leader_election.cc:343] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: c74427cb9d6f4a679e43eae796ff8c3f
I20251211 20:52:47.486132 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b52377c8ffd741a598e5328a00d876fe (table default.Table_686a26915b42421380037144900dd97c [id=8a1b0c9d641b4a3c83d1ca48576d110d]). Expected version 1 got 0
I20251211 20:52:47.486794 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet c7a2610f20df4c1ca81eae2fc7dba6bb (table default.Table_59072235d93b41c0b5cd5e3debcb8ddf [id=9a15a8a075af4969ae2c8b3be1bfed78]). Expected version 1 got 0
I20251211 20:52:47.487412 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 0302cc8613934c43a0650f081cf98797 (table default.Table_bf827b216f1b43c196923dc1828ff0d0 [id=ed371e073250439fa44330c7101e9387]). Expected version 1 got 0
I20251211 20:52:47.488202 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 1f4eb1a174c648c897ae2b7612e44585 (table default.Table_e70873b71ba24d7eae548e2acfd243b3 [id=87adcc552a704d56bd2fe812fb3b791f]). Expected version 1 got 0
I20251211 20:52:47.488523 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet a7afcce522494c9ca1d80e6eea913e6e (table default.Table_cb6a4bac7b694f33a37de2ee78c91ca0 [id=8940f18527c249a9baefc170ed82f45a]). Expected version 1 got 0
I20251211 20:52:47.488801 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 84bb3583857848459e0e865283de623d (table default.Table_92281c5fcfee4355a79215ffedeee588 [id=6e5fe0a78e4b48f98dac028802f1ddf4]). Expected version 4 got 0
I20251211 20:52:47.489063 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 7a299ee03d0e48aba7c4f71e3e0f381a (table default.Table_6402a801d48e4a53827b1b62b6cdaba1 [id=6876565d3a9c48ce841b830cf75139ff]). Expected version 3 got 0
I20251211 20:52:47.489528 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 5b3e8ec3faf445ac9b746e65dc4dbfe0 (table default.Table_7fe80ff5483a4b0292091119980f6639 [id=3c3462a0c09748cea0d2ffb544ae2f13]). Expected version 1 got 0
I20251211 20:52:47.489941 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 7ffa06c4a7ee48b0b337a27aff090592 (table default.Table_647731023b044ebb8954f7fdcc3faf97 [id=4b91abd6e9e94a8a941b33e62e23c083]). Expected version 1 got 0
I20251211 20:52:47.490206 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b81a4cef27164824b52695039744db15 (table default.Table_defdeab28749411590cf75b5251f604b [id=11352566d1744193a40a8bb605aa3912]). Expected version 1 got 0
I20251211 20:52:47.490456 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 7023afec769c4ca3b2242a225004d482 (table default.Table_d5276e3cc46d4331b36a8878680fc55e [id=3811f8235b2f44cca283401302d613d4]). Expected version 1 got 0
I20251211 20:52:47.490711 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 72af55c8045b4f8f8e116a5782794d81 (table default.Table_2cff6fab088c4d939b13b504c133cc12 [id=b6c4b72063b3428593d5650a160dc66b]). Expected version 1 got 0
I20251211 20:52:47.493456 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 4cf0cde801bf44449e539d2799b895a7 (table default.Table_907c596bdddc4edca25d14cbacebb23f [id=321235280d1d4fa0bae5cc0333860f85]). Expected version 1 got 0
I20251211 20:52:47.493744 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 7b9f569554fb4836ac2b62542c3ecf7e (table default.Table_1a73d567e8914a83974e3e8f5f58be0b [id=6ef691749df24251a7a7c5919c219754]). Expected version 1 got 0
I20251211 20:52:47.494045 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 3b2c6064c5eb4c0f8979c485f5d85a75 (table default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f]). Expected version 1 got 0
I20251211 20:52:47.494380 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 59acb8474e114bf7ae7f1f043d611459 (table default.Table_c7ed8c0c309c40a3a371ffa130b47cd2 [id=ed1d0670426945aba888b818ad657225]). Expected version 1 got 0
I20251211 20:52:47.494940 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet a3b1aadf17ea4bf0a896d23cd543fca7 (table default.Table_d18401707c88493abdf90d2c713432ae [id=a07662f8dc3f4178be5dfcc1cdc85837]). Expected version 1 got 0
I20251211 20:52:47.495517 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 9f8aa28c1c9b4fc58dfc128952c180ba (table default.Table_bf7d402a36be44cfb6b79041906d26a1 [id=0327e686e7ed4475a3e8cda8721a3a3a]). Expected version 1 got 0
I20251211 20:52:47.495677 2808 raft_consensus.cc:493] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.495841 2808 raft_consensus.cc:515] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.496755 2808 leader_election.cc:290] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.496675 881 tablet_service.cc:1505] Processing CreateTablet for tablet db0504131ddd4e3c88b2b44bd2bcb920 (DEFAULT_TABLE table=default.Table_28bfcb62d3324b46af3d8e28ab649bf4 [id=9d63f6afeb224e12aaed5b9eccbe234b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.496968 2759 raft_consensus.cc:515] T efe4d6bb07374294b8ab50a9fe605f87 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.497596 2759 leader_election.cc:290] T efe4d6bb07374294b8ab50a9fe605f87 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.500042 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 3c865c99e9314f5f91658b9126da4d39 (table default.Table_0d7df3fba38849a49b65d8290812b95c [id=7f966d5cdad3423bb770414802a3cf7b]). Expected version 1 got 0
I20251211 20:52:47.500272 847 leader_election.cc:304] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:47.500989 883 tablet_service.cc:1505] Processing CreateTablet for tablet 00c4326b662541b8b33242b2a82235a5 (DEFAULT_TABLE table=default.Table_e133e518b951424cba3117dfdf77d480 [id=3639f4db6085489f975da70c3340d5b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.503813 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 33fd09ea06ee47908b85911e4ee79708 (table default.Table_3c3cd48c8fe5411e96cdf811d492925d [id=30620a01c27f456bac05fa7c02a2cb4d]). Expected version 2 got 0
I20251211 20:52:47.503813 770 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "efe4d6bb07374294b8ab50a9fe605f87" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:47.504215 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 6212b3b4425543168837946a54d60834 (table default.Table_8c30c4151d484db0bb9e2b9ca262248d [id=2d489cd7d73e403d84bf10960e6a2dfb]). Expected version 1 got 0
I20251211 20:52:47.504653 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet aa50ddb2789b4be3bfa9aedef9287109 (table default.Table_44f971096d1e41f480e594985ffd5c17 [id=d4e1349374a346a5b8be180f9bdc329a]). Expected version 1 got 0
I20251211 20:52:47.505288 885 tablet_service.cc:1552] Processing DeleteTablet for tablet 5f2c14c7a972414e8328e0b388bde0a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:47.505446 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 8b793241ee3b42b1a38e9f2f05e1173b (table default.Table_7506ffecc2b64648a9ba736e5298ddf4 [id=279f39d718ba45318f52f95e8aa13652]). Expected version 2 got 0
I20251211 20:52:47.505754 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet f2d6c9781b4d4c37bc08703347963d5d (table default.Table_2aa7ff9186a04249abf8cd971cc0cb17 [id=b50ab5b032ba4c98938324460512468f]). Expected version 1 got 0
I20251211 20:52:47.506812 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 645b08b33676420c96e880de3d460829 (table default.Table_e02933dd91ce4fe7a40d1c3b2f9b2d4b [id=6d42f260246e420d8818b3841926302b]). Expected version 2 got 0
I20251211 20:52:47.507869 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet ab4e7a9062134e489d6f9abe6ff26fb4 (table default.Table_4cbc8841b276406cb4434629653a5ce3 [id=b958532650374d609199cab9c07bb64b]). Expected version 2 got 0
I20251211 20:52:47.508245 1810 catalog_manager.cc:5654] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: HEALTHY } } }
I20251211 20:52:47.509079 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 6b7dd8232d974a43b99ec9b28075dc8d (table default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]). Expected version 2 got 0
I20251211 20:52:47.512238 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 78f77d8916d84edabbe95ce17d8df2c1 (table default.Table_e2af0e2fbd3d46d595136c7a7bf24e4c [id=ddc12ae6d8ed4587b5ef4da3ef81a055]). Expected version 1 got 0
I20251211 20:52:47.512648 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 5e47af616e1b4d2db3780c1d67cdcb47 (table default.Table_296bdf30a11541c389afe3df0b0f667a [id=052edcbde60c41449d468c8a449fc3f1]). Expected version 1 got 0
I20251211 20:52:47.513252 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet e3f711f02af843adb4f5c3c750924e33 (table default.Table_34bd5155aa3b4ef7818022ba9346c074 [id=83a61d23e80946439a14096ab3ab1bd9]). Expected version 2 got 0
I20251211 20:52:47.513506 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 7ca98b43863a47c091baae0a6ce65630 (table default.Table_f0a70914acc94231a4e84155c3e4d13f [id=56c070ad80624c698ebe2e01b4929b6e]). Expected version 1 got 0
I20251211 20:52:47.513876 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b6cc819332e1445c96ce3ffac4861454 (table default.Table_db4c0f706e0643b39d39ffa84352826d [id=4d459c42bf584068ac3851227d80331a]). Expected version 1 got 0
I20251211 20:52:47.519330 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b2974f16034c44cf85ff85b5095cf6a0 (table default.Table_031de0f4cc7744bc9e320261f8b46696 [id=be36dabaa6d44325b4855b9f13bda847]). Expected version 1 got 0
I20251211 20:52:47.519740 884 tablet_service.cc:1552] Processing DeleteTablet for tablet 87c4ec07e82d4bf79b4eeefe732674ff with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:47.519893 1810 catalog_manager.cc:5654] T 0763a475527546bb83cee8fb823893ac P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: HEALTHY } } }
I20251211 20:52:47.520936 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 46a2a070cf6a42fe8bc54a42eb802fbc (table default.Table_4e1f03b9d87b4fe59dbcb3a947ee299d [id=8e66ea58499b40e59294a135324aa527]). Expected version 1 got 0
I20251211 20:52:47.521211 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet ff4d5ec1f87247e896f9c099c532265a (table default.Table_3ac4746e17264f5a93890f3600505781 [id=475e0a1ec0344d10addbcfa42fe05b68]). Expected version 1 got 0
I20251211 20:52:47.521615 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 03184dcc56dd4e92adc7d0ad22aa8d8b (table default.Table_68261841754847ad8d22e86c29886144 [id=697f23a617f64762bf406b386cde7473]). Expected version 1 got 0
I20251211 20:52:47.521895 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 18f125dd9efc401181f0644dad7aeffe (table default.Table_fa6592c2e2624d81ad21abb13ed36045 [id=9b48fc2b39db4fd5a4e5da19e34427dc]). Expected version 1 got 0
I20251211 20:52:47.522084 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 5773c13034714a008c8d7dbe777b4828 (table default.Table_66eb1b021eed464f82f9cd223c88297b [id=94ff09564cd649c7a2d0e94b0f3c7b3a]). Expected version 1 got 0
I20251211 20:52:47.522418 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet e89d4d76bf64453892a4d0b55fcc3454 (table default.Table_fba99384e9f14160b3e3504f8d1d0b07 [id=01bba75f854d4c338d6c09a446dd7499]). Expected version 1 got 0
I20251211 20:52:47.522650 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 8dc4e730951144d78b77280f6cce6d0e (table default.Table_890c9027415a44ccaf95c0adede51992 [id=7bc8a2b91fe54b91b6594b371fea7bd3]). Expected version 2 got 0
I20251211 20:52:47.523017 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 4c0455a5b4f44412803475d1c4852b48 (table default.Table_e00092673b0e4f09b8d7eec63e1ee034 [id=18497e06d7254441a4655ce1cdc5a9bf]). Expected version 1 got 0
I20251211 20:52:47.523221 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 497c10742f01476db885bd70c4d2c491 (table default.Table_18b21232dfa74378b27defe672bb2574 [id=602d7eee703948d99a0edcb7cac0651b]). Expected version 1 got 0
I20251211 20:52:47.523440 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 3003ef71a3554c71a3b24cdfb9c293bc (table default.Table_92efd469dd324f7d886e2cbc90ece61c [id=6e7fb3271e0c48c0bb9d56fe4fccee3a]). Expected version 1 got 0
I20251211 20:52:47.523876 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 16d2ff8925e342729a71c93db378d031 (table default.Table_34e6ed4f13da437ea50d349b92e83bd6 [id=4a74b9a60cbb4198b55cee630b2f4bef]). Expected version 2 got 0
I20251211 20:52:47.524263 1810 catalog_manager.cc:5654] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: HEALTHY } } }
I20251211 20:52:47.524726 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet d37a7d0d662e45079b0cfc02765d2e01 (table default.Table_014b6db9f3294658b965c701384b98bb [id=8a12299efe714617a970fc05c3bb6232]). Expected version 2 got 0
I20251211 20:52:47.527016 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet bc6a1c84501d4e179cd2828b50fddc23 (table default.Table_8bb5b228d60e4d29a41d9ba40c33d45d [id=6ef058fada9941629e0ea4d1906abdac]). Expected version 1 got 0
I20251211 20:52:47.527267 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet d6923cfd826d495291d9ac8e0b3b44c6 (table default.Table_68ccb2052d824ccf92f7162d2961fcb1 [id=9ac9591d4f04482d926710fdf6e797c6]). Expected version 1 got 0
I20251211 20:52:47.527622 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 13269a20421240d1a98215129443248b (table default.Table_d887c953026046dc883ede774a58fd8e [id=1d8a79f716a0447cb0393f4759178598]). Expected version 1 got 0
I20251211 20:52:47.527889 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet f94b656e3c1844509fdae36e33b263e8 (table default.Table_ed03ad475bed4804a65abe20efef0625 [id=e23f4305669a474aafa18928f130de3a]). Expected version 2 got 0
I20251211 20:52:47.528148 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 3dfea28c44d84c68850db65a30d52ee6 (table default.Table_c504f8f3796f43b697488bd604268543 [id=2af76b9d32094a4ca656060a4478e08a]). Expected version 1 got 0
I20251211 20:52:47.528270 2500 raft_consensus.cc:493] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.528429 2500 raft_consensus.cc:515] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.529198 2500 leader_election.cc:290] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.529325 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 4c7b74f370b042ee8575f821c07dc085 (table default.Table_731826f26f9940b8afe425d694b642d9 [id=17dd9cea9ac844128ff9f82adb3f919f]). Expected version 1 got 0
I20251211 20:52:47.529597 1810 catalog_manager.cc:5654] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:47.530071 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b85af79c2574443998e847bc19c53b69 (table default.Table_fa3f2f115cd043a9aa6be2b070d320cb [id=15fad5e1cf8d4ea08dae8275a29a11dd]). Expected version 1 got 0
I20251211 20:52:47.531795 770 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1e13965b6361488eaeb7dd13b467ca65" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:47.532159 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f2ae084982f44fdf8bfecdd2bc4c01b9" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:47.532445 770 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "969a8587019c4202b6e705d126f83d44" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:47.533107 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 94658fcf512a48d0a992a7e2e0f7c176 (table default.Table_c9d86e82aebb4ef0bc83789720352e38 [id=97377a8ef56c4cdebe95d15955207404]). Expected version 1 got 0
I20251211 20:52:47.533335 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet f00128ba84c245d9a0e35f26bc842b3e (table default.Table_6f8c133c7a364b93af2c83d8911a0ac3 [id=6a21c38ee7224a02b937df088494f5d8]). Expected version 1 got 0
I20251211 20:52:47.534216 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet fb29f983c4524a49bdc5c0c1144ad708 (table default.Table_7809ac85fdfe4c1d9c58afb54edecb27 [id=c4b0bfff99f64888beb70d9154d2f156]). Expected version 1 got 0
I20251211 20:52:47.534560 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 969a8587019c4202b6e705d126f83d44 (table default.Table_6460313002684bba93b30ad8631f0f49 [id=22347b15537d451d9927a17e8e5a202a]). Expected version 1 got 0
I20251211 20:52:47.534880 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 676faec7f3d1488f99b220f8e968e9a0 (table default.Table_ab826a3eec8e4c4aaa8d4021d87dd382 [id=a2bbfccfe3974c378521d5ef2333df86]). Expected version 2 got 0
I20251211 20:52:47.534943 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de508d79f30f489cb745401f53115165" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:47.535828 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet aab26df748724fe5b2bb7fad9e3bfc0c (table default.Table_22bed4b4e9e24a90818b19c6f5c91e1f [id=df6255d66f334ef2b4406f4b845cf980]). Expected version 1 got 0
I20251211 20:52:47.536136 1810 catalog_manager.cc:5654] T 04086eaa32a74edd84235d3b28f9f965 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: HEALTHY } } }
I20251211 20:52:47.539124 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e41560d2d1147c49e3f3454fb2b069b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.539718 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 8675e2a18f0f4a0e95a87005cf82d8b4 (table default.Table_bd9633c55bd242c18aac3a7f2c18a329 [id=767f50bee5464316a18f73e64bab817a]). Expected version 2 got 0
I20251211 20:52:47.540266 1810 catalog_manager.cc:5654] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:47.541486 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b115d737d7fc483d9f5a1f040b511bae (table default.Table_6d94053661f142b79292e19752df02df [id=c5be59a2c3934b3789641ceeb2d4d6e2]). Expected version 2 got 0
I20251211 20:52:47.541734 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet da81bf8854a24954ba4548185812295d (table default.Table_18262e22c26d4d66ba4c6134fbde0368 [id=da8cf2b99b42409091484a9513623d6f]). Expected version 1 got 0
I20251211 20:52:47.542297 1810 catalog_manager.cc:5654] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:47.548887 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79d92736bb1c46d2bfa5678b2d396ec2" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:47.543192 1802 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46568 (request call id 71) took 2260 ms. Trace:
I20251211 20:52:47.548625 1166 tablet_bootstrap.cc:492] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:47.551823 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 3e6f62c2deff4c11b137fc0cba5ee234 (table default.Table_51db250005ed4b3d947f32acd1d62563 [id=fdfaa0b24d2f4736aee695ed228212cf]). Expected version 1 got 0
I20251211 20:52:47.551764 1802 rpcz_store.cc:276] 1211 20:52:45.282865 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:47.390300 (+2107435us) service_pool.cc:225] Handling call
1211 20:52:47.531963 (+141663us) catalog_manager.cc:2101] Acquired catalog manager lock
1211 20:52:47.532494 (+ 531us) catalog_manager.cc:2147] Created new table and tablet info
1211 20:52:47.541128 (+ 8634us) catalog_manager.cc:2207] Wrote table and tablets to system table
1211 20:52:47.543043 (+ 1915us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
1211 20:52:47.543143 (+ 100us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:47.535225 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:47.535359 (+ 134us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:47.535367 (+ 8us) write_op.cc:435] Acquired schema lock
1211 20:52:47.535369 (+ 2us) tablet.cc:660] Decoding operations
1211 20:52:47.535465 (+ 96us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:47.535479 (+ 14us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:47.535484 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:47.535609 (+ 125us) tablet.cc:699] Row locks acquired
1211 20:52:47.535612 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:47.535650 (+ 38us) write_op.cc:270] Start()
1211 20:52:47.535685 (+ 35us) write_op.cc:276] Timestamp: P: 1765486367535630 usec, L: 0
1211 20:52:47.535689 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:47.535896 (+ 207us) log.cc:844] Serialized 433 byte log entry
1211 20:52:47.539754 (+ 3858us) op_driver.cc:464] REPLICATION: finished
1211 20:52:47.539888 (+ 134us) write_op.cc:301] APPLY: starting
1211 20:52:47.539904 (+ 16us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:47.539952 (+ 48us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:47.539977 (+ 25us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:47.540096 (+ 119us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:47.540135 (+ 39us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1211 20:52:47.540141 (+ 6us) write_op.cc:312] APPLY: finished
1211 20:52:47.540229 (+ 88us) log.cc:844] Serialized 29 byte log entry
1211 20:52:47.540300 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.540323 (+ 23us) write_op.cc:454] Released schema lock
1211 20:52:47.540330 (+ 7us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:47.540408 (+ 78us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.540412 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":84,"apply.run_cpu_time_us":511,"apply.run_wall_time_us":511,"num_ops":2,"prepare.queue_time_us":1064,"prepare.run_cpu_time_us":788,"prepare.run_wall_time_us":789,"raft.queue_time_us":900,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":234,"replication_time_us":4032}]]}
I20251211 20:52:47.552587 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet e4647105ab57454d883ebfcde7bde891 (table default.Table_c817dec9a7674f699b3366fc302ead57 [id=d555d1fc6aef4a21b4d3f2804264d030]). Expected version 1 got 0
I20251211 20:52:47.553504 1166 tablet_bootstrap.cc:654] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:47.554268 881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db0504131ddd4e3c88b2b44bd2bcb920. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.553169 1810 catalog_manager.cc:5654] T e484d499af1b4947adb36c32f8ea5704 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: HEALTHY } } }
I20251211 20:52:47.554832 1810 catalog_manager.cc:5654] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:47.555176 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet ee7b93b0b87541d9af6eb9e68b4cea41 (table default.Table_b6e6aea68c88426fb01353146c06b6ac [id=a4f984b4263a4bc38f6248628f643ae3]). Expected version 3 got 0
I20251211 20:52:47.555395 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet df457783b2534b3ea23bb1fd529ce268 (table default.Table_6e9f3ca164384cc8ac4501deb3f0c6aa [id=342c2469d6e74e3e9217236e3aa863d6]). Expected version 1 got 0
I20251211 20:52:47.555549 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 5453f68789fd4e89b69e98860050865b (table default.Table_b7bf8c5ad5304175a1f2f6a03887f725 [id=50d88b663a4d4891b113aee9c5906699]). Expected version 1 got 0
W20251211 20:52:47.553910 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 9c5c937cc7db4ed18fe03f4a8a99a66a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9c5c937cc7db4ed18fe03f4a8a99a66a already in progress: creating tablet
W20251211 20:52:47.556165 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 5f2c14c7a972414e8328e0b388bde0a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5f2c14c7a972414e8328e0b388bde0a4 already in progress: creating tablet
W20251211 20:52:47.556373 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 87c4ec07e82d4bf79b4eeefe732674ff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 87c4ec07e82d4bf79b4eeefe732674ff already in progress: creating tablet
I20251211 20:52:47.553725 1802 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_1f238c6ab5774766965f06b677a6f5c8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:47.557603 1802 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1f238c6ab5774766965f06b677a6f5c8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:47.558076 2759 raft_consensus.cc:2749] T c74427cb9d6f4a679e43eae796ff8c3f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:47.559095 1810 catalog_manager.cc:5654] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:47.559641 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 46cb1d16a0e94b3a833f32c635213fd2 (table default.Table_3482fbfc5b144a7198a7d01aa003cb6b [id=1bdaf5ce0f6b4b99a038f9f9efb828f6]). Expected version 1 got 0
I20251211 20:52:47.559998 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 834b84a557b04e78b5d2c8c84520a644 (table default.Table_909bbe6da1a648d68199fb27424e4342 [id=cc99459a5039424dbc192ff748b6708b]). Expected version 2 got 0
I20251211 20:52:47.560563 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 7be44df39a8f4d6da793479d425fa684 (table default.Table_f6a10556ff104db5bd5ecc9fc380e228 [id=f75813add54e46189ff1bbed82d83ba7]). Expected version 1 got 0
I20251211 20:52:47.561030 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet fd43a27db69044b68826cda87e16a086 (table default.Table_a8ebfd7d7a7f4f6cae8b6b02b99b80b0 [id=f1d32002c612403a82bdff5baf209b9e]). Expected version 1 got 0
I20251211 20:52:47.561825 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 4fe5c534b572491c911aae01eaa79bae (table default.Table_c1f9dca5c2144d50b67681dcc6800e77 [id=14fa8debfe5443ce81587f02dfa15847]). Expected version 2 got 0
I20251211 20:52:47.566669 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 51d12723f5194afb97d2c05fcbaf3776 (table default.Table_823505200e3142dbbe8cd56ac78daf55 [id=ec0bb10933fa4529bef3580bee6409e7]). Expected version 2 got 0
I20251211 20:52:47.567437 1802 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46568 (request call id 72) took 2284 ms. Trace:
I20251211 20:52:47.569900 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 171d40a8a9124f9bb1b38797fef8e5d5 (table default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5]). Expected version 2 got 0
I20251211 20:52:47.570297 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 2e800d8cc5ea4535a065d7d9243326b4 (table default.Table_083c478bedec45a5bf5203e4ed333860 [id=8baf0ea87d734d58896adbd0d6ef99c2]). Expected version 1 got 0
I20251211 20:52:47.570551 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet d43210ec61af423ebde624397dea9f2e (table default.Table_af0df4d1ee144455a6041e5a6a0b9b80 [id=7c5b2e33fa1c4b12b3941577360908a8]). Expected version 3 got 0
I20251211 20:52:47.571293 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 1c4b2c22bd4c456eb77af45576de466f (table default.Table_e3a99b043e4f420a93ebfee6aa87f2ec [id=3e7d6cefcb7d412191c6540e9ad77c2d]). Expected version 2 got 0
I20251211 20:52:47.569952 1802 rpcz_store.cc:276] 1211 20:52:45.282964 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:47.553491 (+2270527us) service_pool.cc:225] Handling call
1211 20:52:47.557798 (+ 4307us) catalog_manager.cc:2101] Acquired catalog manager lock
1211 20:52:47.558276 (+ 478us) catalog_manager.cc:2147] Created new table and tablet info
1211 20:52:47.567175 (+ 8899us) catalog_manager.cc:2207] Wrote table and tablets to system table
1211 20:52:47.567278 (+ 103us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
1211 20:52:47.567367 (+ 89us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:47.558869 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:47.558977 (+ 108us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:47.558983 (+ 6us) write_op.cc:435] Acquired schema lock
1211 20:52:47.558986 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:47.559087 (+ 101us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:47.559102 (+ 15us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:47.559106 (+ 4us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:47.559194 (+ 88us) tablet.cc:699] Row locks acquired
1211 20:52:47.559197 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:47.559238 (+ 41us) write_op.cc:270] Start()
1211 20:52:47.559272 (+ 34us) write_op.cc:276] Timestamp: P: 1765486367559215 usec, L: 0
1211 20:52:47.559276 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:47.559479 (+ 203us) log.cc:844] Serialized 433 byte log entry
1211 20:52:47.565843 (+ 6364us) op_driver.cc:464] REPLICATION: finished
1211 20:52:47.565982 (+ 139us) write_op.cc:301] APPLY: starting
1211 20:52:47.565999 (+ 17us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:47.566045 (+ 46us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:47.566083 (+ 38us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:47.566192 (+ 109us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:47.566228 (+ 36us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1211 20:52:47.566236 (+ 8us) write_op.cc:312] APPLY: finished
1211 20:52:47.566321 (+ 85us) log.cc:844] Serialized 29 byte log entry
1211 20:52:47.566389 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.566413 (+ 24us) write_op.cc:454] Released schema lock
1211 20:52:47.566420 (+ 7us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:47.566495 (+ 75us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.566499 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":90,"apply.run_cpu_time_us":502,"apply.run_wall_time_us":502,"num_ops":2,"prepare.queue_time_us":48,"prepare.run_cpu_time_us":762,"prepare.run_wall_time_us":762,"raft.queue_time_us":998,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":252,"replication_time_us":6532}]]}
I20251211 20:52:47.572547 1802 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_6b69717880834c989053981a48d5c683"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:47.573256 1802 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6b69717880834c989053981a48d5c683 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:47.574551 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet f26ba8f4bd3a4a1f8fa2ba6e80d543f8 (table default.Table_71e9c158dc22435a8fbb02aecca8919d [id=b966d0c0b49d4023894950c5dc06f678]). Expected version 2 got 0
I20251211 20:52:47.575016 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet dfd194cf00f549c8937cb6ec40987240 (table default.Table_8db9acb9d35149fdabcb9a1d24061b46 [id=d919c6132b3c466e8a1404f981b003b8]). Expected version 1 got 0
I20251211 20:52:47.575959 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00c4326b662541b8b33242b2a82235a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.576257 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet bc4e7362e6d045da81bf39f4135f500e (table default.Table_328b7dd76e9e407c83b47a0f51cfcce5 [id=9e7c08f833e14a90b17a9b9f60b702de]). Expected version 1 got 0
I20251211 20:52:47.576766 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ca3d6cf418e4e718f6bb24c8582d512. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.573988 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 87c4ec07e82d4bf79b4eeefe732674ff with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:47.576783 1810 catalog_manager.cc:5654] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:47.579803 1810 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet ce1799e6604f4ed7875ddedecab6f327 (table default.Table_5db1ac98e36d45b1b01889ee275ad83d [id=a6849726434d41d29dce12f47066a685]). Expected version 1 got 0
I20251211 20:52:47.582149 1802 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46568 (request call id 73) took 2299 ms. Trace:
I20251211 20:52:47.582262 1802 rpcz_store.cc:276] 1211 20:52:45.283049 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:47.572314 (+2289265us) service_pool.cc:225] Handling call
1211 20:52:47.573404 (+ 1090us) catalog_manager.cc:2101] Acquired catalog manager lock
1211 20:52:47.573835 (+ 431us) catalog_manager.cc:2147] Created new table and tablet info
1211 20:52:47.581894 (+ 8059us) catalog_manager.cc:2207] Wrote table and tablets to system table
1211 20:52:47.581994 (+ 100us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
1211 20:52:47.582084 (+ 90us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:47.574383 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:47.574492 (+ 109us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:47.574498 (+ 6us) write_op.cc:435] Acquired schema lock
1211 20:52:47.574501 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:47.574620 (+ 119us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:47.574634 (+ 14us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:47.574638 (+ 4us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:47.574714 (+ 76us) tablet.cc:699] Row locks acquired
1211 20:52:47.574717 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:47.574756 (+ 39us) write_op.cc:270] Start()
1211 20:52:47.574795 (+ 39us) write_op.cc:276] Timestamp: P: 1765486367574732 usec, L: 0
1211 20:52:47.574798 (+ 3us) op_driver.cc:348] REPLICATION: starting
1211 20:52:47.574996 (+ 198us) log.cc:844] Serialized 433 byte log entry
1211 20:52:47.580564 (+ 5568us) op_driver.cc:464] REPLICATION: finished
1211 20:52:47.580702 (+ 138us) write_op.cc:301] APPLY: starting
1211 20:52:47.580717 (+ 15us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:47.580781 (+ 64us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:47.580813 (+ 32us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:47.580918 (+ 105us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:47.580954 (+ 36us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1211 20:52:47.580961 (+ 7us) write_op.cc:312] APPLY: finished
1211 20:52:47.581042 (+ 81us) log.cc:844] Serialized 29 byte log entry
1211 20:52:47.581111 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.581136 (+ 25us) write_op.cc:454] Released schema lock
1211 20:52:47.581143 (+ 7us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:47.581222 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.581226 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":81,"apply.run_cpu_time_us":507,"apply.run_wall_time_us":507,"num_ops":2,"prepare.queue_time_us":62,"prepare.run_cpu_time_us":731,"prepare.run_wall_time_us":731,"raft.queue_time_us":806,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":258,"replication_time_us":5736}]]}
I20251211 20:52:47.583079 1802 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_511715bb615a453fa603163df3152002"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:47.583666 1802 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_511715bb615a453fa603163df3152002 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:47.585731 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 5f2c14c7a972414e8328e0b388bde0a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:47.607955 2429 consensus_queue.cc:1048] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20251211 20:52:47.608176 2736 consensus_queue.cc:1048] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251211 20:52:47.613920 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4f5011512dd4b31853738599384c293. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.619256 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68fe7dfddec04ff79199661dc41a6330. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.648986 2808 raft_consensus.cc:493] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.649255 2808 raft_consensus.cc:515] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.649958 2808 leader_election.cc:290] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.658268 2970 raft_consensus.cc:493] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.658573 2970 raft_consensus.cc:515] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.659299 2970 leader_election.cc:290] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.676008 1802 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46568 (request call id 74) took 2392 ms. Trace:
I20251211 20:52:47.676169 1802 rpcz_store.cc:276] 1211 20:52:45.283135 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:47.582832 (+2299697us) service_pool.cc:225] Handling call
1211 20:52:47.583799 (+ 967us) catalog_manager.cc:2101] Acquired catalog manager lock
1211 20:52:47.584233 (+ 434us) catalog_manager.cc:2147] Created new table and tablet info
1211 20:52:47.675681 (+ 91448us) catalog_manager.cc:2207] Wrote table and tablets to system table
1211 20:52:47.675787 (+ 106us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
1211 20:52:47.675909 (+ 122us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:47.584760 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:47.584862 (+ 102us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:47.584868 (+ 6us) write_op.cc:435] Acquired schema lock
1211 20:52:47.584871 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:47.584955 (+ 84us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:47.584965 (+ 10us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:47.584970 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:47.585038 (+ 68us) tablet.cc:699] Row locks acquired
1211 20:52:47.585041 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:47.585073 (+ 32us) write_op.cc:270] Start()
1211 20:52:47.585102 (+ 29us) write_op.cc:276] Timestamp: P: 1765486367585050 usec, L: 0
1211 20:52:47.585106 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:47.585262 (+ 156us) log.cc:844] Serialized 433 byte log entry
1211 20:52:47.670078 (+ 84816us) op_driver.cc:464] REPLICATION: finished
1211 20:52:47.671034 (+ 956us) write_op.cc:301] APPLY: starting
1211 20:52:47.671045 (+ 11us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:47.671086 (+ 41us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:47.671112 (+ 26us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:47.671245 (+ 133us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:47.671276 (+ 31us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1211 20:52:47.671284 (+ 8us) write_op.cc:312] APPLY: finished
1211 20:52:47.671357 (+ 73us) log.cc:844] Serialized 29 byte log entry
1211 20:52:47.671408 (+ 51us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.671425 (+ 17us) write_op.cc:454] Released schema lock
1211 20:52:47.671430 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":928,"apply.run_cpu_time_us":450,"apply.run_wall_time_us":450,"num_ops":2,"prepare.queue_time_us":50,"prepare.run_cpu_time_us":600,"prepare.run_wall_time_us":601,"raft.queue_time_us":305,"raft.run_cpu_time_us":8,"raft.run_wall_time_us":9,"replication_time_us":84958,"threads_started":1}]]}
I20251211 20:52:47.677112 1160 tablet_bootstrap.cc:492] T 5453f68789fd4e89b69e98860050865b P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:47.677248 1160 ts_tablet_manager.cc:1403] T 5453f68789fd4e89b69e98860050865b P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.374s user 0.002s sys 0.010s
I20251211 20:52:47.678014 1160 raft_consensus.cc:359] T 5453f68789fd4e89b69e98860050865b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.678201 1160 raft_consensus.cc:385] T 5453f68789fd4e89b69e98860050865b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:47.678259 1160 raft_consensus.cc:740] T 5453f68789fd4e89b69e98860050865b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:47.678421 1160 consensus_queue.cc:260] T 5453f68789fd4e89b69e98860050865b P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.678830 1160 ts_tablet_manager.cc:1434] T 5453f68789fd4e89b69e98860050865b P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:47.679273 1160 tablet_bootstrap.cc:492] T 3e6f62c2deff4c11b137fc0cba5ee234 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:47.680961 1160 tablet_bootstrap.cc:654] T 3e6f62c2deff4c11b137fc0cba5ee234 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:47.695922 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "efe4d6bb07374294b8ab50a9fe605f87" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:47.696337 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c5c937cc7db4ed18fe03f4a8a99a66a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:47.699018 2973 raft_consensus.cc:493] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.699990 1802 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_6973147f8fd7408b9cca4506330b8105"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:47.700667 1802 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6973147f8fd7408b9cca4506330b8105 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251211 20:52:47.704228 847 leader_election.cc:343] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: a2058b67ddf1413db338bfa2f64e7a77
I20251211 20:52:47.704356 847 leader_election.cc:304] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:47.705523 2964 raft_consensus.cc:493] T f22e4d34cde24b37a025c1b5a1046b1b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.705672 2964 raft_consensus.cc:515] T f22e4d34cde24b37a025c1b5a1046b1b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.705749 2973 raft_consensus.cc:515] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.706899 2692 raft_consensus.cc:2749] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:47.710624 710 leader_election.cc:304] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:47.710889 2807 raft_consensus.cc:2804] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:47.711030 2807 raft_consensus.cc:493] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:47.711146 2807 raft_consensus.cc:3060] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:47.713778 2807 raft_consensus.cc:515] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.716487 2973 leader_election.cc:290] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:47.695948 982 leader_election.cc:343] T 1e13965b6361488eaeb7dd13b467ca65 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:47.718050 2807 leader_election.cc:290] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.718508 1802 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46568 (request call id 75) took 2417 ms. Trace:
I20251211 20:52:47.718631 1802 rpcz_store.cc:276] 1211 20:52:45.300932 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:47.676835 (+2375903us) service_pool.cc:225] Handling call
1211 20:52:47.699907 (+ 23072us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x616000040f54. stack: 00007ff715c18ee6 00007ff715c18ca0 00007ff714bf43ad 0000000000984f94 0000000000984f5d 00007ff724f1f6bd 00007ff724f1f7b3 00007ff721ae6521 00007ff7297a596d 00007ff7299321c3 00007ff72043c7c6 00007ff72043c4f5 00007ff71cbb856e 00007ff71cbb744e 00007ff71cbbd685 00007ff71cbbdfaf
1211 20:52:47.700783 (+ 876us) catalog_manager.cc:2101] Acquired catalog manager lock
1211 20:52:47.701268 (+ 485us) catalog_manager.cc:2147] Created new table and tablet info
1211 20:52:47.718224 (+ 16956us) catalog_manager.cc:2207] Wrote table and tablets to system table
1211 20:52:47.718337 (+ 113us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
1211 20:52:47.718445 (+ 108us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:47.701944 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:47.702075 (+ 131us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:47.702083 (+ 8us) write_op.cc:435] Acquired schema lock
1211 20:52:47.702086 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:47.702184 (+ 98us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:47.702200 (+ 16us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:47.702206 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:47.702289 (+ 83us) tablet.cc:699] Row locks acquired
1211 20:52:47.702292 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:47.702329 (+ 37us) write_op.cc:270] Start()
1211 20:52:47.702367 (+ 38us) write_op.cc:276] Timestamp: P: 1765486367702291 usec, L: 0
1211 20:52:47.702371 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:47.702619 (+ 248us) log.cc:844] Serialized 433 byte log entry
1211 20:52:47.707009 (+ 4390us) op_driver.cc:464] REPLICATION: finished
1211 20:52:47.714366 (+ 7357us) write_op.cc:301] APPLY: starting
1211 20:52:47.714382 (+ 16us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:47.714438 (+ 56us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:47.714501 (+ 63us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:47.714688 (+ 187us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:47.714724 (+ 36us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1211 20:52:47.714733 (+ 9us) write_op.cc:312] APPLY: finished
1211 20:52:47.714829 (+ 96us) log.cc:844] Serialized 29 byte log entry
1211 20:52:47.716815 (+ 1986us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.716842 (+ 27us) write_op.cc:454] Released schema lock
1211 20:52:47.716850 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":52137728,"child_traces":[["op",{"apply.queue_time_us":7266,"num_ops":2,"prepare.queue_time_us":60,"prepare.run_cpu_time_us":793,"prepare.run_wall_time_us":792,"raft.queue_time_us":1652,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":510,"replication_time_us":4602}]]}
W20251211 20:52:47.720134 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 87c4ec07e82d4bf79b4eeefe732674ff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 87c4ec07e82d4bf79b4eeefe732674ff already in progress: creating tablet
W20251211 20:52:47.720377 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 5f2c14c7a972414e8328e0b388bde0a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5f2c14c7a972414e8328e0b388bde0a4 already in progress: creating tablet
I20251211 20:52:47.720960 1166 tablet_bootstrap.cc:492] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
W20251211 20:52:47.721432 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 9c5c937cc7db4ed18fe03f4a8a99a66a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9c5c937cc7db4ed18fe03f4a8a99a66a already in progress: creating tablet
I20251211 20:52:47.722020 1802 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_0d0d81ce88d44d8aaeb2cc8ad423568e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:47.722651 1802 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0d0d81ce88d44d8aaeb2cc8ad423568e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:47.722576 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5df87ea7d28340da84ef141c92efc710" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:47.731194 1802 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46568 (request call id 76) took 2335 ms. Trace:
I20251211 20:52:47.731717 1802 rpcz_store.cc:276] 1211 20:52:45.395256 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:47.719237 (+2323981us) service_pool.cc:225] Handling call
1211 20:52:47.721948 (+ 2711us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x616000040f54. stack: 00007ff715c18ee6 00007ff715c18ca0 00007ff714bf43ad 0000000000984f94 0000000000984f5d 00007ff724f1f6bd 00007ff724f1f7b3 00007ff721ae6521 00007ff7297a596d 00007ff7299321c3 00007ff72043c7c6 00007ff72043c4f5 00007ff71cbb856e 00007ff71cbb744e 00007ff71cbbd685 00007ff71cbbdfaf
1211 20:52:47.722746 (+ 798us) catalog_manager.cc:2101] Acquired catalog manager lock
1211 20:52:47.723193 (+ 447us) catalog_manager.cc:2147] Created new table and tablet info
1211 20:52:47.730904 (+ 7711us) catalog_manager.cc:2207] Wrote table and tablets to system table
1211 20:52:47.731023 (+ 119us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
1211 20:52:47.731125 (+ 102us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:47.723989 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:47.724119 (+ 130us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:47.724127 (+ 8us) write_op.cc:435] Acquired schema lock
1211 20:52:47.724130 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:47.724233 (+ 103us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:47.724248 (+ 15us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:47.724253 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:47.724365 (+ 112us) tablet.cc:699] Row locks acquired
1211 20:52:47.724368 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:47.724409 (+ 41us) write_op.cc:270] Start()
1211 20:52:47.724447 (+ 38us) write_op.cc:276] Timestamp: P: 1765486367724371 usec, L: 0
1211 20:52:47.724452 (+ 5us) op_driver.cc:348] REPLICATION: starting
1211 20:52:47.724662 (+ 210us) log.cc:844] Serialized 433 byte log entry
1211 20:52:47.729231 (+ 4569us) op_driver.cc:464] REPLICATION: finished
1211 20:52:47.730155 (+ 924us) write_op.cc:301] APPLY: starting
1211 20:52:47.730175 (+ 20us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:47.730227 (+ 52us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:47.730254 (+ 27us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:47.730371 (+ 117us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:47.730410 (+ 39us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1211 20:52:47.730417 (+ 7us) write_op.cc:312] APPLY: finished
1211 20:52:47.730507 (+ 90us) log.cc:844] Serialized 29 byte log entry
1211 20:52:47.730790 (+ 283us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.730815 (+ 25us) write_op.cc:454] Released schema lock
1211 20:52:47.730823 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":5475712,"child_traces":[["op",{"apply.queue_time_us":850,"mutex_wait_us":12,"num_ops":2,"prepare.queue_time_us":230,"prepare.run_cpu_time_us":804,"prepare.run_wall_time_us":803,"raft.queue_time_us":111,"raft.run_cpu_time_us":374,"raft.run_wall_time_us":373,"replication_time_us":4744}]]}
I20251211 20:52:47.733340 1802 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_24b29712550340f2bea51b04f527c4f3" } new_table_name: "default.Table_efc22dc69ad0405ebf59fda961e23330" modify_external_catalogs: true
I20251211 20:52:47.706255 2964 leader_election.cc:290] T f22e4d34cde24b37a025c1b5a1046b1b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.737109 824 heartbeater.cc:499] Master 127.0.104.61:40749 was elected leader, sending a full tablet report...
I20251211 20:52:47.744755 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f22e4d34cde24b37a025c1b5a1046b1b" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:47.747255 2964 raft_consensus.cc:493] T 5453f68789fd4e89b69e98860050865b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.747429 2964 raft_consensus.cc:515] T 5453f68789fd4e89b69e98860050865b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.748037 2964 leader_election.cc:290] T 5453f68789fd4e89b69e98860050865b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:47.751946 982 leader_election.cc:343] T f2ae084982f44fdf8bfecdd2bc4c01b9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:47.753671 982 leader_election.cc:343] T 969a8587019c4202b6e705d126f83d44 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:47.753862 982 leader_election.cc:343] T de508d79f30f489cb745401f53115165 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:47.753993 982 leader_election.cc:343] T 79d92736bb1c46d2bfa5678b2d396ec2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:47.754110 982 leader_election.cc:343] T 5df87ea7d28340da84ef141c92efc710 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:47.754388 1166 ts_tablet_manager.cc:1403] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.285s user 0.003s sys 0.005s
I20251211 20:52:47.755672 1166 raft_consensus.cc:359] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:47.755971 1168 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.594s user 0.032s sys 0.061s
W20251211 20:52:47.756070 1168 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.594s user 0.032s sys 0.061s
W20251211 20:52:47.765990 847 leader_election.cc:343] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: b529b38d754f46ba8cb60b19e9705256
I20251211 20:52:47.767663 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5453f68789fd4e89b69e98860050865b" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:47.770222 1168 tablet_bootstrap.cc:492] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:47.770752 1168 ts_tablet_manager.cc:1403] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.625s user 0.034s sys 0.063s
I20251211 20:52:47.771672 1168 raft_consensus.cc:359] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.771885 1168 raft_consensus.cc:385] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:47.771943 1168 raft_consensus.cc:740] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:47.772156 1168 consensus_queue.cc:260] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.772708 1168 ts_tablet_manager.cc:1434] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:47.772660 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet aab26df748724fe5b2bb7fad9e3bfc0c (table default.Table_22bed4b4e9e24a90818b19c6f5c91e1f [id=df6255d66f334ef2b4406f4b845cf980]). Expected version 1 got 0
I20251211 20:52:47.773016 1168 tablet_bootstrap.cc:492] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:47.774681 1168 tablet_bootstrap.cc:654] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:47.775323 1160 tablet_bootstrap.cc:492] T 3e6f62c2deff4c11b137fc0cba5ee234 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:47.776178 1160 ts_tablet_manager.cc:1403] T 3e6f62c2deff4c11b137fc0cba5ee234 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.097s user 0.006s sys 0.001s
I20251211 20:52:47.776929 1160 raft_consensus.cc:359] T 3e6f62c2deff4c11b137fc0cba5ee234 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.777213 1160 raft_consensus.cc:385] T 3e6f62c2deff4c11b137fc0cba5ee234 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:47.777302 1160 raft_consensus.cc:740] T 3e6f62c2deff4c11b137fc0cba5ee234 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:47.777561 1160 consensus_queue.cc:260] T 3e6f62c2deff4c11b137fc0cba5ee234 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.757948 1037 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "497c10742f01476db885bd70c4d2c491" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:47.778306 1166 raft_consensus.cc:385] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:47.778381 1037 raft_consensus.cc:2468] T 497c10742f01476db885bd70c4d2c491 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:47.778385 1166 raft_consensus.cc:740] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:47.758383 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "06b8f8e46e524545aeab6c931efaf116" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:47.778684 1045 raft_consensus.cc:2468] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:47.778632 1166 consensus_queue.cc:260] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.761328 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd43a27db69044b68826cda87e16a086" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:47.779034 1048 raft_consensus.cc:2468] T fd43a27db69044b68826cda87e16a086 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:47.779124 1166 ts_tablet_manager.cc:1434] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.025s user 0.001s sys 0.000s
I20251211 20:52:47.780514 1166 tablet_bootstrap.cc:492] T f9a3fdf17f274c15ac0bdfa9c174f649 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:47.773001 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b52377c8ffd741a598e5328a00d876fe (table default.Table_686a26915b42421380037144900dd97c [id=8a1b0c9d641b4a3c83d1ca48576d110d]). Expected version 1 got 0
I20251211 20:52:47.781009 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet c7a2610f20df4c1ca81eae2fc7dba6bb (table default.Table_59072235d93b41c0b5cd5e3debcb8ddf [id=9a15a8a075af4969ae2c8b3be1bfed78]). Expected version 1 got 0
I20251211 20:52:47.781519 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 0302cc8613934c43a0650f081cf98797 (table default.Table_bf827b216f1b43c196923dc1828ff0d0 [id=ed371e073250439fa44330c7101e9387]). Expected version 1 got 0
I20251211 20:52:47.781859 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 1f4eb1a174c648c897ae2b7612e44585 (table default.Table_e70873b71ba24d7eae548e2acfd243b3 [id=87adcc552a704d56bd2fe812fb3b791f]). Expected version 1 got 0
I20251211 20:52:47.782176 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet a7afcce522494c9ca1d80e6eea913e6e (table default.Table_cb6a4bac7b694f33a37de2ee78c91ca0 [id=8940f18527c249a9baefc170ed82f45a]). Expected version 1 got 0
I20251211 20:52:47.782405 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 84bb3583857848459e0e865283de623d (table default.Table_92281c5fcfee4355a79215ffedeee588 [id=6e5fe0a78e4b48f98dac028802f1ddf4]). Expected version 4 got 0
I20251211 20:52:47.782712 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 7a299ee03d0e48aba7c4f71e3e0f381a (table default.Table_6402a801d48e4a53827b1b62b6cdaba1 [id=6876565d3a9c48ce841b830cf75139ff]). Expected version 3 got 0
I20251211 20:52:47.783105 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 5b3e8ec3faf445ac9b746e65dc4dbfe0 (table default.Table_7fe80ff5483a4b0292091119980f6639 [id=3c3462a0c09748cea0d2ffb544ae2f13]). Expected version 1 got 0
I20251211 20:52:47.783391 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 7ffa06c4a7ee48b0b337a27aff090592 (table default.Table_647731023b044ebb8954f7fdcc3faf97 [id=4b91abd6e9e94a8a941b33e62e23c083]). Expected version 1 got 0
I20251211 20:52:47.783660 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b81a4cef27164824b52695039744db15 (table default.Table_defdeab28749411590cf75b5251f604b [id=11352566d1744193a40a8bb605aa3912]). Expected version 1 got 0
I20251211 20:52:47.783895 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 7023afec769c4ca3b2242a225004d482 (table default.Table_d5276e3cc46d4331b36a8878680fc55e [id=3811f8235b2f44cca283401302d613d4]). Expected version 1 got 0
I20251211 20:52:47.784132 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 72af55c8045b4f8f8e116a5782794d81 (table default.Table_2cff6fab088c4d939b13b504c133cc12 [id=b6c4b72063b3428593d5650a160dc66b]). Expected version 1 got 0
I20251211 20:52:47.784628 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 4cf0cde801bf44449e539d2799b895a7 (table default.Table_907c596bdddc4edca25d14cbacebb23f [id=321235280d1d4fa0bae5cc0333860f85]). Expected version 1 got 0
I20251211 20:52:47.784861 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 7b9f569554fb4836ac2b62542c3ecf7e (table default.Table_1a73d567e8914a83974e3e8f5f58be0b [id=6ef691749df24251a7a7c5919c219754]). Expected version 1 got 0
I20251211 20:52:47.785080 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3b2c6064c5eb4c0f8979c485f5d85a75 (table default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f]). Expected version 1 got 0
I20251211 20:52:47.785351 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 59acb8474e114bf7ae7f1f043d611459 (table default.Table_c7ed8c0c309c40a3a371ffa130b47cd2 [id=ed1d0670426945aba888b818ad657225]). Expected version 1 got 0
I20251211 20:52:47.785831 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3c865c99e9314f5f91658b9126da4d39 (table default.Table_0d7df3fba38849a49b65d8290812b95c [id=7f966d5cdad3423bb770414802a3cf7b]). Expected version 1 got 0
I20251211 20:52:47.786391 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet a3b1aadf17ea4bf0a896d23cd543fca7 (table default.Table_d18401707c88493abdf90d2c713432ae [id=a07662f8dc3f4178be5dfcc1cdc85837]). Expected version 1 got 0
I20251211 20:52:47.786840 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 9f8aa28c1c9b4fc58dfc128952c180ba (table default.Table_bf7d402a36be44cfb6b79041906d26a1 [id=0327e686e7ed4475a3e8cda8721a3a3a]). Expected version 1 got 0
I20251211 20:52:47.787273 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 8b793241ee3b42b1a38e9f2f05e1173b (table default.Table_7506ffecc2b64648a9ba736e5298ddf4 [id=279f39d718ba45318f52f95e8aa13652]). Expected version 2 got 0
I20251211 20:52:47.789769 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet ab4e7a9062134e489d6f9abe6ff26fb4 (table default.Table_4cbc8841b276406cb4434629653a5ce3 [id=b958532650374d609199cab9c07bb64b]). Expected version 2 got 0
I20251211 20:52:47.790110 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 6b7dd8232d974a43b99ec9b28075dc8d (table default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]). Expected version 2 got 0
I20251211 20:52:47.790479 1808 catalog_manager.cc:5654] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3). New cstate: current_term: 1 leader_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: HEALTHY } } }
I20251211 20:52:47.791105 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 5453f68789fd4e89b69e98860050865b (table default.Table_b7bf8c5ad5304175a1f2f6a03887f725 [id=50d88b663a4d4891b113aee9c5906699]). Expected version 1 got 0
I20251211 20:52:47.797677 1160 ts_tablet_manager.cc:1434] T 3e6f62c2deff4c11b137fc0cba5ee234 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.021s user 0.001s sys 0.001s
I20251211 20:52:47.798015 1160 tablet_bootstrap.cc:492] T 6b7dd8232d974a43b99ec9b28075dc8d P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:47.798296 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 645b08b33676420c96e880de3d460829 (table default.Table_e02933dd91ce4fe7a40d1c3b2f9b2d4b [id=6d42f260246e420d8818b3841926302b]). Expected version 2 got 0
I20251211 20:52:47.798645 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 46cb1d16a0e94b3a833f32c635213fd2 (table default.Table_3482fbfc5b144a7198a7d01aa003cb6b [id=1bdaf5ce0f6b4b99a038f9f9efb828f6]). Expected version 1 got 0
I20251211 20:52:47.799124 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 7ca98b43863a47c091baae0a6ce65630 (table default.Table_f0a70914acc94231a4e84155c3e4d13f [id=56c070ad80624c698ebe2e01b4929b6e]). Expected version 1 got 0
I20251211 20:52:47.799356 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet e3f711f02af843adb4f5c3c750924e33 (table default.Table_34bd5155aa3b4ef7818022ba9346c074 [id=83a61d23e80946439a14096ab3ab1bd9]). Expected version 2 got 0
I20251211 20:52:47.800094 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet aa50ddb2789b4be3bfa9aedef9287109 (table default.Table_44f971096d1e41f480e594985ffd5c17 [id=d4e1349374a346a5b8be180f9bdc329a]). Expected version 1 got 0
I20251211 20:52:47.800402 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 497c10742f01476db885bd70c4d2c491 (table default.Table_18b21232dfa74378b27defe672bb2574 [id=602d7eee703948d99a0edcb7cac0651b]). Expected version 1 got 0
I20251211 20:52:47.800616 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3003ef71a3554c71a3b24cdfb9c293bc (table default.Table_92efd469dd324f7d886e2cbc90ece61c [id=6e7fb3271e0c48c0bb9d56fe4fccee3a]). Expected version 1 got 0
I20251211 20:52:47.800707 1166 tablet_bootstrap.cc:654] T f9a3fdf17f274c15ac0bdfa9c174f649 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:47.801040 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 46a2a070cf6a42fe8bc54a42eb802fbc (table default.Table_4e1f03b9d87b4fe59dbcb3a947ee299d [id=8e66ea58499b40e59294a135324aa527]). Expected version 1 got 0
I20251211 20:52:47.801403 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b2974f16034c44cf85ff85b5095cf6a0 (table default.Table_031de0f4cc7744bc9e320261f8b46696 [id=be36dabaa6d44325b4855b9f13bda847]). Expected version 1 got 0
I20251211 20:52:47.801666 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet bc6a1c84501d4e179cd2828b50fddc23 (table default.Table_8bb5b228d60e4d29a41d9ba40c33d45d [id=6ef058fada9941629e0ea4d1906abdac]). Expected version 1 got 0
I20251211 20:52:47.801901 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet d6923cfd826d495291d9ac8e0b3b44c6 (table default.Table_68ccb2052d824ccf92f7162d2961fcb1 [id=9ac9591d4f04482d926710fdf6e797c6]). Expected version 1 got 0
I20251211 20:52:47.802210 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 13269a20421240d1a98215129443248b (table default.Table_d887c953026046dc883ede774a58fd8e [id=1d8a79f716a0447cb0393f4759178598]). Expected version 1 got 0
I20251211 20:52:47.802579 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet d37a7d0d662e45079b0cfc02765d2e01 (table default.Table_014b6db9f3294658b965c701384b98bb [id=8a12299efe714617a970fc05c3bb6232]). Expected version 2 got 0
I20251211 20:52:47.803318 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 03184dcc56dd4e92adc7d0ad22aa8d8b (table default.Table_68261841754847ad8d22e86c29886144 [id=697f23a617f64762bf406b386cde7473]). Expected version 1 got 0
I20251211 20:52:47.803699 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 4c0455a5b4f44412803475d1c4852b48 (table default.Table_e00092673b0e4f09b8d7eec63e1ee034 [id=18497e06d7254441a4655ce1cdc5a9bf]). Expected version 1 got 0
I20251211 20:52:47.804018 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 16d2ff8925e342729a71c93db378d031 (table default.Table_34e6ed4f13da437ea50d349b92e83bd6 [id=4a74b9a60cbb4198b55cee630b2f4bef]). Expected version 2 got 0
I20251211 20:52:47.804359 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet f2d6c9781b4d4c37bc08703347963d5d (table default.Table_2aa7ff9186a04249abf8cd971cc0cb17 [id=b50ab5b032ba4c98938324460512468f]). Expected version 1 got 0
I20251211 20:52:47.804597 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b6cc819332e1445c96ce3ffac4861454 (table default.Table_db4c0f706e0643b39d39ffa84352826d [id=4d459c42bf584068ac3851227d80331a]). Expected version 1 got 0
I20251211 20:52:47.805119 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 5773c13034714a008c8d7dbe777b4828 (table default.Table_66eb1b021eed464f82f9cd223c88297b [id=94ff09564cd649c7a2d0e94b0f3c7b3a]). Expected version 1 got 0
I20251211 20:52:47.805411 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 18f125dd9efc401181f0644dad7aeffe (table default.Table_fa6592c2e2624d81ad21abb13ed36045 [id=9b48fc2b39db4fd5a4e5da19e34427dc]). Expected version 1 got 0
I20251211 20:52:47.805652 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 8dc4e730951144d78b77280f6cce6d0e (table default.Table_890c9027415a44ccaf95c0adede51992 [id=7bc8a2b91fe54b91b6594b371fea7bd3]). Expected version 2 got 0
I20251211 20:52:47.806619 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet f94b656e3c1844509fdae36e33b263e8 (table default.Table_ed03ad475bed4804a65abe20efef0625 [id=e23f4305669a474aafa18928f130de3a]). Expected version 2 got 0
I20251211 20:52:47.806861 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 78f77d8916d84edabbe95ce17d8df2c1 (table default.Table_e2af0e2fbd3d46d595136c7a7bf24e4c [id=ddc12ae6d8ed4587b5ef4da3ef81a055]). Expected version 1 got 0
I20251211 20:52:47.807163 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b85af79c2574443998e847bc19c53b69 (table default.Table_fa3f2f115cd043a9aa6be2b070d320cb [id=15fad5e1cf8d4ea08dae8275a29a11dd]). Expected version 1 got 0
I20251211 20:52:47.808775 2964 raft_consensus.cc:493] T 3e6f62c2deff4c11b137fc0cba5ee234 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.808941 2964 raft_consensus.cc:515] T 3e6f62c2deff4c11b137fc0cba5ee234 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.809134 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet ff4d5ec1f87247e896f9c099c532265a (table default.Table_3ac4746e17264f5a93890f3600505781 [id=475e0a1ec0344d10addbcfa42fe05b68]). Expected version 1 got 0
I20251211 20:52:47.809412 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3dfea28c44d84c68850db65a30d52ee6 (table default.Table_c504f8f3796f43b697488bd604268543 [id=2af76b9d32094a4ca656060a4478e08a]). Expected version 1 got 0
I20251211 20:52:47.809557 2964 leader_election.cc:290] T 3e6f62c2deff4c11b137fc0cba5ee234 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.809836 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet f00128ba84c245d9a0e35f26bc842b3e (table default.Table_6f8c133c7a364b93af2c83d8911a0ac3 [id=6a21c38ee7224a02b937df088494f5d8]). Expected version 1 got 0
I20251211 20:52:47.810154 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 5e47af616e1b4d2db3780c1d67cdcb47 (table default.Table_296bdf30a11541c389afe3df0b0f667a [id=052edcbde60c41449d468c8a449fc3f1]). Expected version 1 got 0
I20251211 20:52:47.810495 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 4c7b74f370b042ee8575f821c07dc085 (table default.Table_731826f26f9940b8afe425d694b642d9 [id=17dd9cea9ac844128ff9f82adb3f919f]). Expected version 1 got 0
I20251211 20:52:47.810878 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3e6f62c2deff4c11b137fc0cba5ee234" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:47.812319 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3621735fcb294e3a8c0180ba9ef86442" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:47.812493 1048 raft_consensus.cc:2468] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:47.813288 1160 tablet_bootstrap.cc:654] T 6b7dd8232d974a43b99ec9b28075dc8d P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:47.813519 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71dc8d20d93243e289330d7b8c635511" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:47.813691 1048 raft_consensus.cc:2468] T 71dc8d20d93243e289330d7b8c635511 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:47.815110 1166 tablet_bootstrap.cc:492] T f9a3fdf17f274c15ac0bdfa9c174f649 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:47.815250 1166 ts_tablet_manager.cc:1403] T f9a3fdf17f274c15ac0bdfa9c174f649 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.035s user 0.003s sys 0.001s
I20251211 20:52:47.816018 1166 raft_consensus.cc:359] T f9a3fdf17f274c15ac0bdfa9c174f649 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.816183 1166 raft_consensus.cc:385] T f9a3fdf17f274c15ac0bdfa9c174f649 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:47.816232 1166 raft_consensus.cc:740] T f9a3fdf17f274c15ac0bdfa9c174f649 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:47.816429 1166 consensus_queue.cc:260] T f9a3fdf17f274c15ac0bdfa9c174f649 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.816867 1166 ts_tablet_manager.cc:1434] T f9a3fdf17f274c15ac0bdfa9c174f649 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:47.818423 1166 tablet_bootstrap.cc:492] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:47.819425 983 leader_election.cc:343] T 79d92736bb1c46d2bfa5678b2d396ec2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:47.819559 983 leader_election.cc:304] T 79d92736bb1c46d2bfa5678b2d396ec2 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:47.819994 983 leader_election.cc:343] T 1e13965b6361488eaeb7dd13b467ca65 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:47.820083 983 leader_election.cc:304] T 1e13965b6361488eaeb7dd13b467ca65 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:47.820084 2987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:52:47.820611 2974 raft_consensus.cc:2749] T 79d92736bb1c46d2bfa5678b2d396ec2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:47.820696 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet df457783b2534b3ea23bb1fd529ce268 (table default.Table_6e9f3ca164384cc8ac4501deb3f0c6aa [id=342c2469d6e74e3e9217236e3aa863d6]). Expected version 1 got 0
I20251211 20:52:47.820854 2974 raft_consensus.cc:2749] T 1e13965b6361488eaeb7dd13b467ca65 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:47.821055 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 969a8587019c4202b6e705d126f83d44 (table default.Table_6460313002684bba93b30ad8631f0f49 [id=22347b15537d451d9927a17e8e5a202a]). Expected version 1 got 0
I20251211 20:52:47.821326 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 676faec7f3d1488f99b220f8e968e9a0 (table default.Table_ab826a3eec8e4c4aaa8d4021d87dd382 [id=a2bbfccfe3974c378521d5ef2333df86]). Expected version 2 got 0
I20251211 20:52:47.821907 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 834b84a557b04e78b5d2c8c84520a644 (table default.Table_909bbe6da1a648d68199fb27424e4342 [id=cc99459a5039424dbc192ff748b6708b]). Expected version 2 got 0
I20251211 20:52:47.821978 1168 tablet_bootstrap.cc:492] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:47.822124 1168 ts_tablet_manager.cc:1403] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.049s user 0.006s sys 0.009s
I20251211 20:52:47.822383 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet ee7b93b0b87541d9af6eb9e68b4cea41 (table default.Table_b6e6aea68c88426fb01353146c06b6ac [id=a4f984b4263a4bc38f6248628f643ae3]). Expected version 3 got 0
I20251211 20:52:47.822762 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b115d737d7fc483d9f5a1f040b511bae (table default.Table_6d94053661f142b79292e19752df02df [id=c5be59a2c3934b3789641ceeb2d4d6e2]). Expected version 2 got 0
I20251211 20:52:47.822985 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet da81bf8854a24954ba4548185812295d (table default.Table_18262e22c26d4d66ba4c6134fbde0368 [id=da8cf2b99b42409091484a9513623d6f]). Expected version 1 got 0
I20251211 20:52:47.822947 1168 raft_consensus.cc:359] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.823117 1168 raft_consensus.cc:385] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:47.823172 1168 raft_consensus.cc:740] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:47.823392 1168 consensus_queue.cc:260] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.823514 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3e6f62c2deff4c11b137fc0cba5ee234 (table default.Table_51db250005ed4b3d947f32acd1d62563 [id=fdfaa0b24d2f4736aee695ed228212cf]). Expected version 1 got 0
I20251211 20:52:47.824051 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet e4647105ab57454d883ebfcde7bde891 (table default.Table_c817dec9a7674f699b3366fc302ead57 [id=d555d1fc6aef4a21b4d3f2804264d030]). Expected version 1 got 0
I20251211 20:52:47.824364 1168 ts_tablet_manager.cc:1434] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251211 20:52:47.824723 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 33fd09ea06ee47908b85911e4ee79708 (table default.Table_3c3cd48c8fe5411e96cdf811d492925d [id=30620a01c27f456bac05fa7c02a2cb4d]). Expected version 2 got 0
W20251211 20:52:47.824843 2982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:52:47.824999 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 6212b3b4425543168837946a54d60834 (table default.Table_8c30c4151d484db0bb9e2b9ca262248d [id=2d489cd7d73e403d84bf10960e6a2dfb]). Expected version 1 got 0
I20251211 20:52:47.825412 1168 tablet_bootstrap.cc:492] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:47.825385 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 7be44df39a8f4d6da793479d425fa684 (table default.Table_f6a10556ff104db5bd5ecc9fc380e228 [id=f75813add54e46189ff1bbed82d83ba7]). Expected version 1 got 0
I20251211 20:52:47.825704 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet fd43a27db69044b68826cda87e16a086 (table default.Table_a8ebfd7d7a7f4f6cae8b6b02b99b80b0 [id=f1d32002c612403a82bdff5baf209b9e]). Expected version 1 got 0
I20251211 20:52:47.826035 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 94658fcf512a48d0a992a7e2e0f7c176 (table default.Table_c9d86e82aebb4ef0bc83789720352e38 [id=97377a8ef56c4cdebe95d15955207404]). Expected version 1 got 0
I20251211 20:52:47.826253 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet fb29f983c4524a49bdc5c0c1144ad708 (table default.Table_7809ac85fdfe4c1d9c58afb54edecb27 [id=c4b0bfff99f64888beb70d9154d2f156]). Expected version 1 got 0
I20251211 20:52:47.826866 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet e89d4d76bf64453892a4d0b55fcc3454 (table default.Table_fba99384e9f14160b3e3504f8d1d0b07 [id=01bba75f854d4c338d6c09a446dd7499]). Expected version 1 got 0
I20251211 20:52:47.827147 1168 tablet_bootstrap.cc:654] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:47.827171 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 171d40a8a9124f9bb1b38797fef8e5d5 (table default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5]). Expected version 2 got 0
I20251211 20:52:47.827548 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 4fe5c534b572491c911aae01eaa79bae (table default.Table_c1f9dca5c2144d50b67681dcc6800e77 [id=14fa8debfe5443ce81587f02dfa15847]). Expected version 2 got 0
I20251211 20:52:47.827905 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 51d12723f5194afb97d2c05fcbaf3776 (table default.Table_823505200e3142dbbe8cd56ac78daf55 [id=ec0bb10933fa4529bef3580bee6409e7]). Expected version 2 got 0
I20251211 20:52:47.828270 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 2e800d8cc5ea4535a065d7d9243326b4 (table default.Table_083c478bedec45a5bf5203e4ed333860 [id=8baf0ea87d734d58896adbd0d6ef99c2]). Expected version 1 got 0
I20251211 20:52:47.828472 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet d43210ec61af423ebde624397dea9f2e (table default.Table_af0df4d1ee144455a6041e5a6a0b9b80 [id=7c5b2e33fa1c4b12b3941577360908a8]). Expected version 3 got 0
I20251211 20:52:47.828732 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 8675e2a18f0f4a0e95a87005cf82d8b4 (table default.Table_bd9633c55bd242c18aac3a7f2c18a329 [id=767f50bee5464316a18f73e64bab817a]). Expected version 2 got 0
I20251211 20:52:47.829314 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet f26ba8f4bd3a4a1f8fa2ba6e80d543f8 (table default.Table_71e9c158dc22435a8fbb02aecca8919d [id=b966d0c0b49d4023894950c5dc06f678]). Expected version 2 got 0
I20251211 20:52:47.829463 2808 raft_consensus.cc:493] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.829661 2808 raft_consensus.cc:515] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.829712 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 1c4b2c22bd4c456eb77af45576de466f (table default.Table_e3a99b043e4f420a93ebfee6aa87f2ec [id=3e7d6cefcb7d412191c6540e9ad77c2d]). Expected version 2 got 0
I20251211 20:52:47.830160 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet dfd194cf00f549c8937cb6ec40987240 (table default.Table_8db9acb9d35149fdabcb9a1d24061b46 [id=d919c6132b3c466e8a1404f981b003b8]). Expected version 1 got 0
I20251211 20:52:47.830338 2808 leader_election.cc:290] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.830425 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet bc4e7362e6d045da81bf39f4135f500e (table default.Table_328b7dd76e9e407c83b47a0f51cfcce5 [id=9e7c08f833e14a90b17a9b9f60b702de]). Expected version 1 got 0
I20251211 20:52:47.830659 1808 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet ce1799e6604f4ed7875ddedecab6f327 (table default.Table_5db1ac98e36d45b1b01889ee275ad83d [id=a6849726434d41d29dce12f47066a685]). Expected version 1 got 0
I20251211 20:52:47.833585 1168 tablet_bootstrap.cc:492] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:47.833732 1168 ts_tablet_manager.cc:1403] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20251211 20:52:47.834457 1168 raft_consensus.cc:359] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.834620 1168 raft_consensus.cc:385] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:47.834673 1168 raft_consensus.cc:740] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:47.834893 1168 consensus_queue.cc:260] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.835346 1168 ts_tablet_manager.cc:1434] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:47.838836 1168 tablet_bootstrap.cc:492] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:47.842128 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "44e2d2bdecf247baaac26e06bccc94de" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:47.842306 1168 tablet_bootstrap.cc:654] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:47.842450 1048 raft_consensus.cc:3060] T 44e2d2bdecf247baaac26e06bccc94de P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:47.843420 1166 tablet_bootstrap.cc:654] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:47.845211 1048 raft_consensus.cc:2468] T 44e2d2bdecf247baaac26e06bccc94de P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:47.848922 1168 tablet_bootstrap.cc:492] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:47.849066 1168 ts_tablet_manager.cc:1403] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20251211 20:52:47.849768 1168 raft_consensus.cc:359] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.849946 1168 raft_consensus.cc:385] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:47.849996 1168 raft_consensus.cc:740] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:47.851773 1168 consensus_queue.cc:260] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.852311 1168 ts_tablet_manager.cc:1434] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:47.852773 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f2ae084982f44fdf8bfecdd2bc4c01b9" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:47.853559 905 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de508d79f30f489cb745401f53115165" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:47.854501 1168 tablet_bootstrap.cc:492] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:47.857755 1168 tablet_bootstrap.cc:654] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:47.859802 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "969a8587019c4202b6e705d126f83d44" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:47.860426 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5df87ea7d28340da84ef141c92efc710" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:47.861426 913 raft_consensus.cc:3060] T 5453f68789fd4e89b69e98860050865b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:47.864084 1168 tablet_bootstrap.cc:492] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:47.864245 1168 ts_tablet_manager.cc:1403] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.001s
I20251211 20:52:47.867419 847 leader_election.cc:304] T efe4d6bb07374294b8ab50a9fe605f87 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:47.865128 1168 raft_consensus.cc:359] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.867568 1168 raft_consensus.cc:385] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:47.867656 1168 raft_consensus.cc:740] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:47.867794 2964 raft_consensus.cc:2749] T efe4d6bb07374294b8ab50a9fe605f87 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:47.868288 1168 consensus_queue.cc:260] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.868328 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "845b5ac4a5984c0cb4ab188925103dc9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:47.868613 1048 raft_consensus.cc:2468] T 845b5ac4a5984c0cb4ab188925103dc9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:47.870286 1168 ts_tablet_manager.cc:1434] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
W20251211 20:52:47.870618 711 leader_election.cc:343] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:47.871069 711 leader_election.cc:343] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:47.871400 1168 tablet_bootstrap.cc:492] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:47.871673 1160 tablet_bootstrap.cc:492] T 6b7dd8232d974a43b99ec9b28075dc8d P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:47.871821 1160 ts_tablet_manager.cc:1403] T 6b7dd8232d974a43b99ec9b28075dc8d P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.074s user 0.002s sys 0.001s
I20251211 20:52:47.872457 1160 raft_consensus.cc:359] T 6b7dd8232d974a43b99ec9b28075dc8d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.872627 1160 raft_consensus.cc:385] T 6b7dd8232d974a43b99ec9b28075dc8d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:47.872682 1160 raft_consensus.cc:740] T 6b7dd8232d974a43b99ec9b28075dc8d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
W20251211 20:52:47.872999 983 leader_election.cc:343] T f2ae084982f44fdf8bfecdd2bc4c01b9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:47.873116 983 leader_election.cc:304] T f2ae084982f44fdf8bfecdd2bc4c01b9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:47.873528 983 leader_election.cc:343] T de508d79f30f489cb745401f53115165 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:47.873610 983 leader_election.cc:304] T de508d79f30f489cb745401f53115165 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:47.873922 983 leader_election.cc:343] T 969a8587019c4202b6e705d126f83d44 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:47.874004 983 leader_election.cc:304] T 969a8587019c4202b6e705d126f83d44 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:47.874233 2981 raft_consensus.cc:2749] T f2ae084982f44fdf8bfecdd2bc4c01b9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:47.874274 983 leader_election.cc:343] T 5df87ea7d28340da84ef141c92efc710 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:47.874339 983 leader_election.cc:304] T 5df87ea7d28340da84ef141c92efc710 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:47.874370 2981 raft_consensus.cc:2749] T de508d79f30f489cb745401f53115165 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:47.874537 2981 raft_consensus.cc:2749] T 5df87ea7d28340da84ef141c92efc710 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:47.874553 2977 raft_consensus.cc:2749] T 969a8587019c4202b6e705d126f83d44 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:47.872867 1160 consensus_queue.cc:260] T 6b7dd8232d974a43b99ec9b28075dc8d P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.875502 1168 tablet_bootstrap.cc:654] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:47.876155 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 9c5c937cc7db4ed18fe03f4a8a99a66a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9c5c937cc7db4ed18fe03f4a8a99a66a already in progress: creating tablet
I20251211 20:52:47.876210 1160 ts_tablet_manager.cc:1434] T 6b7dd8232d974a43b99ec9b28075dc8d P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:47.876487 1160 tablet_bootstrap.cc:492] T 04086eaa32a74edd84235d3b28f9f965 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:47.877049 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 5f2c14c7a972414e8328e0b388bde0a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5f2c14c7a972414e8328e0b388bde0a4 already in progress: creating tablet
I20251211 20:52:47.878384 1160 tablet_bootstrap.cc:654] T 04086eaa32a74edd84235d3b28f9f965 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:47.882095 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 87c4ec07e82d4bf79b4eeefe732674ff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 87c4ec07e82d4bf79b4eeefe732674ff already in progress: creating tablet
I20251211 20:52:47.883442 2808 consensus_queue.cc:1048] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:47.885876 2736 consensus_queue.cc:1048] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:47.886260 2736 consensus_queue.cc:1048] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:47.886497 2494 consensus_queue.cc:1048] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:47.886550 2736 consensus_queue.cc:1048] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:47.888901 1160 tablet_bootstrap.cc:492] T 04086eaa32a74edd84235d3b28f9f965 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:47.889040 1160 ts_tablet_manager.cc:1403] T 04086eaa32a74edd84235d3b28f9f965 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20251211 20:52:47.889561 905 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "497c10742f01476db885bd70c4d2c491" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:47.890367 1160 raft_consensus.cc:359] T 04086eaa32a74edd84235d3b28f9f965 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.890533 1160 raft_consensus.cc:385] T 04086eaa32a74edd84235d3b28f9f965 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:47.890581 1160 raft_consensus.cc:740] T 04086eaa32a74edd84235d3b28f9f965 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:47.890776 905 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "06b8f8e46e524545aeab6c931efaf116" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:47.892517 1166 tablet_bootstrap.cc:492] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:47.892680 1166 ts_tablet_manager.cc:1403] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.075s user 0.003s sys 0.001s
I20251211 20:52:47.893568 1166 raft_consensus.cc:359] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.893746 1166 raft_consensus.cc:385] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:47.893807 1166 raft_consensus.cc:740] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:47.894058 1166 consensus_queue.cc:260] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.895110 905 raft_consensus.cc:3060] T efe4d6bb07374294b8ab50a9fe605f87 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:47.895377 2948 raft_consensus.cc:493] T be876471f8bf453aa4467c91891cbbde P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.895485 2948 raft_consensus.cc:515] T be876471f8bf453aa4467c91891cbbde P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.896006 2948 leader_election.cc:290] T be876471f8bf453aa4467c91891cbbde P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:47.898433 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f22e4d34cde24b37a025c1b5a1046b1b" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:47.899624 2970 raft_consensus.cc:493] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.903744 1166 ts_tablet_manager.cc:1434] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20251211 20:52:47.904191 1168 tablet_bootstrap.cc:492] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:47.904175 2970 raft_consensus.cc:515] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.904388 1168 ts_tablet_manager.cc:1403] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.034s user 0.006s sys 0.000s
I20251211 20:52:47.909292 847 leader_election.cc:304] T f22e4d34cde24b37a025c1b5a1046b1b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:47.909794 2973 raft_consensus.cc:2749] T f22e4d34cde24b37a025c1b5a1046b1b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:47.912484 2970 leader_election.cc:290] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.913301 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "be876471f8bf453aa4467c91891cbbde" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:47.914589 1168 raft_consensus.cc:359] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.914776 1168 raft_consensus.cc:385] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:47.914849 1168 raft_consensus.cc:740] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:47.915066 1168 consensus_queue.cc:260] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.915647 1168 ts_tablet_manager.cc:1434] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20251211 20:52:47.920691 913 raft_consensus.cc:3060] T 50f9f29c064c4f938a0f20f302ab5e8a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:47.921561 1160 consensus_queue.cc:260] T 04086eaa32a74edd84235d3b28f9f965 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.904961 1038 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3e6f62c2deff4c11b137fc0cba5ee234" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:47.905004 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5453f68789fd4e89b69e98860050865b" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:47.924422 1166 tablet_bootstrap.cc:492] T 834b84a557b04e78b5d2c8c84520a644 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:47.927024 1160 ts_tablet_manager.cc:1434] T 04086eaa32a74edd84235d3b28f9f965 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.038s user 0.002s sys 0.000s
I20251211 20:52:47.929982 1160 tablet_bootstrap.cc:492] T 75dd170c5828409fa69bd68fdf28a72b P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:47.930956 847 leader_election.cc:343] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: b529b38d754f46ba8cb60b19e9705256
I20251211 20:52:47.931097 847 leader_election.cc:304] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:47.931670 2973 raft_consensus.cc:2749] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:47.931533 847 leader_election.cc:304] T 3e6f62c2deff4c11b137fc0cba5ee234 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
W20251211 20:52:47.932077 982 leader_election.cc:343] T be876471f8bf453aa4467c91891cbbde P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:47.932142 847 leader_election.cc:304] T 5453f68789fd4e89b69e98860050865b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:47.932462 2966 raft_consensus.cc:2749] T 3e6f62c2deff4c11b137fc0cba5ee234 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:47.932618 2966 raft_consensus.cc:2749] T 5453f68789fd4e89b69e98860050865b P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:47.932622 2970 consensus_queue.cc:1048] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:47.934449 889 tablet_service.cc:1505] Processing CreateTablet for tablet 034d2d9c10464f2b80d1bcfa3e4a1df8 (DEFAULT_TABLE table=default.Table_3c308b240c3b490d9022c7d292cdb743 [id=c6a02e2ed6a54580b93a48b4cefbdde3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.934476 757 tablet_service.cc:1505] Processing CreateTablet for tablet 034d2d9c10464f2b80d1bcfa3e4a1df8 (DEFAULT_TABLE table=default.Table_3c308b240c3b490d9022c7d292cdb743 [id=c6a02e2ed6a54580b93a48b4cefbdde3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.934891 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 034d2d9c10464f2b80d1bcfa3e4a1df8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.936175 1021 tablet_service.cc:1505] Processing CreateTablet for tablet 034d2d9c10464f2b80d1bcfa3e4a1df8 (DEFAULT_TABLE table=default.Table_3c308b240c3b490d9022c7d292cdb743 [id=c6a02e2ed6a54580b93a48b4cefbdde3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:47.936560 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 034d2d9c10464f2b80d1bcfa3e4a1df8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.938066 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 034d2d9c10464f2b80d1bcfa3e4a1df8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:47.938103 1168 tablet_bootstrap.cc:492] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:47.942646 710 leader_election.cc:304] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:47.942896 2970 raft_consensus.cc:2804] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:47.942970 2970 raft_consensus.cc:493] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:47.943018 1168 tablet_bootstrap.cc:654] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:47.943027 2970 raft_consensus.cc:3060] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:47.944141 2948 raft_consensus.cc:493] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:47.944281 2948 raft_consensus.cc:515] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.944506 710 leader_election.cc:304] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:47.944759 2959 raft_consensus.cc:2804] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:47.944828 2959 raft_consensus.cc:493] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:47.944883 2959 raft_consensus.cc:3060] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:47.944885 2948 leader_election.cc:290] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:47.947330 2959 raft_consensus.cc:515] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.948168 2959 leader_election.cc:290] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.950372 2970 raft_consensus.cc:515] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.951242 2970 leader_election.cc:290] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.952149 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "32f8d104f6a948d1a97a83769ea82449" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:47.952715 982 leader_election.cc:343] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:47.953040 710 leader_election.cc:304] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:47.953465 3001 raft_consensus.cc:2804] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:47.953821 3001 raft_consensus.cc:493] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:47.954097 3001 raft_consensus.cc:3060] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:47.956705 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd43a27db69044b68826cda87e16a086" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:47.957955 1166 tablet_bootstrap.cc:654] T 834b84a557b04e78b5d2c8c84520a644 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:47.959126 1160 tablet_bootstrap.cc:654] T 75dd170c5828409fa69bd68fdf28a72b P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:47.960024 3003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:52:47.961921 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3621735fcb294e3a8c0180ba9ef86442" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:47.962375 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71dc8d20d93243e289330d7b8c635511" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:47.963826 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "44e2d2bdecf247baaac26e06bccc94de" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:47.968868 2970 consensus_queue.cc:1048] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:47.969620 710 leader_election.cc:304] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:47.969851 2970 raft_consensus.cc:2804] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:47.969915 2970 raft_consensus.cc:493] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:47.969969 2970 raft_consensus.cc:3060] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:47.970170 710 leader_election.cc:304] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:47.981148 2969 raft_consensus.cc:2804] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:47.982882 2961 consensus_queue.cc:1048] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:47.983690 2807 consensus_queue.cc:1048] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20251211 20:52:47.981249 2969 raft_consensus.cc:493] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:47.987323 2969 raft_consensus.cc:3060] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:47.991858 2969 raft_consensus.cc:515] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.992514 2969 leader_election.cc:290] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:47.992991 710 leader_election.cc:304] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:47.993321 2961 raft_consensus.cc:2804] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:47.993438 2961 raft_consensus.cc:697] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:47.993678 2961 consensus_queue.cc:237] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.995414 2969 consensus_queue.cc:1048] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:47.995440 2970 raft_consensus.cc:515] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:47.996260 2550 server_base.cc:1047] running on GCE node
I20251211 20:52:47.997627 2550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20251211 20:52:47.998227 1784 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.60:42087: connect: Connection refused (error 111) [suppressed 12 similar messages]
I20251211 20:52:47.999028 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 0302cc8613934c43a0650f081cf98797 (table default.Table_bf827b216f1b43c196923dc1828ff0d0 [id=ed371e073250439fa44330c7101e9387]). Expected version 1 got 0
I20251211 20:52:47.999383 1166 tablet_bootstrap.cc:492] T 834b84a557b04e78b5d2c8c84520a644 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:47.999411 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 1f4eb1a174c648c897ae2b7612e44585 (table default.Table_e70873b71ba24d7eae548e2acfd243b3 [id=87adcc552a704d56bd2fe812fb3b791f]). Expected version 1 got 0
I20251211 20:52:47.999648 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet a7afcce522494c9ca1d80e6eea913e6e (table default.Table_cb6a4bac7b694f33a37de2ee78c91ca0 [id=8940f18527c249a9baefc170ed82f45a]). Expected version 1 got 0
I20251211 20:52:47.999805 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 84bb3583857848459e0e865283de623d (table default.Table_92281c5fcfee4355a79215ffedeee588 [id=6e5fe0a78e4b48f98dac028802f1ddf4]). Expected version 4 got 0
I20251211 20:52:47.999998 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 7a299ee03d0e48aba7c4f71e3e0f381a (table default.Table_6402a801d48e4a53827b1b62b6cdaba1 [id=6876565d3a9c48ce841b830cf75139ff]). Expected version 3 got 0
I20251211 20:52:48.000377 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 5b3e8ec3faf445ac9b746e65dc4dbfe0 (table default.Table_7fe80ff5483a4b0292091119980f6639 [id=3c3462a0c09748cea0d2ffb544ae2f13]). Expected version 1 got 0
I20251211 20:52:48.000566 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 7ffa06c4a7ee48b0b337a27aff090592 (table default.Table_647731023b044ebb8954f7fdcc3faf97 [id=4b91abd6e9e94a8a941b33e62e23c083]). Expected version 1 got 0
I20251211 20:52:48.000694 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet b81a4cef27164824b52695039744db15 (table default.Table_defdeab28749411590cf75b5251f604b [id=11352566d1744193a40a8bb605aa3912]). Expected version 1 got 0
I20251211 20:52:48.000900 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 7023afec769c4ca3b2242a225004d482 (table default.Table_d5276e3cc46d4331b36a8878680fc55e [id=3811f8235b2f44cca283401302d613d4]). Expected version 1 got 0
I20251211 20:52:48.001065 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 72af55c8045b4f8f8e116a5782794d81 (table default.Table_2cff6fab088c4d939b13b504c133cc12 [id=b6c4b72063b3428593d5650a160dc66b]). Expected version 1 got 0
I20251211 20:52:48.001416 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 4cf0cde801bf44449e539d2799b895a7 (table default.Table_907c596bdddc4edca25d14cbacebb23f [id=321235280d1d4fa0bae5cc0333860f85]). Expected version 1 got 0
I20251211 20:52:48.001585 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 7b9f569554fb4836ac2b62542c3ecf7e (table default.Table_1a73d567e8914a83974e3e8f5f58be0b [id=6ef691749df24251a7a7c5919c219754]). Expected version 1 got 0
I20251211 20:52:48.001744 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 3b2c6064c5eb4c0f8979c485f5d85a75 (table default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f]). Expected version 1 got 0
I20251211 20:52:48.001955 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 59acb8474e114bf7ae7f1f043d611459 (table default.Table_c7ed8c0c309c40a3a371ffa130b47cd2 [id=ed1d0670426945aba888b818ad657225]). Expected version 1 got 0
I20251211 20:52:48.006239 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet b52377c8ffd741a598e5328a00d876fe (table default.Table_686a26915b42421380037144900dd97c [id=8a1b0c9d641b4a3c83d1ca48576d110d]). Expected version 1 got 0
I20251211 20:52:48.006752 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet ce1799e6604f4ed7875ddedecab6f327 (table default.Table_5db1ac98e36d45b1b01889ee275ad83d [id=a6849726434d41d29dce12f47066a685]). Expected version 1 got 0
I20251211 20:52:48.006961 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet dfd194cf00f549c8937cb6ec40987240 (table default.Table_8db9acb9d35149fdabcb9a1d24061b46 [id=d919c6132b3c466e8a1404f981b003b8]). Expected version 1 got 0
I20251211 20:52:48.007342 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 1c4b2c22bd4c456eb77af45576de466f (table default.Table_e3a99b043e4f420a93ebfee6aa87f2ec [id=3e7d6cefcb7d412191c6540e9ad77c2d]). Expected version 2 got 0
I20251211 20:52:48.007287 3001 raft_consensus.cc:515] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.007808 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet bc4e7362e6d045da81bf39f4135f500e (table default.Table_328b7dd76e9e407c83b47a0f51cfcce5 [id=9e7c08f833e14a90b17a9b9f60b702de]). Expected version 1 got 0
I20251211 20:52:48.008169 2970 leader_election.cc:290] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:48.008183 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet f26ba8f4bd3a4a1f8fa2ba6e80d543f8 (table default.Table_71e9c158dc22435a8fbb02aecca8919d [id=b966d0c0b49d4023894950c5dc06f678]). Expected version 2 got 0
I20251211 20:52:48.008855 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 8b793241ee3b42b1a38e9f2f05e1173b (table default.Table_7506ffecc2b64648a9ba736e5298ddf4 [id=279f39d718ba45318f52f95e8aa13652]). Expected version 2 got 0
I20251211 20:52:48.009312 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet ab4e7a9062134e489d6f9abe6ff26fb4 (table default.Table_4cbc8841b276406cb4434629653a5ce3 [id=b958532650374d609199cab9c07bb64b]). Expected version 2 got 0
I20251211 20:52:48.009552 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 6b7dd8232d974a43b99ec9b28075dc8d (table default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]). Expected version 2 got 0
I20251211 20:52:48.009688 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet c7a2610f20df4c1ca81eae2fc7dba6bb (table default.Table_59072235d93b41c0b5cd5e3debcb8ddf [id=9a15a8a075af4969ae2c8b3be1bfed78]). Expected version 1 got 0
I20251211 20:52:48.010244 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 645b08b33676420c96e880de3d460829 (table default.Table_e02933dd91ce4fe7a40d1c3b2f9b2d4b [id=6d42f260246e420d8818b3841926302b]). Expected version 2 got 0
I20251211 20:52:48.010447 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 4c0455a5b4f44412803475d1c4852b48 (table default.Table_e00092673b0e4f09b8d7eec63e1ee034 [id=18497e06d7254441a4655ce1cdc5a9bf]). Expected version 1 got 0
I20251211 20:52:48.010623 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 5453f68789fd4e89b69e98860050865b (table default.Table_b7bf8c5ad5304175a1f2f6a03887f725 [id=50d88b663a4d4891b113aee9c5906699]). Expected version 1 got 0
I20251211 20:52:48.010798 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet f00128ba84c245d9a0e35f26bc842b3e (table default.Table_6f8c133c7a364b93af2c83d8911a0ac3 [id=6a21c38ee7224a02b937df088494f5d8]). Expected version 1 got 0
I20251211 20:52:48.011190 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 7ca98b43863a47c091baae0a6ce65630 (table default.Table_f0a70914acc94231a4e84155c3e4d13f [id=56c070ad80624c698ebe2e01b4929b6e]). Expected version 1 got 0
I20251211 20:52:48.011504 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 16d2ff8925e342729a71c93db378d031 (table default.Table_34e6ed4f13da437ea50d349b92e83bd6 [id=4a74b9a60cbb4198b55cee630b2f4bef]). Expected version 2 got 0
I20251211 20:52:48.011895 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 46a2a070cf6a42fe8bc54a42eb802fbc (table default.Table_4e1f03b9d87b4fe59dbcb3a947ee299d [id=8e66ea58499b40e59294a135324aa527]). Expected version 1 got 0
I20251211 20:52:48.012086 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet a3b1aadf17ea4bf0a896d23cd543fca7 (table default.Table_d18401707c88493abdf90d2c713432ae [id=a07662f8dc3f4178be5dfcc1cdc85837]). Expected version 1 got 0
I20251211 20:52:48.012226 913 raft_consensus.cc:3060] T f22e4d34cde24b37a025c1b5a1046b1b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.012455 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet aa50ddb2789b4be3bfa9aedef9287109 (table default.Table_44f971096d1e41f480e594985ffd5c17 [id=d4e1349374a346a5b8be180f9bdc329a]). Expected version 1 got 0
I20251211 20:52:48.012661 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 497c10742f01476db885bd70c4d2c491 (table default.Table_18b21232dfa74378b27defe672bb2574 [id=602d7eee703948d99a0edcb7cac0651b]). Expected version 1 got 0
I20251211 20:52:48.012804 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 3003ef71a3554c71a3b24cdfb9c293bc (table default.Table_92efd469dd324f7d886e2cbc90ece61c [id=6e7fb3271e0c48c0bb9d56fe4fccee3a]). Expected version 1 got 0
I20251211 20:52:48.012998 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet b85af79c2574443998e847bc19c53b69 (table default.Table_fa3f2f115cd043a9aa6be2b070d320cb [id=15fad5e1cf8d4ea08dae8275a29a11dd]). Expected version 1 got 0
I20251211 20:52:48.013207 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet bc6a1c84501d4e179cd2828b50fddc23 (table default.Table_8bb5b228d60e4d29a41d9ba40c33d45d [id=6ef058fada9941629e0ea4d1906abdac]). Expected version 1 got 0
I20251211 20:52:48.013329 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet d6923cfd826d495291d9ac8e0b3b44c6 (table default.Table_68ccb2052d824ccf92f7162d2961fcb1 [id=9ac9591d4f04482d926710fdf6e797c6]). Expected version 1 got 0
I20251211 20:52:48.013460 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 5773c13034714a008c8d7dbe777b4828 (table default.Table_66eb1b021eed464f82f9cd223c88297b [id=94ff09564cd649c7a2d0e94b0f3c7b3a]). Expected version 1 got 0
I20251211 20:52:48.013710 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 18f125dd9efc401181f0644dad7aeffe (table default.Table_fa6592c2e2624d81ad21abb13ed36045 [id=9b48fc2b39db4fd5a4e5da19e34427dc]). Expected version 1 got 0
I20251211 20:52:48.013861 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet f94b656e3c1844509fdae36e33b263e8 (table default.Table_ed03ad475bed4804a65abe20efef0625 [id=e23f4305669a474aafa18928f130de3a]). Expected version 2 got 0
I20251211 20:52:48.014102 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 5e47af616e1b4d2db3780c1d67cdcb47 (table default.Table_296bdf30a11541c389afe3df0b0f667a [id=052edcbde60c41449d468c8a449fc3f1]). Expected version 1 got 0
I20251211 20:52:48.014472 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 03184dcc56dd4e92adc7d0ad22aa8d8b (table default.Table_68261841754847ad8d22e86c29886144 [id=697f23a617f64762bf406b386cde7473]). Expected version 1 got 0
I20251211 20:52:48.014736 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet f2d6c9781b4d4c37bc08703347963d5d (table default.Table_2aa7ff9186a04249abf8cd971cc0cb17 [id=b50ab5b032ba4c98938324460512468f]). Expected version 1 got 0
I20251211 20:52:48.014926 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 13269a20421240d1a98215129443248b (table default.Table_d887c953026046dc883ede774a58fd8e [id=1d8a79f716a0447cb0393f4759178598]). Expected version 1 got 0
I20251211 20:52:48.015049 1168 tablet_bootstrap.cc:492] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:48.015220 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 78f77d8916d84edabbe95ce17d8df2c1 (table default.Table_e2af0e2fbd3d46d595136c7a7bf24e4c [id=ddc12ae6d8ed4587b5ef4da3ef81a055]). Expected version 1 got 0
I20251211 20:52:48.015262 1168 ts_tablet_manager.cc:1403] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.077s user 0.007s sys 0.004s
I20251211 20:52:48.015502 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 8dc4e730951144d78b77280f6cce6d0e (table default.Table_890c9027415a44ccaf95c0adede51992 [id=7bc8a2b91fe54b91b6594b371fea7bd3]). Expected version 2 got 0
I20251211 20:52:48.015676 3001 leader_election.cc:290] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:48.016611 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet b2974f16034c44cf85ff85b5095cf6a0 (table default.Table_031de0f4cc7744bc9e320261f8b46696 [id=be36dabaa6d44325b4855b9f13bda847]). Expected version 1 got 0
I20251211 20:52:48.017760 961 heartbeater.cc:499] Master 127.0.104.61:40749 was elected leader, sending a full tablet report...
I20251211 20:52:48.019963 1168 raft_consensus.cc:359] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.020188 1168 raft_consensus.cc:385] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:48.020395 1168 raft_consensus.cc:740] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:48.020632 1168 consensus_queue.cc:260] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.021070 1168 ts_tablet_manager.cc:1434] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20251211 20:52:48.022029 1168 tablet_bootstrap.cc:492] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:48.022111 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet ff4d5ec1f87247e896f9c099c532265a (table default.Table_3ac4746e17264f5a93890f3600505781 [id=475e0a1ec0344d10addbcfa42fe05b68]). Expected version 1 got 0
I20251211 20:52:48.022501 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet b6cc819332e1445c96ce3ffac4861454 (table default.Table_db4c0f706e0643b39d39ffa84352826d [id=4d459c42bf584068ac3851227d80331a]). Expected version 1 got 0
I20251211 20:52:48.022872 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 6212b3b4425543168837946a54d60834 (table default.Table_8c30c4151d484db0bb9e2b9ca262248d [id=2d489cd7d73e403d84bf10960e6a2dfb]). Expected version 1 got 0
I20251211 20:52:48.023098 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 3dfea28c44d84c68850db65a30d52ee6 (table default.Table_c504f8f3796f43b697488bd604268543 [id=2af76b9d32094a4ca656060a4478e08a]). Expected version 1 got 0
I20251211 20:52:48.023466 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 94658fcf512a48d0a992a7e2e0f7c176 (table default.Table_c9d86e82aebb4ef0bc83789720352e38 [id=97377a8ef56c4cdebe95d15955207404]). Expected version 1 got 0
I20251211 20:52:48.023679 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 4c7b74f370b042ee8575f821c07dc085 (table default.Table_731826f26f9940b8afe425d694b642d9 [id=17dd9cea9ac844128ff9f82adb3f919f]). Expected version 1 got 0
I20251211 20:52:48.023981 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet aab26df748724fe5b2bb7fad9e3bfc0c (table default.Table_22bed4b4e9e24a90818b19c6f5c91e1f [id=df6255d66f334ef2b4406f4b845cf980]). Expected version 1 got 0
I20251211 20:52:48.024183 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet fb29f983c4524a49bdc5c0c1144ad708 (table default.Table_7809ac85fdfe4c1d9c58afb54edecb27 [id=c4b0bfff99f64888beb70d9154d2f156]). Expected version 1 got 0
I20251211 20:52:48.024381 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet df457783b2534b3ea23bb1fd529ce268 (table default.Table_6e9f3ca164384cc8ac4501deb3f0c6aa [id=342c2469d6e74e3e9217236e3aa863d6]). Expected version 1 got 0
I20251211 20:52:48.024773 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 33fd09ea06ee47908b85911e4ee79708 (table default.Table_3c3cd48c8fe5411e96cdf811d492925d [id=30620a01c27f456bac05fa7c02a2cb4d]). Expected version 2 got 0
I20251211 20:52:48.025009 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 3c865c99e9314f5f91658b9126da4d39 (table default.Table_0d7df3fba38849a49b65d8290812b95c [id=7f966d5cdad3423bb770414802a3cf7b]). Expected version 1 got 0
I20251211 20:52:48.025493 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 9f8aa28c1c9b4fc58dfc128952c180ba (table default.Table_bf7d402a36be44cfb6b79041906d26a1 [id=0327e686e7ed4475a3e8cda8721a3a3a]). Expected version 1 got 0
I20251211 20:52:48.025761 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 676faec7f3d1488f99b220f8e968e9a0 (table default.Table_ab826a3eec8e4c4aaa8d4021d87dd382 [id=a2bbfccfe3974c378521d5ef2333df86]). Expected version 2 got 0
I20251211 20:52:48.025923 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 8675e2a18f0f4a0e95a87005cf82d8b4 (table default.Table_bd9633c55bd242c18aac3a7f2c18a329 [id=767f50bee5464316a18f73e64bab817a]). Expected version 2 got 0
I20251211 20:52:48.026329 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet fd43a27db69044b68826cda87e16a086 (table default.Table_a8ebfd7d7a7f4f6cae8b6b02b99b80b0 [id=f1d32002c612403a82bdff5baf209b9e]). Expected version 1 got 0
I20251211 20:52:48.026788 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet e4647105ab57454d883ebfcde7bde891 (table default.Table_c817dec9a7674f699b3366fc302ead57 [id=d555d1fc6aef4a21b4d3f2804264d030]). Expected version 1 got 0
I20251211 20:52:48.027263 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet b115d737d7fc483d9f5a1f040b511bae (table default.Table_6d94053661f142b79292e19752df02df [id=c5be59a2c3934b3789641ceeb2d4d6e2]). Expected version 2 got 0
I20251211 20:52:48.027413 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet da81bf8854a24954ba4548185812295d (table default.Table_18262e22c26d4d66ba4c6134fbde0368 [id=da8cf2b99b42409091484a9513623d6f]). Expected version 1 got 0
I20251211 20:52:48.027792 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 3e6f62c2deff4c11b137fc0cba5ee234 (table default.Table_51db250005ed4b3d947f32acd1d62563 [id=fdfaa0b24d2f4736aee695ed228212cf]). Expected version 1 got 0
I20251211 20:52:48.027966 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet e3f711f02af843adb4f5c3c750924e33 (table default.Table_34bd5155aa3b4ef7818022ba9346c074 [id=83a61d23e80946439a14096ab3ab1bd9]). Expected version 2 got 0
I20251211 20:52:48.028213 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet d37a7d0d662e45079b0cfc02765d2e01 (table default.Table_014b6db9f3294658b965c701384b98bb [id=8a12299efe714617a970fc05c3bb6232]). Expected version 2 got 0
I20251211 20:52:48.028348 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet ee7b93b0b87541d9af6eb9e68b4cea41 (table default.Table_b6e6aea68c88426fb01353146c06b6ac [id=a4f984b4263a4bc38f6248628f643ae3]). Expected version 3 got 0
I20251211 20:52:48.028259 2969 consensus_queue.cc:1048] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:48.028635 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 969a8587019c4202b6e705d126f83d44 (table default.Table_6460313002684bba93b30ad8631f0f49 [id=22347b15537d451d9927a17e8e5a202a]). Expected version 1 got 0
I20251211 20:52:48.028823 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 46cb1d16a0e94b3a833f32c635213fd2 (table default.Table_3482fbfc5b144a7198a7d01aa003cb6b [id=1bdaf5ce0f6b4b99a038f9f9efb828f6]). Expected version 1 got 0
I20251211 20:52:48.029273 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 2e800d8cc5ea4535a065d7d9243326b4 (table default.Table_083c478bedec45a5bf5203e4ed333860 [id=8baf0ea87d734d58896adbd0d6ef99c2]). Expected version 1 got 0
I20251211 20:52:48.029417 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet d43210ec61af423ebde624397dea9f2e (table default.Table_af0df4d1ee144455a6041e5a6a0b9b80 [id=7c5b2e33fa1c4b12b3941577360908a8]). Expected version 3 got 0
I20251211 20:52:48.029583 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 834b84a557b04e78b5d2c8c84520a644 (table default.Table_909bbe6da1a648d68199fb27424e4342 [id=cc99459a5039424dbc192ff748b6708b]). Expected version 2 got 0
I20251211 20:52:48.029515 2969 consensus_queue.cc:1048] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:48.029774 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 7be44df39a8f4d6da793479d425fa684 (table default.Table_f6a10556ff104db5bd5ecc9fc380e228 [id=f75813add54e46189ff1bbed82d83ba7]). Expected version 1 got 0
I20251211 20:52:48.030345 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet e89d4d76bf64453892a4d0b55fcc3454 (table default.Table_fba99384e9f14160b3e3504f8d1d0b07 [id=01bba75f854d4c338d6c09a446dd7499]). Expected version 1 got 0
I20251211 20:52:48.030581 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 171d40a8a9124f9bb1b38797fef8e5d5 (table default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5]). Expected version 2 got 0
I20251211 20:52:48.031004 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 4fe5c534b572491c911aae01eaa79bae (table default.Table_c1f9dca5c2144d50b67681dcc6800e77 [id=14fa8debfe5443ce81587f02dfa15847]). Expected version 2 got 0
I20251211 20:52:48.031210 1798 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 51d12723f5194afb97d2c05fcbaf3776 (table default.Table_823505200e3142dbbe8cd56ac78daf55 [id=ec0bb10933fa4529bef3580bee6409e7]). Expected version 2 got 0
W20251211 20:52:48.032182 1806 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 938a0d8a090f490b816fdf2b8f1a51b8 consensus state has no leader
I20251211 20:52:48.032366 1806 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46568 (request call id 69) took 2977 ms. Trace:
I20251211 20:52:48.032476 1806 rpcz_store.cc:276] 1211 20:52:45.055218 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:47.353261 (+2298043us) service_pool.cc:225] Handling call
1211 20:52:47.353618 (+ 357us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
1211 20:52:47.353841 (+ 223us) catalog_manager.cc:3708] Apply alter schema
1211 20:52:47.354034 (+ 193us) catalog_manager.cc:3740] Acquired catalog manager lock
1211 20:52:47.354109 (+ 75us) catalog_manager.cc:3898] Updating metadata on disk
1211 20:52:47.379711 (+ 25602us) catalog_manager.cc:3929] Committing alterations to in-memory state
1211 20:52:48.032331 (+652620us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:47.355269 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:47.356201 (+ 932us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:47.356208 (+ 7us) write_op.cc:435] Acquired schema lock
1211 20:52:47.356211 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:47.356323 (+ 112us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:47.356339 (+ 16us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:47.356343 (+ 4us) tablet.cc:683] Acquiring locks for 1 operations
1211 20:52:47.356435 (+ 92us) tablet.cc:699] Row locks acquired
1211 20:52:47.356438 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:47.356485 (+ 47us) write_op.cc:270] Start()
1211 20:52:47.356527 (+ 42us) write_op.cc:276] Timestamp: P: 1765486367356422 usec, L: 0
1211 20:52:47.356531 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:47.356871 (+ 340us) log.cc:844] Serialized 405 byte log entry
1211 20:52:47.367330 (+ 10459us) op_driver.cc:464] REPLICATION: finished
1211 20:52:47.367468 (+ 138us) write_op.cc:301] APPLY: starting
1211 20:52:47.367488 (+ 20us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:47.367536 (+ 48us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:47.367597 (+ 61us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:47.367657 (+ 60us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:47.367694 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1211 20:52:47.367701 (+ 7us) write_op.cc:312] APPLY: finished
1211 20:52:47.367795 (+ 94us) log.cc:844] Serialized 23 byte log entry
1211 20:52:47.367864 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.367894 (+ 30us) write_op.cc:454] Released schema lock
1211 20:52:47.367903 (+ 9us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:47.367976 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.367979 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":87,"apply.run_cpu_time_us":498,"apply.run_wall_time_us":498,"num_ops":1,"prepare.queue_time_us":632,"prepare.run_cpu_time_us":984,"prepare.run_wall_time_us":1747,"raft.queue_time_us":1124,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":255,"replication_time_us":10763}]]}
I20251211 20:52:48.033334 1806 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_a1e4b060f2ff4ebab9bc8d88d25aa4c8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:48.033982 1806 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a1e4b060f2ff4ebab9bc8d88d25aa4c8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:47.999552 1166 ts_tablet_manager.cc:1403] T 834b84a557b04e78b5d2c8c84520a644 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.075s user 0.003s sys 0.002s
I20251211 20:52:48.036178 2550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
W20251211 20:52:48.036731 1799 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dfd194cf00f549c8937cb6ec40987240 consensus state has no leader
I20251211 20:52:48.036936 1799 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46568 (request call id 56) took 3154 ms. Trace:
I20251211 20:52:48.037003 1166 raft_consensus.cc:359] T 834b84a557b04e78b5d2c8c84520a644 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:48.037108 1802 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 81fe26e81790448fb436929c03e3c284 consensus state has no leader
I20251211 20:52:48.037165 1166 raft_consensus.cc:385] T 834b84a557b04e78b5d2c8c84520a644 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:48.037214 1166 raft_consensus.cc:740] T 834b84a557b04e78b5d2c8c84520a644 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:48.037300 1802 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46568 (request call id 77) took 2593 ms. Trace:
I20251211 20:52:48.037472 1797 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Sending DeleteTablet for 3 replicas of tablet eb9da081cb9d43fd9a35c4bc51ea133e
I20251211 20:52:48.037570 1796 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: default.Table_b44b03dcd95a41c19061d5e52f5687c9 [id=ef458da415914c138373fbd5e0b8dec4] alter complete (version 1)
I20251211 20:52:48.038043 1797 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46568 (request call id 61) took 3082 ms. Trace:
I20251211 20:52:48.038162 1797 rpcz_store.cc:276] 1211 20:52:44.955243 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:45.072827 (+117584us) service_pool.cc:225] Handling call
1211 20:52:45.073253 (+ 426us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
1211 20:52:45.073388 (+ 135us) catalog_manager.cc:2815] Modifying in-memory table state
1211 20:52:45.073462 (+ 74us) catalog_manager.cc:2824] Locking tablets
1211 20:52:45.073615 (+ 153us) catalog_manager.cc:2838] Removing table and tablets from system table
1211 20:52:45.098647 (+ 25032us) catalog_manager.cc:2856] Removing table from by-name map
1211 20:52:45.098725 (+ 78us) catalog_manager.cc:2872] Committing in-memory state
1211 20:52:48.037392 (+2938667us) catalog_manager.cc:2876] Aborting table tasks
1211 20:52:48.037993 (+ 601us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:45.082201 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:45.082291 (+ 90us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:45.082296 (+ 5us) write_op.cc:435] Acquired schema lock
1211 20:52:45.082298 (+ 2us) tablet.cc:660] Decoding operations
1211 20:52:45.082393 (+ 95us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:45.082402 (+ 9us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:45.082407 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:45.082499 (+ 92us) tablet.cc:699] Row locks acquired
1211 20:52:45.082501 (+ 2us) write_op.cc:260] PREPARE: finished
1211 20:52:45.082534 (+ 33us) write_op.cc:270] Start()
1211 20:52:45.082574 (+ 40us) write_op.cc:276] Timestamp: P: 1765486365082497 usec, L: 0
1211 20:52:45.082578 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:45.082722 (+ 144us) log.cc:844] Serialized 756 byte log entry
1211 20:52:45.097395 (+ 14673us) op_driver.cc:464] REPLICATION: finished
1211 20:52:45.098277 (+ 882us) write_op.cc:301] APPLY: starting
1211 20:52:45.098287 (+ 10us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:45.098323 (+ 36us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:45.098350 (+ 27us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:45.098404 (+ 54us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:45.098431 (+ 27us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
1211 20:52:45.098437 (+ 6us) write_op.cc:312] APPLY: finished
1211 20:52:45.098495 (+ 58us) log.cc:844] Serialized 29 byte log entry
1211 20:52:45.098537 (+ 42us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:45.098552 (+ 15us) write_op.cc:454] Released schema lock
1211 20:52:45.098557 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":851,"apply.run_cpu_time_us":343,"apply.run_wall_time_us":342,"num_ops":2,"prepare.queue_time_us":7933,"prepare.run_cpu_time_us":601,"prepare.run_wall_time_us":601,"raft.queue_time_us":107,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":4,"replication_time_us":14801,"thread_start_us":91839,"threads_started":1}]]}
I20251211 20:52:48.038899 1160 tablet_bootstrap.cc:492] T 75dd170c5828409fa69bd68fdf28a72b P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:48.039037 1160 ts_tablet_manager.cc:1403] T 75dd170c5828409fa69bd68fdf28a72b P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.109s user 0.001s sys 0.004s
I20251211 20:52:48.038997 1797 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_18f6559b16734cbaa5659427c4276f5d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:48.041322 1166 consensus_queue.cc:260] T 834b84a557b04e78b5d2c8c84520a644 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.044178 889 tablet_service.cc:1552] Processing DeleteTablet for tablet eb9da081cb9d43fd9a35c4bc51ea133e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:48.037030 1799 rpcz_store.cc:276] 1211 20:52:44.882458 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:44.882512 (+ 54us) service_pool.cc:225] Handling call
1211 20:52:45.075906 (+193394us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
1211 20:52:45.076115 (+ 209us) catalog_manager.cc:3708] Apply alter schema
1211 20:52:45.076299 (+ 184us) catalog_manager.cc:3740] Acquired catalog manager lock
1211 20:52:45.076356 (+ 57us) catalog_manager.cc:3898] Updating metadata on disk
1211 20:52:45.190532 (+114176us) catalog_manager.cc:3929] Committing alterations to in-memory state
1211 20:52:48.036871 (+2846339us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:45.083406 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:45.083486 (+ 80us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:45.083490 (+ 4us) write_op.cc:435] Acquired schema lock
1211 20:52:45.083492 (+ 2us) tablet.cc:660] Decoding operations
1211 20:52:45.083597 (+ 105us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:45.083605 (+ 8us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:45.083609 (+ 4us) tablet.cc:683] Acquiring locks for 1 operations
1211 20:52:45.083663 (+ 54us) tablet.cc:699] Row locks acquired
1211 20:52:45.083665 (+ 2us) write_op.cc:260] PREPARE: finished
1211 20:52:45.083689 (+ 24us) write_op.cc:270] Start()
1211 20:52:45.083710 (+ 21us) write_op.cc:276] Timestamp: P: 1765486365083656 usec, L: 0
1211 20:52:45.083712 (+ 2us) op_driver.cc:348] REPLICATION: starting
1211 20:52:45.083830 (+ 118us) log.cc:844] Serialized 405 byte log entry
1211 20:52:45.189522 (+105692us) op_driver.cc:464] REPLICATION: finished
1211 20:52:45.190189 (+ 667us) write_op.cc:301] APPLY: starting
1211 20:52:45.190201 (+ 12us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:45.190234 (+ 33us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:45.190258 (+ 24us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:45.190297 (+ 39us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:45.190325 (+ 28us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1211 20:52:45.190332 (+ 7us) write_op.cc:312] APPLY: finished
1211 20:52:45.190390 (+ 58us) log.cc:844] Serialized 23 byte log entry
1211 20:52:45.190431 (+ 41us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:45.190444 (+ 13us) write_op.cc:454] Released schema lock
1211 20:52:45.190450 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":1442176,"child_traces":[["op",{"apply.queue_time_us":637,"apply.run_cpu_time_us":318,"apply.run_wall_time_us":319,"num_ops":1,"prepare.queue_time_us":6658,"prepare.run_cpu_time_us":493,"prepare.run_wall_time_us":492,"raft.queue_time_us":3481,"raft.run_cpu_time_us":7,"raft.run_wall_time_us":9,"replication_time_us":105794}]]}
I20251211 20:52:48.044730 1021 tablet_service.cc:1552] Processing DeleteTablet for tablet eb9da081cb9d43fd9a35c4bc51ea133e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:48.045740 2962 tablet_replica.cc:333] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:48.045965 2962 raft_consensus.cc:2243] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:48.046151 2962 raft_consensus.cc:2272] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:48.046747 2962 ts_tablet_manager.cc:1916] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:48.037402 1802 rpcz_store.cc:276] 1211 20:52:45.444211 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:47.733125 (+2288914us) service_pool.cc:225] Handling call
1211 20:52:47.733651 (+ 526us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
1211 20:52:47.733863 (+ 212us) catalog_manager.cc:3708] Apply alter schema
1211 20:52:47.734066 (+ 203us) catalog_manager.cc:3740] Acquired catalog manager lock
1211 20:52:47.734144 (+ 78us) catalog_manager.cc:3898] Updating metadata on disk
1211 20:52:47.742487 (+ 8343us) catalog_manager.cc:3929] Committing alterations to in-memory state
1211 20:52:48.037235 (+294748us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:47.734803 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:47.734923 (+ 120us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:47.734930 (+ 7us) write_op.cc:435] Acquired schema lock
1211 20:52:47.734933 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:47.735033 (+ 100us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:47.735049 (+ 16us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:47.735053 (+ 4us) tablet.cc:683] Acquiring locks for 1 operations
1211 20:52:47.735130 (+ 77us) tablet.cc:699] Row locks acquired
1211 20:52:47.735133 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:47.735168 (+ 35us) write_op.cc:270] Start()
1211 20:52:47.735202 (+ 34us) write_op.cc:276] Timestamp: P: 1765486367735130 usec, L: 0
1211 20:52:47.735206 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:47.735409 (+ 203us) log.cc:844] Serialized 405 byte log entry
1211 20:52:47.741630 (+ 6221us) op_driver.cc:464] REPLICATION: finished
1211 20:52:47.741830 (+ 200us) write_op.cc:301] APPLY: starting
1211 20:52:47.741847 (+ 17us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:47.741891 (+ 44us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:47.741930 (+ 39us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:47.741982 (+ 52us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:47.742019 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1211 20:52:47.742026 (+ 7us) write_op.cc:312] APPLY: finished
1211 20:52:47.742113 (+ 87us) log.cc:844] Serialized 23 byte log entry
1211 20:52:47.742380 (+ 267us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.742402 (+ 22us) write_op.cc:454] Released schema lock
1211 20:52:47.742410 (+ 8us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:47.742601 (+ 191us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.742605 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":146,"apply.run_cpu_time_us":472,"apply.run_wall_time_us":752,"num_ops":1,"prepare.queue_time_us":46,"prepare.run_cpu_time_us":710,"prepare.run_wall_time_us":711,"raft.queue_time_us":1137,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":6392}]]}
W20251211 20:52:48.053946 1797 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_18f6559b16734cbaa5659427c4276f5d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:48.053987 1795 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Sending DeleteTablet for 3 replicas of tablet bb9fccb6981e446a939a28d7fdd38e25
I20251211 20:52:48.054582 1795 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46568 (request call id 62) took 3087 ms. Trace:
I20251211 20:52:48.054623 757 tablet_service.cc:1552] Processing DeleteTablet for tablet eb9da081cb9d43fd9a35c4bc51ea133e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:48.054718 1795 rpcz_store.cc:276] 1211 20:52:44.967064 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:45.074488 (+107424us) service_pool.cc:225] Handling call
1211 20:52:45.074920 (+ 432us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
1211 20:52:45.075049 (+ 129us) catalog_manager.cc:2815] Modifying in-memory table state
1211 20:52:45.075113 (+ 64us) catalog_manager.cc:2824] Locking tablets
1211 20:52:45.075245 (+ 132us) catalog_manager.cc:2838] Removing table and tablets from system table
1211 20:52:45.190670 (+115425us) catalog_manager.cc:2856] Removing table from by-name map
1211 20:52:45.190744 (+ 74us) catalog_manager.cc:2872] Committing in-memory state
1211 20:52:48.039691 (+2848947us) catalog_manager.cc:2876] Aborting table tasks
1211 20:52:48.054540 (+ 14849us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:45.082832 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:45.082926 (+ 94us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:45.082931 (+ 5us) write_op.cc:435] Acquired schema lock
1211 20:52:45.082934 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:45.083027 (+ 93us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:45.083035 (+ 8us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:45.083039 (+ 4us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:45.083129 (+ 90us) tablet.cc:699] Row locks acquired
1211 20:52:45.083131 (+ 2us) write_op.cc:260] PREPARE: finished
1211 20:52:45.083157 (+ 26us) write_op.cc:270] Start()
1211 20:52:45.083179 (+ 22us) write_op.cc:276] Timestamp: P: 1765486365083123 usec, L: 0
1211 20:52:45.083183 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:45.083303 (+ 120us) log.cc:844] Serialized 756 byte log entry
1211 20:52:45.189440 (+106137us) op_driver.cc:464] REPLICATION: finished
1211 20:52:45.189585 (+ 145us) write_op.cc:301] APPLY: starting
1211 20:52:45.189604 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:45.189654 (+ 50us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:45.189706 (+ 52us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:45.189784 (+ 78us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:45.189822 (+ 38us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
1211 20:52:45.189829 (+ 7us) write_op.cc:312] APPLY: finished
1211 20:52:45.189926 (+ 97us) log.cc:844] Serialized 29 byte log entry
1211 20:52:45.190005 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:45.190026 (+ 21us) write_op.cc:454] Released schema lock
1211 20:52:45.190034 (+ 8us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:45.190118 (+ 84us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:45.190122 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":92,"apply.run_cpu_time_us":523,"apply.run_wall_time_us":523,"num_ops":2,"prepare.queue_time_us":6981,"prepare.run_cpu_time_us":548,"prepare.run_wall_time_us":547,"raft.queue_time_us":33,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":6,"replication_time_us":106217}]]}
I20251211 20:52:48.055451 1795 master_service.cc:954] ReplaceTablet: received request to replace tablet 5262c2241e524395a73c0eb368917169 from {username='slave'} at 127.0.0.1:46568
I20251211 20:52:48.055769 1160 raft_consensus.cc:359] T 75dd170c5828409fa69bd68fdf28a72b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.055914 1795 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46568 (request call id 82) took 1931 ms. Trace:
I20251211 20:52:48.055958 1160 raft_consensus.cc:385] T 75dd170c5828409fa69bd68fdf28a72b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:48.056010 1160 raft_consensus.cc:740] T 75dd170c5828409fa69bd68fdf28a72b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:48.056005 1795 rpcz_store.cc:276] 1211 20:52:46.124590 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:48.055384 (+1930794us) service_pool.cc:225] Handling call
1211 20:52:48.055838 (+ 454us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20251211 20:52:48.056248 1160 consensus_queue.cc:260] T 75dd170c5828409fa69bd68fdf28a72b P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.056351 2968 tablet_replica.cc:333] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:48.056536 2968 raft_consensus.cc:2243] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:48.056811 2968 raft_consensus.cc:2272] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:48.057184 2550 hybrid_clock.cc:648] HybridClock initialized: now 1765486368053117 us; error 3592 us; skew 500 ppm
I20251211 20:52:48.057358 2968 ts_tablet_manager.cc:1916] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:48.057435 1160 ts_tablet_manager.cc:1434] T 75dd170c5828409fa69bd68fdf28a72b P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.018s user 0.002s sys 0.000s
I20251211 20:52:48.057703 1160 tablet_bootstrap.cc:492] T 215f8dc74a71482daa2ca80e15010f51 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:48.057718 2550 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:52:48.059695 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "845b5ac4a5984c0cb4ab188925103dc9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:48.061997 757 tablet_service.cc:1552] Processing DeleteTablet for tablet bb9fccb6981e446a939a28d7fdd38e25 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:48.062098 1021 tablet_service.cc:1552] Processing DeleteTablet for tablet bb9fccb6981e446a939a28d7fdd38e25 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:48.062266 1166 ts_tablet_manager.cc:1434] T 834b84a557b04e78b5d2c8c84520a644 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.026s user 0.002s sys 0.000s
I20251211 20:52:48.062408 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "94658fcf512a48d0a992a7e2e0f7c176" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:48.062932 906 raft_consensus.cc:3060] T 3e6f62c2deff4c11b137fc0cba5ee234 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.063772 907 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "06b8f8e46e524545aeab6c931efaf116" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:48.063987 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71dc8d20d93243e289330d7b8c635511" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:48.064241 910 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd43a27db69044b68826cda87e16a086" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:48.064630 903 raft_consensus.cc:3060] T 04086eaa32a74edd84235d3b28f9f965 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.065096 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3621735fcb294e3a8c0180ba9ef86442" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:48.065182 1795 master_service.cc:954] ReplaceTablet: received request to replace tablet 373d3f03d7e0408cbe06955f8f60b16c from {username='slave'} at 127.0.0.1:46568
I20251211 20:52:48.065605 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "497c10742f01476db885bd70c4d2c491" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:48.066398 1806 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46568 (request call id 78) took 2611 ms. Trace:
I20251211 20:52:48.067858 1806 rpcz_store.cc:276] 1211 20:52:45.444331 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:48.033136 (+2588805us) service_pool.cc:225] Handling call
1211 20:52:48.034090 (+ 954us) catalog_manager.cc:2101] Acquired catalog manager lock
1211 20:52:48.034551 (+ 461us) catalog_manager.cc:2147] Created new table and tablet info
1211 20:52:48.055992 (+ 21441us) catalog_manager.cc:2207] Wrote table and tablets to system table
1211 20:52:48.056090 (+ 98us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
1211 20:52:48.056189 (+ 99us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:48.035620 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:48.035748 (+ 128us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:48.035755 (+ 7us) write_op.cc:435] Acquired schema lock
1211 20:52:48.035758 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:48.035847 (+ 89us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:48.035862 (+ 15us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:48.035866 (+ 4us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:48.035955 (+ 89us) tablet.cc:699] Row locks acquired
1211 20:52:48.035957 (+ 2us) write_op.cc:260] PREPARE: finished
1211 20:52:48.035995 (+ 38us) write_op.cc:270] Start()
1211 20:52:48.036031 (+ 36us) write_op.cc:276] Timestamp: P: 1765486368035973 usec, L: 0
1211 20:52:48.036034 (+ 3us) op_driver.cc:348] REPLICATION: starting
1211 20:52:48.036227 (+ 193us) log.cc:844] Serialized 433 byte log entry
1211 20:52:48.054523 (+ 18296us) op_driver.cc:464] REPLICATION: finished
1211 20:52:48.054694 (+ 171us) write_op.cc:301] APPLY: starting
1211 20:52:48.054716 (+ 22us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:48.054765 (+ 49us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:48.054799 (+ 34us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:48.054933 (+ 134us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:48.054976 (+ 43us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1211 20:52:48.054983 (+ 7us) write_op.cc:312] APPLY: finished
1211 20:52:48.055077 (+ 94us) log.cc:844] Serialized 29 byte log entry
1211 20:52:48.055149 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.055175 (+ 26us) write_op.cc:454] Released schema lock
1211 20:52:48.055183 (+ 8us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:48.055276 (+ 93us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.055280 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":112,"apply.run_cpu_time_us":569,"apply.run_wall_time_us":568,"num_ops":2,"prepare.queue_time_us":526,"prepare.run_cpu_time_us":753,"prepare.run_wall_time_us":754,"raft.queue_time_us":760,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":227,"replication_time_us":18452}]]}
I20251211 20:52:48.069800 1806 master_service.cc:954] ReplaceTablet: received request to replace tablet 903c7f4b312d4677bb35be1aaa46ad6b from {username='slave'} at 127.0.0.1:46568
I20251211 20:52:48.073069 1802 master_service.cc:954] ReplaceTablet: received request to replace tablet 4201da4241aa48dfb670a713e4453ca5 from {username='slave'} at 127.0.0.1:46568
I20251211 20:52:48.073524 1802 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46568 (request call id 81) took 2068 ms. Trace:
W20251211 20:52:48.074851 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet eb9da081cb9d43fd9a35c4bc51ea133e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eb9da081cb9d43fd9a35c4bc51ea133e already in progress: creating tablet
I20251211 20:52:48.075670 889 tablet_service.cc:1552] Processing DeleteTablet for tablet bb9fccb6981e446a939a28d7fdd38e25 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:48.076910 1799 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_bffc8b16e4414214b9e8b77194575e67"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:48.077530 1799 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bffc8b16e4414214b9e8b77194575e67 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:48.082188 1166 tablet_bootstrap.cc:492] T c081396b492049b996ac4661e1318388 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:48.084131 1166 tablet_bootstrap.cc:654] T c081396b492049b996ac4661e1318388 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:48.089172 1168 tablet_bootstrap.cc:654] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:48.090395 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 60064.4us
I20251211 20:52:48.091183 710 leader_election.cc:304] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:48.091439 2969 raft_consensus.cc:2804] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:48.091507 2969 raft_consensus.cc:493] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:48.091570 2969 raft_consensus.cc:3060] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:48.091558 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet bb9fccb6981e446a939a28d7fdd38e25 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb9fccb6981e446a939a28d7fdd38e25 already in progress: creating tablet
I20251211 20:52:48.094264 2969 raft_consensus.cc:515] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.095189 2969 leader_election.cc:290] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:48.076721 1802 rpcz_store.cc:276] 1211 20:52:46.005449 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:48.053057 (+2047608us) service_pool.cc:225] Handling call
1211 20:52:48.073482 (+ 20425us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20251211 20:52:48.097136 1802 master_service.cc:954] ReplaceTablet: received request to replace tablet c8bb92fc25124ba4be887c453abc7d9a from {username='slave'} at 127.0.0.1:46568
I20251211 20:52:48.107173 1795 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Sending DeleteTablet for 3 replicas of tablet 373d3f03d7e0408cbe06955f8f60b16c
I20251211 20:52:48.107260 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "94658fcf512a48d0a992a7e2e0f7c176" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:48.107461 1049 raft_consensus.cc:2468] T 94658fcf512a48d0a992a7e2e0f7c176 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:48.109429 1160 tablet_bootstrap.cc:654] T 215f8dc74a71482daa2ca80e15010f51 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:48.112937 1160 tablet_bootstrap.cc:492] T 215f8dc74a71482daa2ca80e15010f51 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:48.113111 1160 ts_tablet_manager.cc:1403] T 215f8dc74a71482daa2ca80e15010f51 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.055s user 0.002s sys 0.001s
I20251211 20:52:48.114135 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 373d3f03d7e0408cbe06955f8f60b16c with delete_type TABLET_DATA_DELETED (replaced by tablet 9ae2a68a1d0c45ae93d42472221bdc65) from {username='slave'} at 127.0.0.1:50110
W20251211 20:52:48.114967 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 373d3f03d7e0408cbe06955f8f60b16c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 373d3f03d7e0408cbe06955f8f60b16c already in progress: creating tablet
I20251211 20:52:48.116333 1166 tablet_bootstrap.cc:492] T c081396b492049b996ac4661e1318388 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:48.116508 1166 ts_tablet_manager.cc:1403] T c081396b492049b996ac4661e1318388 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.000s
I20251211 20:52:48.117323 1166 raft_consensus.cc:359] T c081396b492049b996ac4661e1318388 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.117507 1166 raft_consensus.cc:385] T c081396b492049b996ac4661e1318388 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:48.117555 1166 raft_consensus.cc:740] T c081396b492049b996ac4661e1318388 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:48.121742 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 373d3f03d7e0408cbe06955f8f60b16c with delete_type TABLET_DATA_DELETED (replaced by tablet 9ae2a68a1d0c45ae93d42472221bdc65) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:48.122920 1160 raft_consensus.cc:359] T 215f8dc74a71482daa2ca80e15010f51 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.123409 1021 tablet_service.cc:1552] Processing DeleteTablet for tablet 373d3f03d7e0408cbe06955f8f60b16c with delete_type TABLET_DATA_DELETED (replaced by tablet 9ae2a68a1d0c45ae93d42472221bdc65) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:48.123646 1160 raft_consensus.cc:385] T 215f8dc74a71482daa2ca80e15010f51 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:48.123733 1160 raft_consensus.cc:740] T 215f8dc74a71482daa2ca80e15010f51 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:48.124008 1160 consensus_queue.cc:260] T 215f8dc74a71482daa2ca80e15010f51 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.124353 1795 catalog_manager.cc:6724] ReplaceTablet: tablet 373d3f03d7e0408cbe06955f8f60b16c deleted and replaced by tablet 9ae2a68a1d0c45ae93d42472221bdc65
I20251211 20:52:48.126171 1795 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46568 (request call id 83) took 1904 ms. Trace:
I20251211 20:52:48.126652 1160 ts_tablet_manager.cc:1434] T 215f8dc74a71482daa2ca80e15010f51 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.013s user 0.003s sys 0.000s
I20251211 20:52:48.127816 1160 tablet_bootstrap.cc:492] T bbe3529c301a4a65b58b634a443e1fb6 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:48.129520 1160 tablet_bootstrap.cc:654] T bbe3529c301a4a65b58b634a443e1fb6 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:48.132647 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "06b8f8e46e524545aeab6c931efaf116" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:48.132822 1048 raft_consensus.cc:3060] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.132987 1036 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "497c10742f01476db885bd70c4d2c491" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:48.133116 1036 raft_consensus.cc:3060] T 497c10742f01476db885bd70c4d2c491 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.135425 1048 raft_consensus.cc:2468] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:48.136289 1036 raft_consensus.cc:2468] T 497c10742f01476db885bd70c4d2c491 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:48.132766 2550 webserver.cc:492] Webserver started at http://127.0.104.60:40729/ using document root <none> and password file <none>
I20251211 20:52:48.138998 2550 fs_manager.cc:362] Metadata directory not provided
I20251211 20:52:48.139160 2550 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:52:48.127701 1795 rpcz_store.cc:276] 1211 20:52:46.221998 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:48.056173 (+1834175us) service_pool.cc:225] Handling call
1211 20:52:48.126122 (+ 69949us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:48.066729 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:48.066868 (+ 139us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:48.066875 (+ 7us) write_op.cc:435] Acquired schema lock
1211 20:52:48.066878 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:48.066998 (+ 120us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:48.067013 (+ 15us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:48.067018 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:48.067094 (+ 76us) tablet.cc:699] Row locks acquired
1211 20:52:48.067097 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:48.067135 (+ 38us) write_op.cc:270] Start()
1211 20:52:48.067171 (+ 36us) write_op.cc:276] Timestamp: P: 1765486368067110 usec, L: 0
1211 20:52:48.067174 (+ 3us) op_driver.cc:348] REPLICATION: starting
1211 20:52:48.068424 (+ 1250us) log.cc:844] Serialized 622 byte log entry
1211 20:52:48.087047 (+ 18623us) op_driver.cc:464] REPLICATION: finished
1211 20:52:48.087172 (+ 125us) write_op.cc:301] APPLY: starting
1211 20:52:48.087188 (+ 16us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:48.087236 (+ 48us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:48.087284 (+ 48us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:48.087397 (+ 113us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:48.087453 (+ 56us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1211 20:52:48.087460 (+ 7us) write_op.cc:312] APPLY: finished
1211 20:52:48.087557 (+ 97us) log.cc:844] Serialized 29 byte log entry
1211 20:52:48.087660 (+ 103us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.087684 (+ 24us) write_op.cc:454] Released schema lock
1211 20:52:48.087692 (+ 8us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:48.087770 (+ 78us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.087774 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":80,"apply.run_cpu_time_us":584,"apply.run_wall_time_us":584,"num_ops":2,"prepare.queue_time_us":335,"prepare.run_cpu_time_us":801,"prepare.run_wall_time_us":1820,"raft.queue_time_us":919,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":1788,"replication_time_us":19839}]]}
I20251211 20:52:48.147331 1036 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71dc8d20d93243e289330d7b8c635511" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:48.147538 1036 raft_consensus.cc:3060] T 71dc8d20d93243e289330d7b8c635511 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.150365 1036 raft_consensus.cc:2468] T 71dc8d20d93243e289330d7b8c635511 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
W20251211 20:52:48.151118 711 consensus_peers.cc:597] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:48.151510 711 consensus_peers.cc:597] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:48.151793 711 leader_election.cc:343] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:48.152688 711 leader_election.cc:343] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:48.153364 711 consensus_peers.cc:597] T f25bde14792248b188660036f07ae902 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:48.156481 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3621735fcb294e3a8c0180ba9ef86442" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:48.156903 1049 raft_consensus.cc:3060] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.163110 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd43a27db69044b68826cda87e16a086" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:48.163309 1044 raft_consensus.cc:3060] T fd43a27db69044b68826cda87e16a086 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.165277 1037 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "845b5ac4a5984c0cb4ab188925103dc9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:48.165472 1037 raft_consensus.cc:3060] T 845b5ac4a5984c0cb4ab188925103dc9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.165838 1044 raft_consensus.cc:2468] T fd43a27db69044b68826cda87e16a086 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:48.168416 1037 raft_consensus.cc:2468] T 845b5ac4a5984c0cb4ab188925103dc9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:48.145643 1797 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46568 (request call id 79) took 2509 ms. Trace:
I20251211 20:52:48.173034 1797 rpcz_store.cc:276] 1211 20:52:45.636435 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:48.038774 (+2402339us) service_pool.cc:225] Handling call
1211 20:52:48.054090 (+ 15316us) catalog_manager.cc:2101] Acquired catalog manager lock
1211 20:52:48.088295 (+ 34205us) catalog_manager.cc:2147] Created new table and tablet info
1211 20:52:48.145369 (+ 57074us) catalog_manager.cc:2207] Wrote table and tablets to system table
1211 20:52:48.145492 (+ 123us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
1211 20:52:48.145603 (+ 111us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:48.090356 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:48.090463 (+ 107us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:48.090468 (+ 5us) write_op.cc:435] Acquired schema lock
1211 20:52:48.090470 (+ 2us) tablet.cc:660] Decoding operations
1211 20:52:48.090561 (+ 91us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:48.090571 (+ 10us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:48.090575 (+ 4us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:48.090654 (+ 79us) tablet.cc:699] Row locks acquired
1211 20:52:48.090657 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:48.090688 (+ 31us) write_op.cc:270] Start()
1211 20:52:48.090716 (+ 28us) write_op.cc:276] Timestamp: P: 1765486368090661 usec, L: 0
1211 20:52:48.090720 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:48.090869 (+ 149us) log.cc:844] Serialized 433 byte log entry
1211 20:52:48.143493 (+ 52624us) op_driver.cc:464] REPLICATION: finished
1211 20:52:48.144375 (+ 882us) write_op.cc:301] APPLY: starting
1211 20:52:48.144384 (+ 9us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:48.144416 (+ 32us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:48.144445 (+ 29us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:48.144516 (+ 71us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:48.144541 (+ 25us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1211 20:52:48.144548 (+ 7us) write_op.cc:312] APPLY: finished
1211 20:52:48.144612 (+ 64us) log.cc:844] Serialized 29 byte log entry
1211 20:52:48.144649 (+ 37us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.144663 (+ 14us) write_op.cc:454] Released schema lock
1211 20:52:48.144667 (+ 4us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:48.144724 (+ 57us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.144728 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":853,"apply.run_cpu_time_us":343,"apply.run_wall_time_us":342,"num_ops":2,"prepare.queue_time_us":1500,"prepare.run_cpu_time_us":593,"prepare.run_wall_time_us":593,"raft.queue_time_us":101,"raft.run_cpu_time_us":2,"raft.run_wall_time_us":2,"replication_time_us":52756}]]}
I20251211 20:52:48.150378 1799 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46568 (request call id 80) took 2490 ms. Trace:
I20251211 20:52:48.174518 1799 rpcz_store.cc:276] 1211 20:52:45.660287 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:48.044420 (+2384133us) service_pool.cc:225] Handling call
1211 20:52:48.077643 (+ 33223us) catalog_manager.cc:2101] Acquired catalog manager lock
1211 20:52:48.078122 (+ 479us) catalog_manager.cc:2147] Created new table and tablet info
1211 20:52:48.150081 (+ 71959us) catalog_manager.cc:2207] Wrote table and tablets to system table
1211 20:52:48.150208 (+ 127us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
1211 20:52:48.150312 (+ 104us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:48.089582 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:48.089719 (+ 137us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:48.089725 (+ 6us) write_op.cc:435] Acquired schema lock
1211 20:52:48.089728 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:48.089823 (+ 95us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:48.089836 (+ 13us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:48.089840 (+ 4us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:48.089942 (+ 102us) tablet.cc:699] Row locks acquired
1211 20:52:48.089944 (+ 2us) write_op.cc:260] PREPARE: finished
1211 20:52:48.089982 (+ 38us) write_op.cc:270] Start()
1211 20:52:48.090015 (+ 33us) write_op.cc:276] Timestamp: P: 1765486368089954 usec, L: 0
1211 20:52:48.090018 (+ 3us) op_driver.cc:348] REPLICATION: starting
1211 20:52:48.090233 (+ 215us) log.cc:844] Serialized 433 byte log entry
1211 20:52:48.143407 (+ 53174us) op_driver.cc:464] REPLICATION: finished
1211 20:52:48.143783 (+ 376us) write_op.cc:301] APPLY: starting
1211 20:52:48.143804 (+ 21us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:48.143856 (+ 52us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:48.143892 (+ 36us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:48.144004 (+ 112us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:48.144042 (+ 38us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1211 20:52:48.144050 (+ 8us) write_op.cc:312] APPLY: finished
1211 20:52:48.144138 (+ 88us) log.cc:844] Serialized 29 byte log entry
1211 20:52:48.144212 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.144232 (+ 20us) write_op.cc:454] Released schema lock
1211 20:52:48.144239 (+ 7us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:48.144308 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.144311 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":303,"apply.run_cpu_time_us":530,"apply.run_wall_time_us":529,"num_ops":2,"prepare.queue_time_us":10957,"prepare.run_cpu_time_us":749,"prepare.run_wall_time_us":750,"raft.queue_time_us":698,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":53344}]]}
I20251211 20:52:48.175262 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "be876471f8bf453aa4467c91891cbbde" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:48.152281 1806 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Sending DeleteTablet for 3 replicas of tablet 903c7f4b312d4677bb35be1aaa46ad6b
I20251211 20:52:48.176362 1806 catalog_manager.cc:6724] ReplaceTablet: tablet 903c7f4b312d4677bb35be1aaa46ad6b deleted and replaced by tablet 200fbba33f8947cca99ddc81e37dfc28
I20251211 20:52:48.176741 1806 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46568 (request call id 84) took 1921 ms. Trace:
I20251211 20:52:48.176828 1806 rpcz_store.cc:276] 1211 20:52:46.255328 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:48.069741 (+1814413us) service_pool.cc:225] Handling call
1211 20:52:48.176688 (+106947us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:48.099118 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:48.099249 (+ 131us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:48.099256 (+ 7us) write_op.cc:435] Acquired schema lock
1211 20:52:48.099259 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:48.099375 (+ 116us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:48.099387 (+ 12us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:48.099392 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:48.099494 (+ 102us) tablet.cc:699] Row locks acquired
1211 20:52:48.099497 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:48.099537 (+ 40us) write_op.cc:270] Start()
1211 20:52:48.099573 (+ 36us) write_op.cc:276] Timestamp: P: 1765486368099508 usec, L: 0
1211 20:52:48.099604 (+ 31us) op_driver.cc:348] REPLICATION: starting
1211 20:52:48.099806 (+ 202us) log.cc:844] Serialized 622 byte log entry
1211 20:52:48.143544 (+ 43738us) op_driver.cc:464] REPLICATION: finished
1211 20:52:48.144779 (+ 1235us) write_op.cc:301] APPLY: starting
1211 20:52:48.144788 (+ 9us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:48.144813 (+ 25us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:48.144832 (+ 19us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:48.144898 (+ 66us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:48.144945 (+ 47us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1211 20:52:48.144951 (+ 6us) write_op.cc:312] APPLY: finished
1211 20:52:48.144996 (+ 45us) log.cc:844] Serialized 29 byte log entry
1211 20:52:48.145029 (+ 33us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.145041 (+ 12us) write_op.cc:454] Released schema lock
1211 20:52:48.145045 (+ 4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1209,"apply.run_cpu_time_us":312,"apply.run_wall_time_us":311,"num_ops":2,"prepare.queue_time_us":2225,"prepare.run_cpu_time_us":801,"prepare.run_wall_time_us":863,"raft.queue_time_us":38,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":6,"replication_time_us":43926,"threads_started":1}]]}
I20251211 20:52:48.178090 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 903c7f4b312d4677bb35be1aaa46ad6b with delete_type TABLET_DATA_DELETED (replaced by tablet 200fbba33f8947cca99ddc81e37dfc28) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:48.178225 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 903c7f4b312d4677bb35be1aaa46ad6b with delete_type TABLET_DATA_DELETED (replaced by tablet 200fbba33f8947cca99ddc81e37dfc28) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:48.178546 1021 tablet_service.cc:1552] Processing DeleteTablet for tablet 903c7f4b312d4677bb35be1aaa46ad6b with delete_type TABLET_DATA_DELETED (replaced by tablet 200fbba33f8947cca99ddc81e37dfc28) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:48.119511 1166 consensus_queue.cc:260] T c081396b492049b996ac4661e1318388 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:48.179046 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 903c7f4b312d4677bb35be1aaa46ad6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 903c7f4b312d4677bb35be1aaa46ad6b already in progress: creating tablet
I20251211 20:52:48.179420 1166 ts_tablet_manager.cc:1434] T c081396b492049b996ac4661e1318388 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.063s user 0.002s sys 0.000s
I20251211 20:52:48.181059 1166 tablet_bootstrap.cc:492] T f91487dd7e0f4c8b8483c2b83945da3b P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:48.183517 1166 tablet_bootstrap.cc:654] T f91487dd7e0f4c8b8483c2b83945da3b P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:48.189918 2962 ts_tablet_manager.cc:1929] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:48.190232 2962 log.cc:1199] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/eb9da081cb9d43fd9a35c4bc51ea133e
I20251211 20:52:48.190796 2962 ts_tablet_manager.cc:1950] T eb9da081cb9d43fd9a35c4bc51ea133e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:48.191521 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "32f8d104f6a948d1a97a83769ea82449" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:48.192485 1049 raft_consensus.cc:2468] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:48.197063 906 raft_consensus.cc:3060] T 6b7dd8232d974a43b99ec9b28075dc8d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.201196 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "845b5ac4a5984c0cb4ab188925103dc9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:48.199746 711 consensus_peers.cc:597] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:48.202598 711 leader_election.cc:343] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:48.202935 711 consensus_peers.cc:597] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:48.203982 711 consensus_peers.cc:597] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:48.204126 711 consensus_peers.cc:597] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:48.206962 3000 consensus_queue.cc:1048] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:48.207459 710 leader_election.cc:304] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:48.207710 3000 raft_consensus.cc:2804] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:48.207777 3000 raft_consensus.cc:493] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:48.207827 3000 raft_consensus.cc:3060] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.211930 710 leader_election.cc:304] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:48.213655 2959 consensus_queue.cc:1048] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:48.214085 2959 raft_consensus.cc:2804] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:48.214182 2959 raft_consensus.cc:697] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:48.214380 2959 consensus_queue.cc:237] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:48.215178 711 leader_election.cc:343] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:48.215418 711 leader_election.cc:343] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:48.215730 711 consensus_peers.cc:597] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:48.215896 711 leader_election.cc:343] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:48.216230 711 consensus_peers.cc:597] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:48.217056 711 consensus_peers.cc:597] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:48.218171 711 leader_election.cc:343] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:48.218835 711 consensus_peers.cc:597] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:48.218958 711 consensus_peers.cc:597] T 95866de91f9d437d80f40b7962253610 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:48.219050 711 consensus_peers.cc:597] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:48.219141 711 leader_election.cc:343] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:48.219480 711 leader_election.cc:343] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:48.219725 711 leader_election.cc:343] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.219782 2968 ts_tablet_manager.cc:1929] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:48.219905 2968 log.cc:1199] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/eb9da081cb9d43fd9a35c4bc51ea133e
W20251211 20:52:48.219923 711 leader_election.cc:343] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.225024 1160 tablet_bootstrap.cc:492] T bbe3529c301a4a65b58b634a443e1fb6 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:48.225207 1160 ts_tablet_manager.cc:1403] T bbe3529c301a4a65b58b634a443e1fb6 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.097s user 0.002s sys 0.001s
I20251211 20:52:48.226112 1160 raft_consensus.cc:359] T bbe3529c301a4a65b58b634a443e1fb6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.226320 1160 raft_consensus.cc:385] T bbe3529c301a4a65b58b634a443e1fb6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:48.226372 1160 raft_consensus.cc:740] T bbe3529c301a4a65b58b634a443e1fb6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:48.217555 710 leader_election.cc:304] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:48.226584 1160 consensus_queue.cc:260] T bbe3529c301a4a65b58b634a443e1fb6 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.227334 2968 ts_tablet_manager.cc:1950] T eb9da081cb9d43fd9a35c4bc51ea133e P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:48.227942 1160 ts_tablet_manager.cc:1434] T bbe3529c301a4a65b58b634a443e1fb6 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:48.228156 2736 raft_consensus.cc:2804] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:48.228235 1160 tablet_bootstrap.cc:492] T eb9da081cb9d43fd9a35c4bc51ea133e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:48.228267 2736 raft_consensus.cc:697] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:48.228475 2736 consensus_queue.cc:237] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.229347 2964 raft_consensus.cc:493] T 215f8dc74a71482daa2ca80e15010f51 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:48.229470 2964 raft_consensus.cc:515] T 215f8dc74a71482daa2ca80e15010f51 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.229988 2964 leader_election.cc:290] T 215f8dc74a71482daa2ca80e15010f51 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:48.231384 710 leader_election.cc:304] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:48.231930 2955 raft_consensus.cc:2804] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:48.232036 2955 raft_consensus.cc:697] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:48.234715 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet eb9da081cb9d43fd9a35c4bc51ea133e (table default.Table_b43dda4aa0674e3f8ef0d55e0e2431e3 [id=e0cd4de0a942462d9e3b50079f6b757d]) successfully deleted
I20251211 20:52:48.234787 2968 tablet_replica.cc:333] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:48.235013 2968 raft_consensus.cc:2243] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
W20251211 20:52:48.235054 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet bb9fccb6981e446a939a28d7fdd38e25 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb9fccb6981e446a939a28d7fdd38e25 already in progress: creating tablet
I20251211 20:52:48.235216 2968 pending_rounds.cc:70] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919: Trying to abort 1 pending ops.
I20251211 20:52:48.235255 2968 pending_rounds.cc:77] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432159489486848 op_type: NO_OP noop_request { }
I20251211 20:52:48.235394 2968 raft_consensus.cc:2889] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:48.235494 2968 raft_consensus.cc:2272] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:48.236028 3000 raft_consensus.cc:515] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.236719 3000 leader_election.cc:290] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:48.237119 1168 tablet_bootstrap.cc:492] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:48.237246 1168 ts_tablet_manager.cc:1403] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.216s user 0.004s sys 0.006s
I20251211 20:52:48.237900 1168 raft_consensus.cc:359] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.238037 1168 raft_consensus.cc:385] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:48.238075 1168 raft_consensus.cc:740] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:48.238226 1168 consensus_queue.cc:260] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.238679 1168 ts_tablet_manager.cc:1434] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251211 20:52:48.238937 1168 tablet_bootstrap.cc:492] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:48.239878 711 consensus_peers.cc:597] T a7979798e92e45eb969197c8c7ac2580 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:48.240100 711 leader_election.cc:343] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.239794 2955 consensus_queue.cc:237] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:48.240471 711 leader_election.cc:343] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:48.241624 711 consensus_peers.cc:597] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:48.241921 711 leader_election.cc:343] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.243165 710 leader_election.cc:304] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:48.244212 3029 tablet.cc:1722] T d37a7d0d662e45079b0cfc02765d2e01 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:48.245704 2494 raft_consensus.cc:2804] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:48.245813 2494 raft_consensus.cc:697] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:48.246029 2494 consensus_queue.cc:237] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.247430 2962 tablet_replica.cc:333] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:48.247707 2962 raft_consensus.cc:2243] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:48.247855 2962 raft_consensus.cc:2272] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:48.248500 2962 ts_tablet_manager.cc:1916] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:48.248529 2806 consensus_queue.cc:1048] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:48.249869 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "94658fcf512a48d0a992a7e2e0f7c176" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:48.253592 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet eb9da081cb9d43fd9a35c4bc51ea133e (table default.Table_b43dda4aa0674e3f8ef0d55e0e2431e3 [id=e0cd4de0a942462d9e3b50079f6b757d]) successfully deleted
W20251211 20:52:48.253898 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet bb9fccb6981e446a939a28d7fdd38e25 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb9fccb6981e446a939a28d7fdd38e25 already in progress: creating tablet
W20251211 20:52:48.255247 711 consensus_peers.cc:597] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:48.255208 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "215f8dc74a71482daa2ca80e15010f51" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:48.276314 1799 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_25d3985a897841b8b55ad7c44bbff154" } modify_external_catalogs: true
I20251211 20:52:48.276556 1799 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_25d3985a897841b8b55ad7c44bbff154" } modify_external_catalogs: true
I20251211 20:52:48.277170 1806 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_440d3f9c240c448bb5bf5bfbdaf35d00" } modify_external_catalogs: true
I20251211 20:52:48.277314 1806 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_440d3f9c240c448bb5bf5bfbdaf35d00" } modify_external_catalogs: true
I20251211 20:52:48.277837 1797 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_03298b88200d42148accc9f0dffbf739" } new_table_name: "default.Table_16a12ed370b64a38b5e145c4eb024349" modify_external_catalogs: true
I20251211 20:52:48.279772 710 leader_election.cc:304] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:48.280025 2806 raft_consensus.cc:2804] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:48.280148 2806 raft_consensus.cc:697] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:48.280400 2806 consensus_queue.cc:237] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.280534 1795 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_310ce452c2a54f069cf6c65f233c0603" } new_table_name: "default.Table_00ae5c111de94ba0a2680ad22c726e16" modify_external_catalogs: true
W20251211 20:52:48.283548 711 leader_election.cc:343] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.284920 1802 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Sending DeleteTablet for 3 replicas of tablet c8bb92fc25124ba4be887c453abc7d9a
W20251211 20:52:48.284920 711 consensus_peers.cc:597] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:48.285421 1802 catalog_manager.cc:6724] ReplaceTablet: tablet c8bb92fc25124ba4be887c453abc7d9a deleted and replaced by tablet 80486aaeb28849c58c855475871853c0
I20251211 20:52:48.285555 1802 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46568 (request call id 85) took 1728 ms. Trace:
I20251211 20:52:48.285633 1802 rpcz_store.cc:276] 1211 20:52:46.556710 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:48.097059 (+1540349us) service_pool.cc:225] Handling call
1211 20:52:48.285503 (+188444us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:48.099993 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:48.100092 (+ 99us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:48.100097 (+ 5us) write_op.cc:435] Acquired schema lock
1211 20:52:48.100099 (+ 2us) tablet.cc:660] Decoding operations
1211 20:52:48.100202 (+ 103us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:48.100211 (+ 9us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:48.100215 (+ 4us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:48.100275 (+ 60us) tablet.cc:699] Row locks acquired
1211 20:52:48.100278 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:48.100308 (+ 30us) write_op.cc:270] Start()
1211 20:52:48.100336 (+ 28us) write_op.cc:276] Timestamp: P: 1765486368100282 usec, L: 0
1211 20:52:48.100340 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:48.100504 (+ 164us) log.cc:844] Serialized 622 byte log entry
1211 20:52:48.275878 (+175374us) op_driver.cc:464] REPLICATION: finished
1211 20:52:48.283839 (+ 7961us) write_op.cc:301] APPLY: starting
1211 20:52:48.283858 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:48.283918 (+ 60us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:48.283981 (+ 63us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:48.284170 (+ 189us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:48.284231 (+ 61us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1211 20:52:48.284239 (+ 8us) write_op.cc:312] APPLY: finished
1211 20:52:48.284347 (+ 108us) log.cc:844] Serialized 29 byte log entry
1211 20:52:48.284604 (+ 257us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.284634 (+ 30us) write_op.cc:454] Released schema lock
1211 20:52:48.284648 (+ 14us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":7847,"num_ops":2,"prepare.queue_time_us":1655,"prepare.run_cpu_time_us":590,"prepare.run_wall_time_us":656,"raft.queue_time_us":38,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":175504}]]}
I20251211 20:52:48.288776 889 tablet_service.cc:1552] Processing DeleteTablet for tablet c8bb92fc25124ba4be887c453abc7d9a with delete_type TABLET_DATA_DELETED (replaced by tablet 80486aaeb28849c58c855475871853c0) from {username='slave'} at 127.0.0.1:50110
W20251211 20:52:48.289698 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet c8bb92fc25124ba4be887c453abc7d9a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c8bb92fc25124ba4be887c453abc7d9a already in progress: creating tablet
I20251211 20:52:48.289723 757 tablet_service.cc:1552] Processing DeleteTablet for tablet c8bb92fc25124ba4be887c453abc7d9a with delete_type TABLET_DATA_DELETED (replaced by tablet 80486aaeb28849c58c855475871853c0) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:48.297616 710 leader_election.cc:304] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:48.298295 2970 raft_consensus.cc:2804] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:48.298432 2970 raft_consensus.cc:697] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:48.298696 2970 consensus_queue.cc:237] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.321310 2962 ts_tablet_manager.cc:1929] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:48.322064 2962 log.cc:1199] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/373d3f03d7e0408cbe06955f8f60b16c
I20251211 20:52:48.323419 2962 ts_tablet_manager.cc:1950] T 373d3f03d7e0408cbe06955f8f60b16c P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:48.327353 1809 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: default.Table_3721b5725e8c4297a392d3986a081128 [id=383392d835da4c50848068d83261cecc] alter complete (version 7)
I20251211 20:52:48.330168 2500 raft_consensus.cc:493] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:48.330319 2500 raft_consensus.cc:515] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.330885 2500 leader_election.cc:290] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:48.332054 906 raft_consensus.cc:3060] T bbe3529c301a4a65b58b634a443e1fb6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:48.195767 983 leader_election.cc:343] T be876471f8bf453aa4467c91891cbbde P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.335559 983 leader_election.cc:304] T be876471f8bf453aa4467c91891cbbde P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:48.335958 983 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 140197us
W20251211 20:52:48.336221 983 leader_election.cc:343] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.336310 983 leader_election.cc:304] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:48.336915 2948 raft_consensus.cc:2749] T be876471f8bf453aa4467c91891cbbde P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:48.337074 2948 raft_consensus.cc:2749] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:48.337627 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "84b2a67c836844ceb3628cc3b7d8ba10" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:48.338183 711 leader_election.cc:343] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.341988 1038 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "215f8dc74a71482daa2ca80e15010f51" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:48.343401 1043 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "94658fcf512a48d0a992a7e2e0f7c176" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:48.343562 1043 raft_consensus.cc:3060] T 94658fcf512a48d0a992a7e2e0f7c176 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.344756 1032 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "84b2a67c836844ceb3628cc3b7d8ba10" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:48.344907 1032 raft_consensus.cc:2468] T 84b2a67c836844ceb3628cc3b7d8ba10 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:48.344884 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 373d3f03d7e0408cbe06955f8f60b16c (table default.Table_43e9cb840d48444b9360e2fa8671dfb8 [id=f6cab1da32324fe6afc1df2007f0bdbf]) successfully deleted
W20251211 20:52:48.345191 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 903c7f4b312d4677bb35be1aaa46ad6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 903c7f4b312d4677bb35be1aaa46ad6b already in progress: creating tablet
I20251211 20:52:48.346155 1043 raft_consensus.cc:2468] T 94658fcf512a48d0a992a7e2e0f7c176 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:48.351296 847 leader_election.cc:304] T 215f8dc74a71482daa2ca80e15010f51 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:48.351637 2964 raft_consensus.cc:2749] T 215f8dc74a71482daa2ca80e15010f51 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:48.353802 2969 consensus_queue.cc:1048] T d69fb1f6d9594afcbfb844860b41a377 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:48.356271 908 raft_consensus.cc:3060] T 75dd170c5828409fa69bd68fdf28a72b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.362457 2948 raft_consensus.cc:493] T 962b80ae747b4470abd23643e711fad9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:48.362653 2948 raft_consensus.cc:515] T 962b80ae747b4470abd23643e711fad9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.363305 2948 leader_election.cc:290] T 962b80ae747b4470abd23643e711fad9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:48.363982 2948 raft_consensus.cc:493] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:48.364118 2948 raft_consensus.cc:515] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.364698 2948 leader_election.cc:290] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:48.364986 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "962b80ae747b4470abd23643e711fad9" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:48.365597 983 leader_election.cc:343] T 962b80ae747b4470abd23643e711fad9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.365923 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7b06cd366de5494780b5ac81abbf0702" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:48.366322 983 leader_election.cc:343] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.388038 1168 tablet_bootstrap.cc:654] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:48.427729 1096 heartbeater.cc:499] Master 127.0.104.61:40749 was elected leader, sending a full tablet report...
I20251211 20:52:48.429066 1021 tablet_service.cc:1552] Processing DeleteTablet for tablet c8bb92fc25124ba4be887c453abc7d9a with delete_type TABLET_DATA_DELETED (replaced by tablet 80486aaeb28849c58c855475871853c0) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:48.430382 2956 raft_consensus.cc:493] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:48.430632 2956 raft_consensus.cc:515] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.432178 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de508d79f30f489cb745401f53115165" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:48.432672 2956 leader_election.cc:290] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:48.432894 711 leader_election.cc:343] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.435267 710 leader_election.cc:304] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:48.436789 3054 raft_consensus.cc:2804] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:48.436896 3054 raft_consensus.cc:493] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:48.436959 3054 raft_consensus.cc:3060] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.436797 2429 consensus_queue.cc:1048] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:48.440788 710 leader_election.cc:304] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:48.440922 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet c8bb92fc25124ba4be887c453abc7d9a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c8bb92fc25124ba4be887c453abc7d9a already in progress: creating tablet
I20251211 20:52:48.441396 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "962b80ae747b4470abd23643e711fad9" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:48.441495 2808 raft_consensus.cc:2804] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:48.442167 3048 consensus_queue.cc:1048] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:48.442203 3054 raft_consensus.cc:515] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.442214 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7b06cd366de5494780b5ac81abbf0702" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:48.443248 3054 leader_election.cc:290] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:48.441937 2808 raft_consensus.cc:697] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:48.444438 2429 raft_consensus.cc:493] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:48.444566 2429 raft_consensus.cc:515] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.445129 2429 leader_election.cc:290] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:48.446061 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bd2bf1113dfe4b67b263b56f3d3db07d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:48.446713 711 leader_election.cc:343] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.447634 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "84b2a67c836844ceb3628cc3b7d8ba10" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:48.448159 711 leader_election.cc:343] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.444348 2808 consensus_queue.cc:237] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.458071 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de508d79f30f489cb745401f53115165" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:48.458289 1049 raft_consensus.cc:2468] T de508d79f30f489cb745401f53115165 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:48.459841 3069 tablet.cc:1722] T d37a7d0d662e45079b0cfc02765d2e01 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:48.467051 1031 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bd2bf1113dfe4b67b263b56f3d3db07d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:48.467092 1047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "84b2a67c836844ceb3628cc3b7d8ba10" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:48.467334 1031 raft_consensus.cc:2468] T bd2bf1113dfe4b67b263b56f3d3db07d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:48.467357 1047 raft_consensus.cc:3060] T 84b2a67c836844ceb3628cc3b7d8ba10 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.468264 710 leader_election.cc:304] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:48.478304 982 leader_election.cc:343] T 962b80ae747b4470abd23643e711fad9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.478485 982 leader_election.cc:304] T 962b80ae747b4470abd23643e711fad9 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:48.479126 2948 raft_consensus.cc:2749] T 962b80ae747b4470abd23643e711fad9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:48.479318 982 leader_election.cc:343] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.479418 982 leader_election.cc:304] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:48.483402 2948 raft_consensus.cc:2749] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:48.483419 2970 raft_consensus.cc:2804] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:48.483523 2970 raft_consensus.cc:493] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:48.483608 2970 raft_consensus.cc:3060] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.487066 1047 raft_consensus.cc:2468] T 84b2a67c836844ceb3628cc3b7d8ba10 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:48.488883 710 leader_election.cc:304] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:48.489745 710 leader_election.cc:304] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:48.490473 2956 raft_consensus.cc:2804] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:48.490585 2956 raft_consensus.cc:697] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:48.490826 2956 consensus_queue.cc:237] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.491938 2494 raft_consensus.cc:2804] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:48.492069 2494 raft_consensus.cc:493] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:48.492167 2494 raft_consensus.cc:3060] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.494771 2494 raft_consensus.cc:515] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.495661 2494 leader_election.cc:290] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:48.498383 1797 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bbe6e6220eb14e59a821805c438230c9 consensus state has no leader
I20251211 20:52:48.500113 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bd2bf1113dfe4b67b263b56f3d3db07d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:48.500483 1806 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Sending DeleteTablet for 3 replicas of tablet 09cd11f8b23c4f8a98a2b2eb77bb7b8c
I20251211 20:52:48.501215 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 09cd11f8b23c4f8a98a2b2eb77bb7b8c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:48 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:48.501751 1168 tablet_bootstrap.cc:492] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:48.501899 1168 ts_tablet_manager.cc:1403] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.263s user 0.009s sys 0.003s
W20251211 20:52:48.501920 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 09cd11f8b23c4f8a98a2b2eb77bb7b8c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09cd11f8b23c4f8a98a2b2eb77bb7b8c already in progress: creating tablet
I20251211 20:52:48.502760 1168 raft_consensus.cc:359] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.502761 1806 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_5e5286ea174048f8ba9abd14daf14bcf" } modify_external_catalogs: true
I20251211 20:52:48.502923 1168 raft_consensus.cc:385] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:48.502969 1168 raft_consensus.cc:740] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:48.503006 1806 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_5e5286ea174048f8ba9abd14daf14bcf" } modify_external_catalogs: true
I20251211 20:52:48.503183 1168 consensus_queue.cc:260] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.503795 1168 ts_tablet_manager.cc:1434] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:48.504055 1168 tablet_bootstrap.cc:492] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:48.504458 711 leader_election.cc:343] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.505267 1799 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Sending DeleteTablet for 3 replicas of tablet 5d225553d0984b6e9685b5841616c9d8
I20251211 20:52:48.505810 1168 tablet_bootstrap.cc:654] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:48.505981 1799 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_73e99aea9d194b0ebf4d4d052506595d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:48.506589 1799 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_73e99aea9d194b0ebf4d4d052506595d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:48.507936 2968 ts_tablet_manager.cc:1916] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:48.508100 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 5d225553d0984b6e9685b5841616c9d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:48 UTC) from {username='slave'} at 127.0.0.1:50110
W20251211 20:52:48.508797 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 5d225553d0984b6e9685b5841616c9d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5d225553d0984b6e9685b5841616c9d8 already in progress: creating tablet
I20251211 20:52:48.515213 889 tablet_service.cc:1505] Processing CreateTablet for tablet 612f11971cec4548b55133e3fe426988 (DEFAULT_TABLE table=default.Table_0d0d81ce88d44d8aaeb2cc8ad423568e [id=cca9ffe426eb448fb65b622a93581133]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:48.515688 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 612f11971cec4548b55133e3fe426988. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:48.518411 886 tablet_service.cc:1505] Processing CreateTablet for tablet 69401677ec934d74b0213c7b4abb7c2d (DEFAULT_TABLE table=default.Table_511715bb615a453fa603163df3152002 [id=7606e265d102417a89969fa2c703b7ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:48.518893 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69401677ec934d74b0213c7b4abb7c2d. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:48.519902 1783 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 7023afec769c4ca3b2242a225004d482 (table default.Table_d5276e3cc46d4331b36a8878680fc55e [id=3811f8235b2f44cca283401302d613d4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 7023afec769c4ca3b2242a225004d482 consensus state has no leader
I20251211 20:52:48.522176 891 tablet_service.cc:1505] Processing CreateTablet for tablet 09fc248144fc4370987bf06babf2cf28 (DEFAULT_TABLE table=default.Table_6b69717880834c989053981a48d5c683 [id=355ca319f6554c1aa79c8dc411a8e542]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:48.522609 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09fc248144fc4370987bf06babf2cf28. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:48.528043 888 tablet_service.cc:1505] Processing CreateTablet for tablet 9801d48a84ec4efd85771e996c734bf9 (DEFAULT_TABLE table=default.Table_1f238c6ab5774766965f06b677a6f5c8 [id=29a928c7969846139040e9fba233376e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:48.528360 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 09cd11f8b23c4f8a98a2b2eb77bb7b8c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:48 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:48.528496 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9801d48a84ec4efd85771e996c734bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:48.528800 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 5d225553d0984b6e9685b5841616c9d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:48 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:48.530761 2429 raft_consensus.cc:493] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:48.530920 2429 raft_consensus.cc:515] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.531984 2429 leader_election.cc:290] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:48.531932 887 tablet_service.cc:1505] Processing CreateTablet for tablet 6abac62c57394d8e88338ffe26d60e09 (DEFAULT_TABLE table=default.Table_6973147f8fd7408b9cca4506330b8105 [id=0c8bbe5b430b4705b0c03a2db9557db9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:48.532366 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6abac62c57394d8e88338ffe26d60e09. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:48.532819 1795 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c081396b492049b996ac4661e1318388 consensus state has no leader
W20251211 20:52:48.537513 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 47819.7us
I20251211 20:52:48.542070 757 tablet_service.cc:1505] Processing CreateTablet for tablet 612f11971cec4548b55133e3fe426988 (DEFAULT_TABLE table=default.Table_0d0d81ce88d44d8aaeb2cc8ad423568e [id=cca9ffe426eb448fb65b622a93581133]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:48.542537 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 612f11971cec4548b55133e3fe426988. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:48.543154 2970 raft_consensus.cc:515] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.544019 2970 leader_election.cc:290] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:48.549129 757 tablet_service.cc:1505] Processing CreateTablet for tablet 69401677ec934d74b0213c7b4abb7c2d (DEFAULT_TABLE table=default.Table_511715bb615a453fa603163df3152002 [id=7606e265d102417a89969fa2c703b7ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:48.549360 1021 tablet_service.cc:1552] Processing DeleteTablet for tablet 09cd11f8b23c4f8a98a2b2eb77bb7b8c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:48 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:48.549957 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69401677ec934d74b0213c7b4abb7c2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:48.550060 1021 tablet_service.cc:1552] Processing DeleteTablet for tablet 5d225553d0984b6e9685b5841616c9d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:48 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:48.550954 1021 tablet_service.cc:1505] Processing CreateTablet for tablet 612f11971cec4548b55133e3fe426988 (DEFAULT_TABLE table=default.Table_0d0d81ce88d44d8aaeb2cc8ad423568e [id=cca9ffe426eb448fb65b622a93581133]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:48.551365 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 612f11971cec4548b55133e3fe426988. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:48.563277 2948 raft_consensus.cc:493] T f9a3fdf17f274c15ac0bdfa9c174f649 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:48.563488 2948 raft_consensus.cc:515] T f9a3fdf17f274c15ac0bdfa9c174f649 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.564241 2948 leader_election.cc:290] T f9a3fdf17f274c15ac0bdfa9c174f649 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:48.571509 1015 tablet_service.cc:1505] Processing CreateTablet for tablet 09fc248144fc4370987bf06babf2cf28 (DEFAULT_TABLE table=default.Table_6b69717880834c989053981a48d5c683 [id=355ca319f6554c1aa79c8dc411a8e542]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:48.572068 1015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09fc248144fc4370987bf06babf2cf28. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:48.582008 2550 fs_manager.cc:714] Time spent opening directory manager: real 0.439s user 0.006s sys 0.001s
I20251211 20:52:48.587479 742 tablet_service.cc:1505] Processing CreateTablet for tablet 09fc248144fc4370987bf06babf2cf28 (DEFAULT_TABLE table=default.Table_6b69717880834c989053981a48d5c683 [id=355ca319f6554c1aa79c8dc411a8e542]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:48.587913 746 tablet_service.cc:1505] Processing CreateTablet for tablet 6abac62c57394d8e88338ffe26d60e09 (DEFAULT_TABLE table=default.Table_6973147f8fd7408b9cca4506330b8105 [id=0c8bbe5b430b4705b0c03a2db9557db9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:48.587479 756 tablet_service.cc:1505] Processing CreateTablet for tablet 9801d48a84ec4efd85771e996c734bf9 (DEFAULT_TABLE table=default.Table_1f238c6ab5774766965f06b677a6f5c8 [id=29a928c7969846139040e9fba233376e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:48.588004 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09fc248144fc4370987bf06babf2cf28. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:48.588392 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9801d48a84ec4efd85771e996c734bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:48.591703 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6abac62c57394d8e88338ffe26d60e09. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:48.606124 3037 log.cc:927] Time spent T d391d4e6a101462d8d1220f1e17d0daa P e715a811a82446ce8d5bb4aafd5b81c7: Append to log took a long time: real 0.149s user 0.000s sys 0.000s
I20251211 20:52:48.606302 1024 tablet_service.cc:1505] Processing CreateTablet for tablet 9801d48a84ec4efd85771e996c734bf9 (DEFAULT_TABLE table=default.Table_1f238c6ab5774766965f06b677a6f5c8 [id=29a928c7969846139040e9fba233376e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:48.606765 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9801d48a84ec4efd85771e996c734bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:48.607896 1019 tablet_service.cc:1505] Processing CreateTablet for tablet 239355c39c064819a3b992c3bc2ece9c (DEFAULT_TABLE table=default.Table_5b3f562b5129421a9d27a06bebc88fe4 [id=4f970446d3de437b9b811db0bd41659f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:48.608304 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 239355c39c064819a3b992c3bc2ece9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:48.609124 1037 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bd2bf1113dfe4b67b263b56f3d3db07d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:48.609202 1044 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79cc81e91e1846038621af7a0a9f7a2e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:48.609306 1037 raft_consensus.cc:3060] T bd2bf1113dfe4b67b263b56f3d3db07d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.609356 1044 raft_consensus.cc:2468] T 79cc81e91e1846038621af7a0a9f7a2e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:48.609758 1021 tablet_service.cc:1505] Processing CreateTablet for tablet 69401677ec934d74b0213c7b4abb7c2d (DEFAULT_TABLE table=default.Table_511715bb615a453fa603163df3152002 [id=7606e265d102417a89969fa2c703b7ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:48.610146 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69401677ec934d74b0213c7b4abb7c2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:48.612695 1037 raft_consensus.cc:2468] T bd2bf1113dfe4b67b263b56f3d3db07d P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:48.615867 2956 raft_consensus.cc:493] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:48.616011 2956 raft_consensus.cc:515] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.616575 2956 leader_election.cc:290] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:48.616828 1018 tablet_service.cc:1505] Processing CreateTablet for tablet 6abac62c57394d8e88338ffe26d60e09 (DEFAULT_TABLE table=default.Table_6973147f8fd7408b9cca4506330b8105 [id=0c8bbe5b430b4705b0c03a2db9557db9]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:48.621632 3068 log.cc:927] Time spent T a7979798e92e45eb969197c8c7ac2580 P e715a811a82446ce8d5bb4aafd5b81c7: Append to log took a long time: real 0.160s user 0.000s sys 0.000s
I20251211 20:52:48.624353 1035 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de508d79f30f489cb745401f53115165" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:48.624531 1035 raft_consensus.cc:3060] T de508d79f30f489cb745401f53115165 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.624867 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6abac62c57394d8e88338ffe26d60e09. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:48.636948 756 tablet_service.cc:1505] Processing CreateTablet for tablet 239355c39c064819a3b992c3bc2ece9c (DEFAULT_TABLE table=default.Table_5b3f562b5129421a9d27a06bebc88fe4 [id=4f970446d3de437b9b811db0bd41659f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:48.637552 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 239355c39c064819a3b992c3bc2ece9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:48.647936 1166 tablet_bootstrap.cc:492] T f91487dd7e0f4c8b8483c2b83945da3b P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:48.655711 3050 raft_consensus.cc:493] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:48.655892 3050 raft_consensus.cc:515] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.656608 3050 leader_election.cc:290] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:48.659298 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f9a3fdf17f274c15ac0bdfa9c174f649" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:48.654345 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1e13965b6361488eaeb7dd13b467ca65" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:48.669734 1042 raft_consensus.cc:2468] T 1e13965b6361488eaeb7dd13b467ca65 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:48.648202 1166 ts_tablet_manager.cc:1403] T f91487dd7e0f4c8b8483c2b83945da3b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.468s user 0.005s sys 0.003s
W20251211 20:52:48.654809 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet b52377c8ffd741a598e5328a00d876fe (table default.Table_686a26915b42421380037144900dd97c [id=8a1b0c9d641b4a3c83d1ca48576d110d], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet b52377c8ffd741a598e5328a00d876fe consensus state has no leader
I20251211 20:52:48.671643 1168 tablet_bootstrap.cc:492] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:48.671798 1168 ts_tablet_manager.cc:1403] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.168s user 0.019s sys 0.052s
I20251211 20:52:48.662297 1799 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46568 (request call id 91) took 1070 ms. Trace:
I20251211 20:52:48.672144 1799 rpcz_store.cc:276] 1211 20:52:47.591471 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:48.505781 (+914310us) service_pool.cc:225] Handling call
1211 20:52:48.506682 (+ 901us) catalog_manager.cc:2101] Acquired catalog manager lock
1211 20:52:48.507205 (+ 523us) catalog_manager.cc:2147] Created new table and tablet info
1211 20:52:48.567707 (+ 60502us) catalog_manager.cc:2207] Wrote table and tablets to system table
1211 20:52:48.662131 (+ 94424us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
1211 20:52:48.662253 (+ 122us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:48.560995 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:48.561141 (+ 146us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:48.561146 (+ 5us) write_op.cc:435] Acquired schema lock
1211 20:52:48.561149 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:48.561253 (+ 104us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:48.561270 (+ 17us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:48.561275 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:48.561361 (+ 86us) tablet.cc:699] Row locks acquired
1211 20:52:48.561364 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:48.561413 (+ 49us) write_op.cc:270] Start()
1211 20:52:48.561450 (+ 37us) write_op.cc:276] Timestamp: P: 1765486368561387 usec, L: 0
1211 20:52:48.561454 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:48.561674 (+ 220us) log.cc:844] Serialized 433 byte log entry
1211 20:52:48.566659 (+ 4985us) op_driver.cc:464] REPLICATION: finished
1211 20:52:48.566750 (+ 91us) write_op.cc:301] APPLY: starting
1211 20:52:48.566769 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:48.566821 (+ 52us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:48.566886 (+ 65us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:48.567021 (+ 135us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:48.567061 (+ 40us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1211 20:52:48.567069 (+ 8us) write_op.cc:312] APPLY: finished
1211 20:52:48.567162 (+ 93us) log.cc:844] Serialized 29 byte log entry
1211 20:52:48.567230 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.567256 (+ 26us) write_op.cc:454] Released schema lock
1211 20:52:48.567264 (+ 8us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:48.574143 (+ 6879us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.574153 (+ 10us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":36,"apply.run_cpu_time_us":576,"apply.run_wall_time_us":575,"num_ops":2,"prepare.queue_time_us":29495,"prepare.run_cpu_time_us":835,"prepare.run_wall_time_us":13115,"raft.queue_time_us":143,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":813,"replication_time_us":5168}]]}
I20251211 20:52:48.672520 1168 raft_consensus.cc:359] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.672711 1168 raft_consensus.cc:385] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:48.672767 1168 raft_consensus.cc:740] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:48.673039 1799 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_187cac16164c4f23bdf2b49d86a68bde"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:48.673633 1799 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_187cac16164c4f23bdf2b49d86a68bde in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:48.662037 1795 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46568 (request call id 89) took 1104 ms. Trace:
I20251211 20:52:48.675797 1795 rpcz_store.cc:276] 1211 20:52:47.557966 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:48.181493 (+623527us) service_pool.cc:225] Handling call
1211 20:52:48.280483 (+ 98990us) spinlock_profiling.cc:243] Waited 98.7 ms on lock 0x616000040f54. stack: 00007ff715c18ee6 00007ff715c18ca0 00007ff714bf43ad 0000000000984f94 0000000000984f5d 00007ff724f1f6bd 00007ff724f1f7b3 00007ff721ae6521 00007ff7297a596d 00007ff729934456 00007ff720440146 00007ff72043fe75 00007ff71cbb856e 00007ff71cbb744e 00007ff71cbbd685 00007ff71cbbdfaf
1211 20:52:48.280688 (+ 205us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
1211 20:52:48.280872 (+ 184us) catalog_manager.cc:3708] Apply alter schema
1211 20:52:48.281041 (+ 169us) catalog_manager.cc:3740] Acquired catalog manager lock
1211 20:52:48.281111 (+ 70us) catalog_manager.cc:3898] Updating metadata on disk
1211 20:52:48.332808 (+ 51697us) catalog_manager.cc:3929] Committing alterations to in-memory state
1211 20:52:48.661982 (+329174us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:48.281608 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:48.281700 (+ 92us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:48.281705 (+ 5us) write_op.cc:435] Acquired schema lock
1211 20:52:48.281708 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:48.281800 (+ 92us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:48.281811 (+ 11us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:48.281815 (+ 4us) tablet.cc:683] Acquiring locks for 1 operations
1211 20:52:48.282092 (+ 277us) tablet.cc:699] Row locks acquired
1211 20:52:48.282096 (+ 4us) write_op.cc:260] PREPARE: finished
1211 20:52:48.282144 (+ 48us) write_op.cc:270] Start()
1211 20:52:48.282178 (+ 34us) write_op.cc:276] Timestamp: P: 1765486368282087 usec, L: 0
1211 20:52:48.282182 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:48.282352 (+ 170us) log.cc:844] Serialized 405 byte log entry
1211 20:52:48.324827 (+ 42475us) op_driver.cc:464] REPLICATION: finished
1211 20:52:48.325901 (+ 1074us) write_op.cc:301] APPLY: starting
1211 20:52:48.325911 (+ 10us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:48.325938 (+ 27us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:48.325977 (+ 39us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:48.326011 (+ 34us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:48.326035 (+ 24us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1211 20:52:48.326041 (+ 6us) write_op.cc:312] APPLY: finished
1211 20:52:48.326094 (+ 53us) log.cc:844] Serialized 23 byte log entry
1211 20:52:48.326135 (+ 41us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.326147 (+ 12us) write_op.cc:454] Released schema lock
1211 20:52:48.326151 (+ 4us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:48.326207 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.326211 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":227091712,"child_traces":[["op",{"apply.queue_time_us":1043,"apply.run_cpu_time_us":302,"apply.run_wall_time_us":301,"num_ops":1,"prepare.queue_time_us":44,"prepare.run_cpu_time_us":753,"prepare.run_wall_time_us":1262,"raft.queue_time_us":38,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":42630}]]}
I20251211 20:52:48.677366 1168 consensus_queue.cc:260] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.678274 880 tablet_service.cc:1505] Processing CreateTablet for tablet 239355c39c064819a3b992c3bc2ece9c (DEFAULT_TABLE table=default.Table_5b3f562b5129421a9d27a06bebc88fe4 [id=4f970446d3de437b9b811db0bd41659f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:48.631409 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79cc81e91e1846038621af7a0a9f7a2e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:48.679250 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 09cd11f8b23c4f8a98a2b2eb77bb7b8c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09cd11f8b23c4f8a98a2b2eb77bb7b8c already in progress: creating tablet
W20251211 20:52:48.679527 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 5d225553d0984b6e9685b5841616c9d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5d225553d0984b6e9685b5841616c9d8 already in progress: creating tablet
I20251211 20:52:48.679992 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e9ead34065e240eeb1f946319537d32a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:48.680296 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 239355c39c064819a3b992c3bc2ece9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:48.680994 1019 tablet_service.cc:1552] Processing DeleteTablet for tablet 4776c456b4d74c90b1fc71d57b62a461 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:48.681394 1045 raft_consensus.cc:2468] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:48.681561 1035 raft_consensus.cc:2468] T de508d79f30f489cb745401f53115165 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:48.682063 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet d0673c37c2fd47eea8ab10d9e91b8d7a with delete_type TABLET_DATA_DELETED (replaced by tablet 86c5228210954839ae7bf5bf0b5aa027) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:48.682396 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 516f7181aaac48cdadf9fbc08d0d3f57 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:48.677392 1795 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_6a4f0ea2d527419c9d66de5d00f826d7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:48.687141 1795 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6a4f0ea2d527419c9d66de5d00f826d7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:48.687769 1021 tablet_service.cc:1552] Processing DeleteTablet for tablet e95c63b4d99a4dd0b2c0fa384ca9d552 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:48.689261 710 leader_election.cc:304] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:48.689817 2956 raft_consensus.cc:2804] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:48.689906 2956 raft_consensus.cc:493] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:48.689980 2956 raft_consensus.cc:3060] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.689911 710 leader_election.cc:304] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:48.690446 3001 raft_consensus.cc:2804] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:48.690563 3001 raft_consensus.cc:697] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:48.690804 3001 consensus_queue.cc:237] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.692548 710 leader_election.cc:304] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:48.692845 3051 raft_consensus.cc:2804] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:48.693004 3051 raft_consensus.cc:493] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:48.693073 3051 raft_consensus.cc:3060] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.699893 1168 ts_tablet_manager.cc:1434] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.028s user 0.002s sys 0.000s
I20251211 20:52:48.699803 3051 raft_consensus.cc:515] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.700271 1168 tablet_bootstrap.cc:492] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:48.700567 3051 leader_election.cc:290] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:48.692534 2956 raft_consensus.cc:515] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.701383 2956 leader_election.cc:290] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:48.706691 905 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de508d79f30f489cb745401f53115165" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:48.707621 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f9a3fdf17f274c15ac0bdfa9c174f649" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:48.707751 1021 tablet_service.cc:1552] Processing DeleteTablet for tablet 6029c1fd563d4023a8cf7cfb8ffa0876 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:48.708256 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 2e33606930774fe4aafc6e3819618af8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:36220
W20251211 20:52:48.712204 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 4776c456b4d74c90b1fc71d57b62a461 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4776c456b4d74c90b1fc71d57b62a461 already in progress: creating tablet
I20251211 20:52:48.712453 1021 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a61cf8f5de14adaa4a1de5bddde9a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:36220
W20251211 20:52:48.712532 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet d0673c37c2fd47eea8ab10d9e91b8d7a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d0673c37c2fd47eea8ab10d9e91b8d7a already in progress: creating tablet
I20251211 20:52:48.712600 710 leader_election.cc:304] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:48.712745 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 5710fc2f24764f818f7e5c86c6fd985e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:48.712874 710 leader_election.cc:304] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:48.713165 2961 raft_consensus.cc:2804] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:48.713332 2961 raft_consensus.cc:697] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:48.713099 3051 raft_consensus.cc:2804] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:48.713522 3051 raft_consensus.cc:493] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:48.713603 3051 raft_consensus.cc:3060] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.713575 2961 consensus_queue.cc:237] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.714421 1168 tablet_bootstrap.cc:654] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:48.718416 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1e13965b6361488eaeb7dd13b467ca65" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:48.718583 1048 raft_consensus.cc:3060] T 1e13965b6361488eaeb7dd13b467ca65 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.719130 3051 raft_consensus.cc:515] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.719789 3051 leader_election.cc:290] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:48.721158 1048 raft_consensus.cc:2468] T 1e13965b6361488eaeb7dd13b467ca65 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:48.724236 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79cc81e91e1846038621af7a0a9f7a2e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:48.724391 1168 tablet_bootstrap.cc:492] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:48.724525 1048 raft_consensus.cc:3060] T 79cc81e91e1846038621af7a0a9f7a2e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.724542 1168 ts_tablet_manager.cc:1403] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.000s
I20251211 20:52:48.729326 1168 raft_consensus.cc:359] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.729553 1168 raft_consensus.cc:385] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:48.729624 1168 raft_consensus.cc:740] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:48.729840 1168 consensus_queue.cc:260] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.730376 1168 ts_tablet_manager.cc:1434] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20251211 20:52:48.731168 1048 raft_consensus.cc:2468] T 79cc81e91e1846038621af7a0a9f7a2e P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:48.740572 905 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e9ead34065e240eeb1f946319537d32a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:48.741122 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79cc81e91e1846038621af7a0a9f7a2e" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:48.741483 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1e13965b6361488eaeb7dd13b467ca65" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:48.746356 1160 tablet_bootstrap.cc:654] T eb9da081cb9d43fd9a35c4bc51ea133e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:48.747315 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1e13965b6361488eaeb7dd13b467ca65" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:48.748580 983 leader_election.cc:343] T f9a3fdf17f274c15ac0bdfa9c174f649 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.749369 2964 raft_consensus.cc:493] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6f8d784fd1cd492bbe3809403ec34919)
I20251211 20:52:48.749518 2964 raft_consensus.cc:515] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.750195 2964 leader_election.cc:290] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:48.750519 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e9ead34065e240eeb1f946319537d32a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:48.749276 1168 tablet_bootstrap.cc:492] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:48.751077 711 leader_election.cc:343] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:48.751554 711 leader_election.cc:343] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:48.752359 847 leader_election.cc:343] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 4f807596fa6a4e66b89566b5d5dd1080
W20251211 20:52:48.752614 711 leader_election.cc:343] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:48.752969 711 leader_election.cc:343] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:48.753211 711 leader_election.cc:343] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:48.753870 711 leader_election.cc:343] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.759819 1168 tablet_bootstrap.cc:654] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:48.759878 711 leader_election.cc:343] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:48.767480 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 834b84a557b04e78b5d2c8c84520a644 (table default.Table_909bbe6da1a648d68199fb27424e4342 [id=cc99459a5039424dbc192ff748b6708b], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 834b84a557b04e78b5d2c8c84520a644 consensus state has no leader
I20251211 20:52:48.773435 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e9ead34065e240eeb1f946319537d32a" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:48.773717 1048 raft_consensus.cc:3060] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:48.776713 1048 raft_consensus.cc:2468] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:48.777107 2956 raft_consensus.cc:493] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:48.778512 2956 raft_consensus.cc:515] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.779160 2956 leader_election.cc:290] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:48.781558 902 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5df87ea7d28340da84ef141c92efc710" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:48.782141 3029 tablet.cc:1722] T 3e6f62c2deff4c11b137fc0cba5ee234 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20251211 20:52:48.782294 711 leader_election.cc:343] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.784265 1168 tablet_bootstrap.cc:492] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:48.790284 1166 raft_consensus.cc:359] T f91487dd7e0f4c8b8483c2b83945da3b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.790482 1166 raft_consensus.cc:385] T f91487dd7e0f4c8b8483c2b83945da3b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:48.790536 1166 raft_consensus.cc:740] T f91487dd7e0f4c8b8483c2b83945da3b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:48.790733 1166 consensus_queue.cc:260] T f91487dd7e0f4c8b8483c2b83945da3b P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.791188 1166 ts_tablet_manager.cc:1434] T f91487dd7e0f4c8b8483c2b83945da3b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.121s user 0.001s sys 0.000s
I20251211 20:52:48.791504 1168 ts_tablet_manager.cc:1403] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.061s user 0.004s sys 0.000s
W20251211 20:52:48.794215 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet b2974f16034c44cf85ff85b5095cf6a0 (table default.Table_031de0f4cc7744bc9e320261f8b46696 [id=be36dabaa6d44325b4855b9f13bda847], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet b2974f16034c44cf85ff85b5095cf6a0 consensus state has no leader
W20251211 20:52:48.795425 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet ab4e7a9062134e489d6f9abe6ff26fb4 (table default.Table_4cbc8841b276406cb4434629653a5ce3 [id=b958532650374d609199cab9c07bb64b], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet ab4e7a9062134e489d6f9abe6ff26fb4 consensus state has no leader
I20251211 20:52:48.796022 1168 raft_consensus.cc:359] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.800132 1168 raft_consensus.cc:385] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:48.801059 1168 raft_consensus.cc:740] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:48.803524 1168 consensus_queue.cc:260] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.805467 1168 ts_tablet_manager.cc:1434] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20251211 20:52:48.810014 1168 tablet_bootstrap.cc:492] T 7a0a20f95416425489af94408ac1e97c P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:48.811847 1166 tablet_bootstrap.cc:492] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:48.811893 2962 tablet_replica.cc:333] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:48.812057 2962 raft_consensus.cc:2243] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:48.812211 2962 raft_consensus.cc:2272] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:48.813182 1168 tablet_bootstrap.cc:654] T 7a0a20f95416425489af94408ac1e97c P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:48.813246 2962 ts_tablet_manager.cc:1916] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:48.813488 1166 tablet_bootstrap.cc:654] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:48.825508 2962 ts_tablet_manager.cc:1929] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:48.827283 3001 raft_consensus.cc:493] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:48.827553 3001 raft_consensus.cc:515] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.828357 3001 leader_election.cc:290] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:48.828967 901 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "be876471f8bf453aa4467c91891cbbde" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:48.829890 711 leader_election.cc:343] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.826714 1031 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5df87ea7d28340da84ef141c92efc710" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:48.830662 1031 raft_consensus.cc:2468] T 5df87ea7d28340da84ef141c92efc710 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:48.830947 2962 log.cc:1199] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/6029c1fd563d4023a8cf7cfb8ffa0876
I20251211 20:52:48.831431 2962 ts_tablet_manager.cc:1950] T 6029c1fd563d4023a8cf7cfb8ffa0876 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:48.846508 1799 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46568 (request call id 92) took 1092 ms. Trace:
I20251211 20:52:48.846804 1799 rpcz_store.cc:276] 1211 20:52:47.754467 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:48.672791 (+918324us) service_pool.cc:225] Handling call
1211 20:52:48.835963 (+163172us) catalog_manager.cc:2101] Acquired catalog manager lock
1211 20:52:48.836526 (+ 563us) catalog_manager.cc:2147] Created new table and tablet info
1211 20:52:48.846242 (+ 9716us) catalog_manager.cc:2207] Wrote table and tablets to system table
1211 20:52:48.846357 (+ 115us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
1211 20:52:48.846454 (+ 97us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:48.839890 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:48.840035 (+ 145us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:48.840041 (+ 6us) write_op.cc:435] Acquired schema lock
1211 20:52:48.840044 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:48.840152 (+ 108us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:48.840167 (+ 15us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:48.840172 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:48.840277 (+ 105us) tablet.cc:699] Row locks acquired
1211 20:52:48.840279 (+ 2us) write_op.cc:260] PREPARE: finished
1211 20:52:48.840321 (+ 42us) write_op.cc:270] Start()
1211 20:52:48.840356 (+ 35us) write_op.cc:276] Timestamp: P: 1765486368840265 usec, L: 0
1211 20:52:48.840359 (+ 3us) op_driver.cc:348] REPLICATION: starting
1211 20:52:48.840588 (+ 229us) log.cc:844] Serialized 433 byte log entry
1211 20:52:48.844882 (+ 4294us) op_driver.cc:464] REPLICATION: finished
1211 20:52:48.845016 (+ 134us) write_op.cc:301] APPLY: starting
1211 20:52:48.845033 (+ 17us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:48.845083 (+ 50us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:48.845109 (+ 26us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:48.845228 (+ 119us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:48.845265 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1211 20:52:48.845272 (+ 7us) write_op.cc:312] APPLY: finished
1211 20:52:48.845351 (+ 79us) log.cc:844] Serialized 29 byte log entry
1211 20:52:48.845416 (+ 65us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.845437 (+ 21us) write_op.cc:454] Released schema lock
1211 20:52:48.845443 (+ 6us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:48.845516 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.845520 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":84,"apply.run_cpu_time_us":492,"apply.run_wall_time_us":491,"num_ops":2,"prepare.queue_time_us":2790,"prepare.run_cpu_time_us":838,"prepare.run_wall_time_us":838,"raft.queue_time_us":876,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":246,"replication_time_us":4483}]]}
I20251211 20:52:48.849920 3029 tablet.cc:1722] T 6b7dd8232d974a43b99ec9b28075dc8d P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:48.850925 1795 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46568 (request call id 93) took 1077 ms. Trace:
I20251211 20:52:48.851572 1795 rpcz_store.cc:276] 1211 20:52:47.773440 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:48.677189 (+903749us) service_pool.cc:225] Handling call
1211 20:52:48.837085 (+159896us) catalog_manager.cc:2101] Acquired catalog manager lock
1211 20:52:48.837506 (+ 421us) catalog_manager.cc:2147] Created new table and tablet info
1211 20:52:48.850620 (+ 13114us) catalog_manager.cc:2207] Wrote table and tablets to system table
1211 20:52:48.850749 (+ 129us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
1211 20:52:48.850878 (+ 129us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:48.841946 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:48.842055 (+ 109us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:48.842062 (+ 7us) write_op.cc:435] Acquired schema lock
1211 20:52:48.842064 (+ 2us) tablet.cc:660] Decoding operations
1211 20:52:48.842148 (+ 84us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:48.842158 (+ 10us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:48.842163 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:48.842233 (+ 70us) tablet.cc:699] Row locks acquired
1211 20:52:48.842236 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:48.842269 (+ 33us) write_op.cc:270] Start()
1211 20:52:48.842301 (+ 32us) write_op.cc:276] Timestamp: P: 1765486368842215 usec, L: 0
1211 20:52:48.842304 (+ 3us) op_driver.cc:348] REPLICATION: starting
1211 20:52:48.842455 (+ 151us) log.cc:844] Serialized 433 byte log entry
1211 20:52:48.849285 (+ 6830us) op_driver.cc:464] REPLICATION: finished
1211 20:52:48.849425 (+ 140us) write_op.cc:301] APPLY: starting
1211 20:52:48.849445 (+ 20us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:48.849487 (+ 42us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:48.849520 (+ 33us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:48.849632 (+ 112us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:48.849665 (+ 33us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1211 20:52:48.849672 (+ 7us) write_op.cc:312] APPLY: finished
1211 20:52:48.849750 (+ 78us) log.cc:844] Serialized 29 byte log entry
1211 20:52:48.849821 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.849841 (+ 20us) write_op.cc:454] Released schema lock
1211 20:52:48.849847 (+ 6us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:48.849952 (+ 105us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.849957 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":92,"apply.run_cpu_time_us":498,"apply.run_wall_time_us":498,"mutex_wait_us":3938,"num_ops":2,"prepare.queue_time_us":4007,"prepare.run_cpu_time_us":600,"prepare.run_wall_time_us":600,"raft.queue_time_us":70,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":5,"replication_time_us":6952}]]}
W20251211 20:52:48.853082 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet df457783b2534b3ea23bb1fd529ce268 (table default.Table_6e9f3ca164384cc8ac4501deb3f0c6aa [id=342c2469d6e74e3e9217236e3aa863d6], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet df457783b2534b3ea23bb1fd529ce268 consensus state has no leader
I20251211 20:52:48.853529 1795 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_37d416a121e34cc8abb9520f5320dfbd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:52:48.854815 1795 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_37d416a121e34cc8abb9520f5320dfbd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:48.856799 1031 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "be876471f8bf453aa4467c91891cbbde" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:48.857025 1031 raft_consensus.cc:2468] T be876471f8bf453aa4467c91891cbbde P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:48.879554 2968 ts_tablet_manager.cc:1929] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:48.879918 2968 log.cc:1199] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/373d3f03d7e0408cbe06955f8f60b16c
I20251211 20:52:48.880466 2968 ts_tablet_manager.cc:1950] T 373d3f03d7e0408cbe06955f8f60b16c P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:48.881597 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 373d3f03d7e0408cbe06955f8f60b16c (table default.Table_43e9cb840d48444b9360e2fa8671dfb8 [id=f6cab1da32324fe6afc1df2007f0bdbf]) successfully deleted
W20251211 20:52:48.881981 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 903c7f4b312d4677bb35be1aaa46ad6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 903c7f4b312d4677bb35be1aaa46ad6b already in progress: creating tablet
W20251211 20:52:48.883163 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet c8bb92fc25124ba4be887c453abc7d9a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c8bb92fc25124ba4be887c453abc7d9a already in progress: creating tablet
W20251211 20:52:48.883438 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 09cd11f8b23c4f8a98a2b2eb77bb7b8c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09cd11f8b23c4f8a98a2b2eb77bb7b8c already in progress: creating tablet
W20251211 20:52:48.883647 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 5d225553d0984b6e9685b5841616c9d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5d225553d0984b6e9685b5841616c9d8 already in progress: creating tablet
I20251211 20:52:48.885936 1160 tablet_bootstrap.cc:492] T eb9da081cb9d43fd9a35c4bc51ea133e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:48.886176 1160 ts_tablet_manager.cc:1403] T eb9da081cb9d43fd9a35c4bc51ea133e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.658s user 0.002s sys 0.011s
I20251211 20:52:48.886354 3029 tablet.cc:1722] T e4647105ab57454d883ebfcde7bde891 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:48.886979 1160 raft_consensus.cc:359] T eb9da081cb9d43fd9a35c4bc51ea133e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.887171 1160 raft_consensus.cc:385] T eb9da081cb9d43fd9a35c4bc51ea133e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:48.887219 1160 raft_consensus.cc:740] T eb9da081cb9d43fd9a35c4bc51ea133e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:48.887428 1160 consensus_queue.cc:260] T eb9da081cb9d43fd9a35c4bc51ea133e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.888139 1160 ts_tablet_manager.cc:1434] T eb9da081cb9d43fd9a35c4bc51ea133e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:48.888402 3110 tablet.cc:1722] T 3e6f62c2deff4c11b137fc0cba5ee234 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:48.867558 2962 tablet_replica.cc:333] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:48.900750 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 0302cc8613934c43a0650f081cf98797 (table default.Table_bf827b216f1b43c196923dc1828ff0d0 [id=ed371e073250439fa44330c7101e9387]). Expected version 1 got 0
I20251211 20:52:48.901234 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 1f4eb1a174c648c897ae2b7612e44585 (table default.Table_e70873b71ba24d7eae548e2acfd243b3 [id=87adcc552a704d56bd2fe812fb3b791f]). Expected version 1 got 0
I20251211 20:52:48.901438 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet a7afcce522494c9ca1d80e6eea913e6e (table default.Table_cb6a4bac7b694f33a37de2ee78c91ca0 [id=8940f18527c249a9baefc170ed82f45a]). Expected version 1 got 0
I20251211 20:52:48.901602 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 84bb3583857848459e0e865283de623d (table default.Table_92281c5fcfee4355a79215ffedeee588 [id=6e5fe0a78e4b48f98dac028802f1ddf4]). Expected version 4 got 0
I20251211 20:52:48.901789 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 7a299ee03d0e48aba7c4f71e3e0f381a (table default.Table_6402a801d48e4a53827b1b62b6cdaba1 [id=6876565d3a9c48ce841b830cf75139ff]). Expected version 3 got 0
I20251211 20:52:48.902240 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 5b3e8ec3faf445ac9b746e65dc4dbfe0 (table default.Table_7fe80ff5483a4b0292091119980f6639 [id=3c3462a0c09748cea0d2ffb544ae2f13]). Expected version 1 got 0
I20251211 20:52:48.902411 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 7ffa06c4a7ee48b0b337a27aff090592 (table default.Table_647731023b044ebb8954f7fdcc3faf97 [id=4b91abd6e9e94a8a941b33e62e23c083]). Expected version 1 got 0
I20251211 20:52:48.902562 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet b81a4cef27164824b52695039744db15 (table default.Table_defdeab28749411590cf75b5251f604b [id=11352566d1744193a40a8bb605aa3912]). Expected version 1 got 0
I20251211 20:52:48.902747 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 81fe26e81790448fb436929c03e3c284 (table default.Table_efc22dc69ad0405ebf59fda961e23330 [id=d8a0839df7524fb7b95bfc05fd4ada25]). Expected version 1 got 0
I20251211 20:52:48.902877 3094 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:52:48.902913 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 7023afec769c4ca3b2242a225004d482 (table default.Table_d5276e3cc46d4331b36a8878680fc55e [id=3811f8235b2f44cca283401302d613d4]). Expected version 1 got 0
I20251211 20:52:48.903220 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 72af55c8045b4f8f8e116a5782794d81 (table default.Table_2cff6fab088c4d939b13b504c133cc12 [id=b6c4b72063b3428593d5650a160dc66b]). Expected version 1 got 0
I20251211 20:52:48.903398 3000 raft_consensus.cc:493] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:48.903620 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 4cf0cde801bf44449e539d2799b895a7 (table default.Table_907c596bdddc4edca25d14cbacebb23f [id=321235280d1d4fa0bae5cc0333860f85]). Expected version 1 got 0
I20251211 20:52:48.903551 3000 raft_consensus.cc:515] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.903861 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 7b9f569554fb4836ac2b62542c3ecf7e (table default.Table_1a73d567e8914a83974e3e8f5f58be0b [id=6ef691749df24251a7a7c5919c219754]). Expected version 1 got 0
I20251211 20:52:48.904003 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 3b2c6064c5eb4c0f8979c485f5d85a75 (table default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f]). Expected version 1 got 0
I20251211 20:52:48.904177 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 938a0d8a090f490b816fdf2b8f1a51b8 (table default.Table_5ca4ab25a21d456c95a0ff68c9d73330 [id=9ffcb594267a4b0eb8d792dbe46d585f]). Expected version 1 got 0
I20251211 20:52:48.904242 3000 leader_election.cc:290] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:48.904292 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 59acb8474e114bf7ae7f1f043d611459 (table default.Table_c7ed8c0c309c40a3a371ffa130b47cd2 [id=ed1d0670426945aba888b818ad657225]). Expected version 1 got 0
I20251211 20:52:48.904529 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet b52377c8ffd741a598e5328a00d876fe (table default.Table_686a26915b42421380037144900dd97c [id=8a1b0c9d641b4a3c83d1ca48576d110d]). Expected version 1 got 0
I20251211 20:52:48.904651 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet bbe6e6220eb14e59a821805c438230c9 (table default.Table_16a12ed370b64a38b5e145c4eb024349 [id=b7429ea93123473ab5d795d5f3d5a5f8]). Expected version 1 got 0
I20251211 20:52:48.905058 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet ce1799e6604f4ed7875ddedecab6f327 (table default.Table_5db1ac98e36d45b1b01889ee275ad83d [id=a6849726434d41d29dce12f47066a685]). Expected version 1 got 0
I20251211 20:52:48.905182 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet dfd194cf00f549c8937cb6ec40987240 (table default.Table_25e38d0576684687890bae04bd81d4f4 [id=d919c6132b3c466e8a1404f981b003b8]). Expected version 2 got 0
I20251211 20:52:48.905473 909 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "98d575ac680d49d0990877666638f013" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:48.905635 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 1c4b2c22bd4c456eb77af45576de466f (table default.Table_e3a99b043e4f420a93ebfee6aa87f2ec [id=3e7d6cefcb7d412191c6540e9ad77c2d]). Expected version 2 got 0
W20251211 20:52:48.906034 711 leader_election.cc:343] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.906065 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet bc4e7362e6d045da81bf39f4135f500e (table default.Table_328b7dd76e9e407c83b47a0f51cfcce5 [id=9e7c08f833e14a90b17a9b9f60b702de]). Expected version 1 got 0
I20251211 20:52:48.906417 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet f26ba8f4bd3a4a1f8fa2ba6e80d543f8 (table default.Table_71e9c158dc22435a8fbb02aecca8919d [id=b966d0c0b49d4023894950c5dc06f678]). Expected version 2 got 0
I20251211 20:52:48.907047 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 51d12723f5194afb97d2c05fcbaf3776 (table default.Table_823505200e3142dbbe8cd56ac78daf55 [id=ec0bb10933fa4529bef3580bee6409e7]). Expected version 2 got 0
I20251211 20:52:48.907507 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 4fe5c534b572491c911aae01eaa79bae (table default.Table_c1f9dca5c2144d50b67681dcc6800e77 [id=14fa8debfe5443ce81587f02dfa15847]). Expected version 2 got 0
I20251211 20:52:48.907531 3029 tablet.cc:1722] T 8b793241ee3b42b1a38e9f2f05e1173b P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:48.908471 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 8b793241ee3b42b1a38e9f2f05e1173b (table default.Table_7506ffecc2b64648a9ba736e5298ddf4 [id=279f39d718ba45318f52f95e8aa13652]). Expected version 2 got 0
I20251211 20:52:48.910923 2962 raft_consensus.cc:2243] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:48.911121 2962 raft_consensus.cc:2272] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:48.911710 2962 ts_tablet_manager.cc:1916] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:48.913793 3029 tablet.cc:1722] T a3b1aadf17ea4bf0a896d23cd543fca7 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:48.924643 1795 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46568 (request call id 94) took 1139 ms. Trace:
I20251211 20:52:48.925210 1795 rpcz_store.cc:276] 1211 20:52:47.784743 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:48.852975 (+1068232us) service_pool.cc:225] Handling call
1211 20:52:48.867819 (+ 14844us) catalog_manager.cc:2101] Acquired catalog manager lock
1211 20:52:48.868397 (+ 578us) catalog_manager.cc:2147] Created new table and tablet info
1211 20:52:48.924381 (+ 55984us) catalog_manager.cc:2207] Wrote table and tablets to system table
1211 20:52:48.924489 (+ 108us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
1211 20:52:48.924586 (+ 97us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:48.918253 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:48.918397 (+ 144us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:48.918403 (+ 6us) write_op.cc:435] Acquired schema lock
1211 20:52:48.918406 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:48.918511 (+ 105us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:48.918526 (+ 15us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:48.918531 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:48.918618 (+ 87us) tablet.cc:699] Row locks acquired
1211 20:52:48.918620 (+ 2us) write_op.cc:260] PREPARE: finished
1211 20:52:48.918664 (+ 44us) write_op.cc:270] Start()
1211 20:52:48.918704 (+ 40us) write_op.cc:276] Timestamp: P: 1765486368918600 usec, L: 0
1211 20:52:48.918707 (+ 3us) op_driver.cc:348] REPLICATION: starting
1211 20:52:48.918927 (+ 220us) log.cc:844] Serialized 433 byte log entry
1211 20:52:48.923041 (+ 4114us) op_driver.cc:464] REPLICATION: finished
1211 20:52:48.923196 (+ 155us) write_op.cc:301] APPLY: starting
1211 20:52:48.923213 (+ 17us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:48.923260 (+ 47us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:48.923291 (+ 31us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:48.923401 (+ 110us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:48.923438 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1211 20:52:48.923444 (+ 6us) write_op.cc:312] APPLY: finished
1211 20:52:48.923522 (+ 78us) log.cc:844] Serialized 29 byte log entry
1211 20:52:48.923615 (+ 93us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.923649 (+ 34us) write_op.cc:454] Released schema lock
1211 20:52:48.923657 (+ 8us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:48.923736 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.923740 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":103,"apply.run_cpu_time_us":533,"apply.run_wall_time_us":532,"mutex_wait_us":566,"num_ops":2,"prepare.queue_time_us":49264,"prepare.run_cpu_time_us":892,"prepare.run_wall_time_us":1449,"raft.queue_time_us":646,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":252,"replication_time_us":4303,"spinlock_wait_cycles":14720}]]}
I20251211 20:52:48.926981 1795 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_56eb53be351a424e80c1f051e43e8b5d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:48.927881 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet ab4e7a9062134e489d6f9abe6ff26fb4 (table default.Table_4cbc8841b276406cb4434629653a5ce3 [id=b958532650374d609199cab9c07bb64b]). Expected version 2 got 0
W20251211 20:52:48.928457 1795 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_56eb53be351a424e80c1f051e43e8b5d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:48.928555 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 6b7dd8232d974a43b99ec9b28075dc8d (table default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]). Expected version 2 got 0
I20251211 20:52:48.929579 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 645b08b33676420c96e880de3d460829 (table default.Table_e02933dd91ce4fe7a40d1c3b2f9b2d4b [id=6d42f260246e420d8818b3841926302b]). Expected version 2 got 0
I20251211 20:52:48.929857 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 4c0455a5b4f44412803475d1c4852b48 (table default.Table_e00092673b0e4f09b8d7eec63e1ee034 [id=18497e06d7254441a4655ce1cdc5a9bf]). Expected version 1 got 0
I20251211 20:52:48.930320 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 7ca98b43863a47c091baae0a6ce65630 (table default.Table_f0a70914acc94231a4e84155c3e4d13f [id=56c070ad80624c698ebe2e01b4929b6e]). Expected version 1 got 0
I20251211 20:52:48.930600 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 6212b3b4425543168837946a54d60834 (table default.Table_8c30c4151d484db0bb9e2b9ca262248d [id=2d489cd7d73e403d84bf10960e6a2dfb]). Expected version 1 got 0
I20251211 20:52:48.930850 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet df457783b2534b3ea23bb1fd529ce268 (table default.Table_6e9f3ca164384cc8ac4501deb3f0c6aa [id=342c2469d6e74e3e9217236e3aa863d6]). Expected version 1 got 0
I20251211 20:52:48.931185 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet fb29f983c4524a49bdc5c0c1144ad708 (table default.Table_7809ac85fdfe4c1d9c58afb54edecb27 [id=c4b0bfff99f64888beb70d9154d2f156]). Expected version 1 got 0
I20251211 20:52:48.931493 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet d6923cfd826d495291d9ac8e0b3b44c6 (table default.Table_68ccb2052d824ccf92f7162d2961fcb1 [id=9ac9591d4f04482d926710fdf6e797c6]). Expected version 1 got 0
I20251211 20:52:48.934772 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet a3b1aadf17ea4bf0a896d23cd543fca7 (table default.Table_d18401707c88493abdf90d2c713432ae [id=a07662f8dc3f4178be5dfcc1cdc85837]). Expected version 1 got 0
I20251211 20:52:48.935190 1166 tablet_bootstrap.cc:492] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:48.935397 1166 ts_tablet_manager.cc:1403] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.124s user 0.001s sys 0.002s
I20251211 20:52:48.935534 3029 tablet.cc:1722] T 9f8aa28c1c9b4fc58dfc128952c180ba P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:48.936008 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet b2974f16034c44cf85ff85b5095cf6a0 (table default.Table_031de0f4cc7744bc9e320261f8b46696 [id=be36dabaa6d44325b4855b9f13bda847]). Expected version 1 got 0
I20251211 20:52:48.936288 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 16d2ff8925e342729a71c93db378d031 (table default.Table_34e6ed4f13da437ea50d349b92e83bd6 [id=4a74b9a60cbb4198b55cee630b2f4bef]). Expected version 2 got 0
I20251211 20:52:48.936455 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet f2d6c9781b4d4c37bc08703347963d5d (table default.Table_2aa7ff9186a04249abf8cd971cc0cb17 [id=b50ab5b032ba4c98938324460512468f]). Expected version 1 got 0
I20251211 20:52:48.936637 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet f94b656e3c1844509fdae36e33b263e8 (table default.Table_ed03ad475bed4804a65abe20efef0625 [id=e23f4305669a474aafa18928f130de3a]). Expected version 2 got 0
I20251211 20:52:48.936653 2974 raft_consensus.cc:493] T 834b84a557b04e78b5d2c8c84520a644 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:48.936800 2974 raft_consensus.cc:515] T 834b84a557b04e78b5d2c8c84520a644 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.936900 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 676faec7f3d1488f99b220f8e968e9a0 (table default.Table_ab826a3eec8e4c4aaa8d4021d87dd382 [id=a2bbfccfe3974c378521d5ef2333df86]). Expected version 2 got 0
I20251211 20:52:48.937021 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 5e47af616e1b4d2db3780c1d67cdcb47 (table default.Table_296bdf30a11541c389afe3df0b0f667a [id=052edcbde60c41449d468c8a449fc3f1]). Expected version 1 got 0
I20251211 20:52:48.937414 2974 leader_election.cc:290] T 834b84a557b04e78b5d2c8c84520a644 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:48.938725 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet aab26df748724fe5b2bb7fad9e3bfc0c (table default.Table_22bed4b4e9e24a90818b19c6f5c91e1f [id=df6255d66f334ef2b4406f4b845cf980]). Expected version 1 got 0
I20251211 20:52:48.938727 2959 raft_consensus.cc:493] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:48.938853 2959 raft_consensus.cc:515] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.939411 2959 leader_election.cc:290] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:48.940745 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet bc6a1c84501d4e179cd2828b50fddc23 (table default.Table_8bb5b228d60e4d29a41d9ba40c33d45d [id=6ef058fada9941629e0ea4d1906abdac]). Expected version 1 got 0
I20251211 20:52:48.941210 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 03184dcc56dd4e92adc7d0ad22aa8d8b (table default.Table_68261841754847ad8d22e86c29886144 [id=697f23a617f64762bf406b386cde7473]). Expected version 1 got 0
I20251211 20:52:48.941151 3132 tablet.cc:1722] T 6b7dd8232d974a43b99ec9b28075dc8d P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:48.941426 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet e3f711f02af843adb4f5c3c750924e33 (table default.Table_34bd5155aa3b4ef7818022ba9346c074 [id=83a61d23e80946439a14096ab3ab1bd9]). Expected version 2 got 0
I20251211 20:52:48.941723 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 5453f68789fd4e89b69e98860050865b (table default.Table_b7bf8c5ad5304175a1f2f6a03887f725 [id=50d88b663a4d4891b113aee9c5906699]). Expected version 1 got 0
I20251211 20:52:48.942283 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 94658fcf512a48d0a992a7e2e0f7c176 (table default.Table_c9d86e82aebb4ef0bc83789720352e38 [id=97377a8ef56c4cdebe95d15955207404]). Expected version 1 got 0
I20251211 20:52:48.942613 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet aa50ddb2789b4be3bfa9aedef9287109 (table default.Table_44f971096d1e41f480e594985ffd5c17 [id=d4e1349374a346a5b8be180f9bdc329a]). Expected version 1 got 0
I20251211 20:52:48.942860 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 497c10742f01476db885bd70c4d2c491 (table default.Table_18b21232dfa74378b27defe672bb2574 [id=602d7eee703948d99a0edcb7cac0651b]). Expected version 1 got 0
I20251211 20:52:48.942936 2550 fs_manager.cc:730] Time spent opening block manager: real 0.359s user 0.002s sys 0.000s
I20251211 20:52:48.943019 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 3003ef71a3554c71a3b24cdfb9c293bc (table default.Table_92efd469dd324f7d886e2cbc90ece61c [id=6e7fb3271e0c48c0bb9d56fe4fccee3a]). Expected version 1 got 0
I20251211 20:52:48.943444 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 5773c13034714a008c8d7dbe777b4828 (table default.Table_66eb1b021eed464f82f9cd223c88297b [id=94ff09564cd649c7a2d0e94b0f3c7b3a]). Expected version 1 got 0
I20251211 20:52:48.943660 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 18f125dd9efc401181f0644dad7aeffe (table default.Table_fa6592c2e2624d81ad21abb13ed36045 [id=9b48fc2b39db4fd5a4e5da19e34427dc]). Expected version 1 got 0
I20251211 20:52:48.943801 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 8dc4e730951144d78b77280f6cce6d0e (table default.Table_890c9027415a44ccaf95c0adede51992 [id=7bc8a2b91fe54b91b6594b371fea7bd3]). Expected version 2 got 0
I20251211 20:52:48.943984 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 46a2a070cf6a42fe8bc54a42eb802fbc (table default.Table_4e1f03b9d87b4fe59dbcb3a947ee299d [id=8e66ea58499b40e59294a135324aa527]). Expected version 1 got 0
I20251211 20:52:48.943117 2550 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/data,/tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/wal
uuid: "9230f94a6a1a49cead51aeea9f069087"
format_stamp: "Formatted at 2025-12-11 20:52:19 on dist-test-slave-fp9b"
I20251211 20:52:48.944151 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 13269a20421240d1a98215129443248b (table default.Table_d887c953026046dc883ede774a58fd8e [id=1d8a79f716a0447cb0393f4759178598]). Expected version 1 got 0
I20251211 20:52:48.944123 2550 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:52:48.944438 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 78f77d8916d84edabbe95ce17d8df2c1 (table default.Table_e2af0e2fbd3d46d595136c7a7bf24e4c [id=ddc12ae6d8ed4587b5ef4da3ef81a055]). Expected version 1 got 0
I20251211 20:52:48.944681 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet ff4d5ec1f87247e896f9c099c532265a (table default.Table_3ac4746e17264f5a93890f3600505781 [id=475e0a1ec0344d10addbcfa42fe05b68]). Expected version 1 got 0
I20251211 20:52:48.944907 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet b6cc819332e1445c96ce3ffac4861454 (table default.Table_db4c0f706e0643b39d39ffa84352826d [id=4d459c42bf584068ac3851227d80331a]). Expected version 1 got 0
I20251211 20:52:48.945148 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet f00128ba84c245d9a0e35f26bc842b3e (table default.Table_6f8c133c7a364b93af2c83d8911a0ac3 [id=6a21c38ee7224a02b937df088494f5d8]). Expected version 1 got 0
I20251211 20:52:48.945324 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 3dfea28c44d84c68850db65a30d52ee6 (table default.Table_c504f8f3796f43b697488bd604268543 [id=2af76b9d32094a4ca656060a4478e08a]). Expected version 1 got 0
I20251211 20:52:48.945518 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 4c7b74f370b042ee8575f821c07dc085 (table default.Table_731826f26f9940b8afe425d694b642d9 [id=17dd9cea9ac844128ff9f82adb3f919f]). Expected version 1 got 0
I20251211 20:52:48.945770 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet b85af79c2574443998e847bc19c53b69 (table default.Table_fa3f2f115cd043a9aa6be2b070d320cb [id=15fad5e1cf8d4ea08dae8275a29a11dd]). Expected version 1 got 0
I20251211 20:52:48.946147 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet c7a2610f20df4c1ca81eae2fc7dba6bb (table default.Table_59072235d93b41c0b5cd5e3debcb8ddf [id=9a15a8a075af4969ae2c8b3be1bfed78]). Expected version 1 got 0
I20251211 20:52:48.946414 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 33fd09ea06ee47908b85911e4ee79708 (table default.Table_3c3cd48c8fe5411e96cdf811d492925d [id=30620a01c27f456bac05fa7c02a2cb4d]). Expected version 2 got 0
I20251211 20:52:48.946604 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 3c865c99e9314f5f91658b9126da4d39 (table default.Table_0d7df3fba38849a49b65d8290812b95c [id=7f966d5cdad3423bb770414802a3cf7b]). Expected version 1 got 0
I20251211 20:52:48.947072 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 9f8aa28c1c9b4fc58dfc128952c180ba (table default.Table_bf7d402a36be44cfb6b79041906d26a1 [id=0327e686e7ed4475a3e8cda8721a3a3a]). Expected version 1 got 0
I20251211 20:52:48.947830 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 8675e2a18f0f4a0e95a87005cf82d8b4 (table default.Table_bd9633c55bd242c18aac3a7f2c18a329 [id=767f50bee5464316a18f73e64bab817a]). Expected version 2 got 0
I20251211 20:52:48.948336 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet fd43a27db69044b68826cda87e16a086 (table default.Table_a8ebfd7d7a7f4f6cae8b6b02b99b80b0 [id=f1d32002c612403a82bdff5baf209b9e]). Expected version 1 got 0
I20251211 20:52:48.948818 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet e4647105ab57454d883ebfcde7bde891 (table default.Table_c817dec9a7674f699b3366fc302ead57 [id=d555d1fc6aef4a21b4d3f2804264d030]). Expected version 1 got 0
I20251211 20:52:48.949260 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet b115d737d7fc483d9f5a1f040b511bae (table default.Table_6d94053661f142b79292e19752df02df [id=c5be59a2c3934b3789641ceeb2d4d6e2]). Expected version 2 got 0
I20251211 20:52:48.949414 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet da81bf8854a24954ba4548185812295d (table default.Table_18262e22c26d4d66ba4c6134fbde0368 [id=da8cf2b99b42409091484a9513623d6f]). Expected version 1 got 0
I20251211 20:52:48.949760 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 3e6f62c2deff4c11b137fc0cba5ee234 (table default.Table_51db250005ed4b3d947f32acd1d62563 [id=fdfaa0b24d2f4736aee695ed228212cf]). Expected version 1 got 0
I20251211 20:52:48.949944 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet d37a7d0d662e45079b0cfc02765d2e01 (table default.Table_014b6db9f3294658b965c701384b98bb [id=8a12299efe714617a970fc05c3bb6232]). Expected version 2 got 0
I20251211 20:52:48.950107 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet ee7b93b0b87541d9af6eb9e68b4cea41 (table default.Table_b6e6aea68c88426fb01353146c06b6ac [id=a4f984b4263a4bc38f6248628f643ae3]). Expected version 3 got 0
I20251211 20:52:48.950588 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 969a8587019c4202b6e705d126f83d44 (table default.Table_6460313002684bba93b30ad8631f0f49 [id=22347b15537d451d9927a17e8e5a202a]). Expected version 1 got 0
I20251211 20:52:48.950814 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 46cb1d16a0e94b3a833f32c635213fd2 (table default.Table_3482fbfc5b144a7198a7d01aa003cb6b [id=1bdaf5ce0f6b4b99a038f9f9efb828f6]). Expected version 1 got 0
I20251211 20:52:48.951342 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet e89d4d76bf64453892a4d0b55fcc3454 (table default.Table_fba99384e9f14160b3e3504f8d1d0b07 [id=01bba75f854d4c338d6c09a446dd7499]). Expected version 1 got 0
I20251211 20:52:48.951653 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 2e800d8cc5ea4535a065d7d9243326b4 (table default.Table_083c478bedec45a5bf5203e4ed333860 [id=8baf0ea87d734d58896adbd0d6ef99c2]). Expected version 1 got 0
I20251211 20:52:48.951804 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet d43210ec61af423ebde624397dea9f2e (table default.Table_af0df4d1ee144455a6041e5a6a0b9b80 [id=7c5b2e33fa1c4b12b3941577360908a8]). Expected version 3 got 0
I20251211 20:52:48.951973 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 834b84a557b04e78b5d2c8c84520a644 (table default.Table_909bbe6da1a648d68199fb27424e4342 [id=cc99459a5039424dbc192ff748b6708b]). Expected version 2 got 0
I20251211 20:52:48.952159 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 7be44df39a8f4d6da793479d425fa684 (table default.Table_f6a10556ff104db5bd5ecc9fc380e228 [id=f75813add54e46189ff1bbed82d83ba7]). Expected version 1 got 0
I20251211 20:52:48.952682 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 171d40a8a9124f9bb1b38797fef8e5d5 (table default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5]). Expected version 2 got 0
I20251211 20:52:48.952910 1802 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet c081396b492049b996ac4661e1318388 (table default.Table_00ae5c111de94ba0a2680ad22c726e16 [id=95c9ed1ec1834e2dbfd3198887b75369]). Expected version 1 got 0
I20251211 20:52:48.967970 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "98d575ac680d49d0990877666638f013" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:48.968187 1049 raft_consensus.cc:2468] T 98d575ac680d49d0990877666638f013 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:48.968588 1040 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01104975fd8b470aa026578b97ff6b8d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:48.968818 1040 raft_consensus.cc:2468] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:48.972354 3124 tablet.cc:1722] T e4647105ab57454d883ebfcde7bde891 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:48.974786 3133 tablet.cc:1722] T 676faec7f3d1488f99b220f8e968e9a0 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:48.936677 1166 raft_consensus.cc:359] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.976119 1166 raft_consensus.cc:385] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:48.976172 1166 raft_consensus.cc:740] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:48.976366 1166 consensus_queue.cc:260] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.976873 1166 ts_tablet_manager.cc:1434] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.041s user 0.000s sys 0.000s
I20251211 20:52:48.977490 907 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01104975fd8b470aa026578b97ff6b8d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:48.977981 1160 tablet_bootstrap.cc:492] T d37a7d0d662e45079b0cfc02765d2e01 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:48.978092 711 leader_election.cc:343] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.979020 3029 tablet.cc:1722] T c7a2610f20df4c1ca81eae2fc7dba6bb P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20251211 20:52:48.854168 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 18f125dd9efc401181f0644dad7aeffe (table default.Table_fa6592c2e2624d81ad21abb13ed36045 [id=9b48fc2b39db4fd5a4e5da19e34427dc], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 18f125dd9efc401181f0644dad7aeffe consensus state has no leader
I20251211 20:52:48.979779 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "834b84a557b04e78b5d2c8c84520a644" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:48.981040 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet b115d737d7fc483d9f5a1f040b511bae (table default.Table_6d94053661f142b79292e19752df02df [id=c5be59a2c3934b3789641ceeb2d4d6e2], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet b115d737d7fc483d9f5a1f040b511bae consensus state has no leader
W20251211 20:52:48.981282 983 leader_election.cc:343] T 834b84a557b04e78b5d2c8c84520a644 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:48.981973 2966 raft_consensus.cc:493] T eb9da081cb9d43fd9a35c4bc51ea133e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:48.982097 2966 raft_consensus.cc:515] T eb9da081cb9d43fd9a35c4bc51ea133e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:48.982718 2966 leader_election.cc:290] T eb9da081cb9d43fd9a35c4bc51ea133e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:48.981910 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet da81bf8854a24954ba4548185812295d (table default.Table_18262e22c26d4d66ba4c6134fbde0368 [id=da8cf2b99b42409091484a9513623d6f], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet da81bf8854a24954ba4548185812295d consensus state has no leader
W20251211 20:52:48.983641 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet f00128ba84c245d9a0e35f26bc842b3e (table default.Table_6f8c133c7a364b93af2c83d8911a0ac3 [id=6a21c38ee7224a02b937df088494f5d8], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet f00128ba84c245d9a0e35f26bc842b3e consensus state has no leader
W20251211 20:52:48.988965 847 leader_election.cc:343] T eb9da081cb9d43fd9a35c4bc51ea133e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: eb9da081cb9d43fd9a35c4bc51ea133e
I20251211 20:52:48.996417 3130 tablet.cc:1722] T 8b793241ee3b42b1a38e9f2f05e1173b P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:49.006814 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet 80c566822fa94ada983aa7266df4bf9d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.007845 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet e1aeae8c18b6499e9dc1accabdc3f633 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.009652 3029 tablet.cc:1722] T 645b08b33676420c96e880de3d460829 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:49.010867 3110 tablet.cc:1722] T a3b1aadf17ea4bf0a896d23cd543fca7 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.014012 1795 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46568 (request call id 95) took 1134 ms. Trace:
I20251211 20:52:49.014889 1795 rpcz_store.cc:276] 1211 20:52:47.879170 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:48.926738 (+1047568us) service_pool.cc:225] Handling call
1211 20:52:48.929052 (+ 2314us) catalog_manager.cc:2101] Acquired catalog manager lock
1211 20:52:48.929594 (+ 542us) catalog_manager.cc:2147] Created new table and tablet info
1211 20:52:49.013738 (+ 84144us) catalog_manager.cc:2207] Wrote table and tablets to system table
1211 20:52:49.013855 (+ 117us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
1211 20:52:49.013964 (+ 109us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:48.992388 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:48.992520 (+ 132us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:48.992527 (+ 7us) write_op.cc:435] Acquired schema lock
1211 20:52:48.992530 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:48.992651 (+ 121us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:48.992668 (+ 17us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:48.992674 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:48.992757 (+ 83us) tablet.cc:699] Row locks acquired
1211 20:52:48.992759 (+ 2us) write_op.cc:260] PREPARE: finished
1211 20:52:48.992799 (+ 40us) write_op.cc:270] Start()
1211 20:52:48.992835 (+ 36us) write_op.cc:276] Timestamp: P: 1765486368992488 usec, L: 0
1211 20:52:48.992839 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:48.993031 (+ 192us) log.cc:844] Serialized 433 byte log entry
1211 20:52:48.996524 (+ 3493us) op_driver.cc:464] REPLICATION: finished
1211 20:52:48.996663 (+ 139us) write_op.cc:301] APPLY: starting
1211 20:52:48.996679 (+ 16us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:48.996725 (+ 46us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:48.996749 (+ 24us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:48.996851 (+ 102us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:48.996886 (+ 35us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1211 20:52:48.996893 (+ 7us) write_op.cc:312] APPLY: finished
1211 20:52:48.996965 (+ 72us) log.cc:844] Serialized 29 byte log entry
1211 20:52:48.997028 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.997049 (+ 21us) write_op.cc:454] Released schema lock
1211 20:52:48.997056 (+ 7us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:48.997127 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.997131 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":93,"apply.run_cpu_time_us":455,"apply.run_wall_time_us":454,"num_ops":2,"prepare.queue_time_us":43,"prepare.run_cpu_time_us":771,"prepare.run_wall_time_us":771,"raft.queue_time_us":605,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":268,"replication_time_us":3657}]]}
I20251211 20:52:49.020118 1795 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_3d02ad4ef2a44392bf94b4aaac7cba48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:49.029268 1160 tablet_bootstrap.cc:654] T d37a7d0d662e45079b0cfc02765d2e01 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:49.035221 1795 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3d02ad4ef2a44392bf94b4aaac7cba48 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:49.036888 3124 tablet.cc:1722] T 9f8aa28c1c9b4fc58dfc128952c180ba P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.039729 742 tablet_service.cc:1505] Processing CreateTablet for tablet 2f81a30cd47c4af08761c9de9fff97f8 (DEFAULT_TABLE table=default.Table_bffc8b16e4414214b9e8b77194575e67 [id=479500af29b14703b8b6fc14b4540124]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:49.040150 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f81a30cd47c4af08761c9de9fff97f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:49.038388 880 tablet_service.cc:1505] Processing CreateTablet for tablet 2f81a30cd47c4af08761c9de9fff97f8 (DEFAULT_TABLE table=default.Table_bffc8b16e4414214b9e8b77194575e67 [id=479500af29b14703b8b6fc14b4540124]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:49.040540 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f81a30cd47c4af08761c9de9fff97f8. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:49.040584 711 consensus_peers.cc:597] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:49.040822 2974 raft_consensus.cc:493] T f91487dd7e0f4c8b8483c2b83945da3b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:49.041008 2974 raft_consensus.cc:515] T f91487dd7e0f4c8b8483c2b83945da3b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.041633 2974 leader_election.cc:290] T f91487dd7e0f4c8b8483c2b83945da3b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:49.042220 903 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f91487dd7e0f4c8b8483c2b83945da3b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:49.042721 983 leader_election.cc:343] T f91487dd7e0f4c8b8483c2b83945da3b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:49.054651 1166 tablet_bootstrap.cc:492] T 3f1840e4b5214ddb878935a79b214eae P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:49.059196 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet 042f8bf13a7a4712b5bdc04671cd9938 with delete_type TABLET_DATA_DELETED (replaced by tablet 53b4116db0a44dfbb4c1ec5b2acebc67) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.059778 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet 84ed434868aa437a83455b1605deb26c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.060220 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet 4810f8d8b53a45eb939bf3c8bf1b4e77 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.060607 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet c7bd218b3ff340b2a589c9517b49a649 with delete_type TABLET_DATA_DELETED (replaced by tablet dd1ae033dedc436fa3a3f864a7a208a2) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.060971 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet 07e4336171714da8adc21977ce33ce50 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.061339 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet eb9da081cb9d43fd9a35c4bc51ea133e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.061702 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet 06b8f8e46e524545aeab6c931efaf116 with delete_type TABLET_DATA_DELETED (replaced by tablet 7bc03ed8c35740f58fe48ddbaa120ea0) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.062064 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b1ce6d2a12a4c7ea044b349412b4bf5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.062429 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet e95b0d8c507a4779a087c8933d464228 with delete_type TABLET_DATA_DELETED (replaced by tablet 9c5c937cc7db4ed18fe03f4a8a99a66a) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.062794 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet 877aa237fe43452cbdbb4dac46f613ae with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.063407 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet 28ddb8a59a214576a25d613d42c2fef5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.063798 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet 7452dd736b0440fcb4626858e794676f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.063210 880 tablet_service.cc:1505] Processing CreateTablet for tablet f5818d01760546c7a7dbc2aba74af67b (DEFAULT_TABLE table=default.Table_a1e4b060f2ff4ebab9bc8d88d25aa4c8 [id=c8402eee95af4440bef63c757fcf341f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:49.064149 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet 3562b5effa124378beac7ccf0d27ee7e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.064478 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet 14f367adb60744c8a593d3b9d4711823 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.064843 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet e9ead34065e240eeb1f946319537d32a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.064917 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5818d01760546c7a7dbc2aba74af67b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:49.065227 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet 3621735fcb294e3a8c0180ba9ef86442 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.065569 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet 3acfef6f082641a199b80eefcfc772ac with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.065922 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet 59e955aa07c14fbbb347b88eca59a74e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:36220
W20251211 20:52:49.067977 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet aa50ddb2789b4be3bfa9aedef9287109 (table default.Table_44f971096d1e41f480e594985ffd5c17 [id=d4e1349374a346a5b8be180f9bdc329a], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet aa50ddb2789b4be3bfa9aedef9287109 consensus state has no leader
W20251211 20:52:49.068311 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet f94b656e3c1844509fdae36e33b263e8 (table default.Table_ed03ad475bed4804a65abe20efef0625 [id=e23f4305669a474aafa18928f130de3a], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet f94b656e3c1844509fdae36e33b263e8 consensus state has no leader
W20251211 20:52:49.068862 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 4c7b74f370b042ee8575f821c07dc085 (table default.Table_731826f26f9940b8afe425d694b642d9 [id=17dd9cea9ac844128ff9f82adb3f919f], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 4c7b74f370b042ee8575f821c07dc085 consensus state has no leader
I20251211 20:52:49.069919 3000 raft_consensus.cc:493] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:49.070077 3000 raft_consensus.cc:515] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:49.070474 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 8dc4e730951144d78b77280f6cce6d0e (table default.Table_890c9027415a44ccaf95c0adede51992 [id=7bc8a2b91fe54b91b6594b371fea7bd3], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 8dc4e730951144d78b77280f6cce6d0e consensus state has no leader
I20251211 20:52:49.070744 3000 leader_election.cc:290] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:49.071228 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 969a8587019c4202b6e705d126f83d44 (table default.Table_6460313002684bba93b30ad8631f0f49 [id=22347b15537d451d9927a17e8e5a202a], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 969a8587019c4202b6e705d126f83d44 consensus state has no leader
I20251211 20:52:49.071401 903 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "962b80ae747b4470abd23643e711fad9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:49.072230 711 leader_election.cc:343] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:49.074177 1166 tablet_bootstrap.cc:654] T 3f1840e4b5214ddb878935a79b214eae P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.078404 1160 tablet_bootstrap.cc:492] T d37a7d0d662e45079b0cfc02765d2e01 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:49.079394 1160 ts_tablet_manager.cc:1403] T d37a7d0d662e45079b0cfc02765d2e01 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.101s user 0.001s sys 0.003s
I20251211 20:52:49.080053 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet c97d410272a947de8b5d7fb40fe4777d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.080870 1160 raft_consensus.cc:359] T d37a7d0d662e45079b0cfc02765d2e01 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.082216 1160 raft_consensus.cc:385] T d37a7d0d662e45079b0cfc02765d2e01 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.082440 1160 raft_consensus.cc:740] T d37a7d0d662e45079b0cfc02765d2e01 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.082681 1168 tablet_bootstrap.cc:492] T 7a0a20f95416425489af94408ac1e97c P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:49.082850 1168 ts_tablet_manager.cc:1403] T 7a0a20f95416425489af94408ac1e97c P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.277s user 0.037s sys 0.041s
I20251211 20:52:49.083076 1160 consensus_queue.cc:260] T d37a7d0d662e45079b0cfc02765d2e01 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.085150 1160 ts_tablet_manager.cc:1434] T d37a7d0d662e45079b0cfc02765d2e01 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251211 20:52:49.087226 1160 tablet_bootstrap.cc:492] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:49.087992 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 3dfea28c44d84c68850db65a30d52ee6 (table default.Table_c504f8f3796f43b697488bd604268543 [id=2af76b9d32094a4ca656060a4478e08a], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 3dfea28c44d84c68850db65a30d52ee6 consensus state has no leader
W20251211 20:52:49.088212 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 59acb8474e114bf7ae7f1f043d611459 (table default.Table_c7ed8c0c309c40a3a371ffa130b47cd2 [id=ed1d0670426945aba888b818ad657225], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 59acb8474e114bf7ae7f1f043d611459 consensus state has no leader
I20251211 20:52:49.088322 1795 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46568 (request call id 96) took 1167 ms. Trace:
I20251211 20:52:49.089570 742 tablet_service.cc:1505] Processing CreateTablet for tablet f5818d01760546c7a7dbc2aba74af67b (DEFAULT_TABLE table=default.Table_a1e4b060f2ff4ebab9bc8d88d25aa4c8 [id=c8402eee95af4440bef63c757fcf341f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:49.089941 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet 88970b575ba943129729eceef5636519 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.090056 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5818d01760546c7a7dbc2aba74af67b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:49.089895 1795 rpcz_store.cc:276] 1211 20:52:47.921164 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.019905 (+1098741us) service_pool.cc:225] Handling call
1211 20:52:49.036121 (+ 16216us) catalog_manager.cc:2101] Acquired catalog manager lock
1211 20:52:49.050184 (+ 14063us) catalog_manager.cc:2147] Created new table and tablet info
1211 20:52:49.076154 (+ 25970us) catalog_manager.cc:2207] Wrote table and tablets to system table
1211 20:52:49.087867 (+ 11713us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
1211 20:52:49.088281 (+ 414us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:49.050779 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:49.050917 (+ 138us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:49.050924 (+ 7us) write_op.cc:435] Acquired schema lock
1211 20:52:49.050926 (+ 2us) tablet.cc:660] Decoding operations
1211 20:52:49.051020 (+ 94us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:49.051035 (+ 15us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:49.051040 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:49.051133 (+ 93us) tablet.cc:699] Row locks acquired
1211 20:52:49.051135 (+ 2us) write_op.cc:260] PREPARE: finished
1211 20:52:49.051172 (+ 37us) write_op.cc:270] Start()
1211 20:52:49.051208 (+ 36us) write_op.cc:276] Timestamp: P: 1765486369050856 usec, L: 0
1211 20:52:49.051212 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:49.051402 (+ 190us) log.cc:844] Serialized 433 byte log entry
1211 20:52:49.061694 (+ 10292us) op_driver.cc:464] REPLICATION: finished
1211 20:52:49.061912 (+ 218us) write_op.cc:301] APPLY: starting
1211 20:52:49.061932 (+ 20us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:49.061980 (+ 48us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:49.062006 (+ 26us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:49.062130 (+ 124us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:49.062168 (+ 38us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1211 20:52:49.062176 (+ 8us) write_op.cc:312] APPLY: finished
1211 20:52:49.062264 (+ 88us) log.cc:844] Serialized 29 byte log entry
1211 20:52:49.062331 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:49.062355 (+ 24us) write_op.cc:454] Released schema lock
1211 20:52:49.062363 (+ 8us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:49.062442 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:49.062446 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":137,"apply.run_cpu_time_us":540,"apply.run_wall_time_us":539,"num_ops":2,"prepare.queue_time_us":44,"prepare.run_cpu_time_us":740,"prepare.run_wall_time_us":740,"raft.queue_time_us":753,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":616,"replication_time_us":10439}]]}
I20251211 20:52:49.090760 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet d15eff49af284a38bb1babdb231a35df with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.091475 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet a471a9accbfc4875bd1d7802f5620808 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.091411 3130 tablet.cc:1722] T c7a2610f20df4c1ca81eae2fc7dba6bb P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.091933 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.092324 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet aeff1fbabf7845dc9863f59f286ddcec with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.092713 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.039966 3132 tablet.cc:1722] T 676faec7f3d1488f99b220f8e968e9a0 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:49.097388 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet c8214c97ed1640f5965aa5c4b999e2f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.097826 1795 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46568:
name: "default.Table_7ea32bac879c48dd90e4a2b54f38a61a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:52:49.099771 2962 ts_tablet_manager.cc:1929] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:49.101022 2962 log.cc:1199] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/6a61cf8f5de14adaa4a1de5bddde9a2f
I20251211 20:52:49.101827 2962 ts_tablet_manager.cc:1950] T 6a61cf8f5de14adaa4a1de5bddde9a2f P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:49.102407 886 tablet_service.cc:1505] Processing CreateTablet for tablet 18725296eff744ff82ce7e1c4403bd97 (DEFAULT_TABLE table=default.Table_18f6559b16734cbaa5659427c4276f5d [id=f32f878e54f34bc6b54ed80c4d9385e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:49.102855 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18725296eff744ff82ce7e1c4403bd97. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:49.103969 1168 raft_consensus.cc:359] T 7a0a20f95416425489af94408ac1e97c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.104214 1168 raft_consensus.cc:385] T 7a0a20f95416425489af94408ac1e97c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.104302 1168 raft_consensus.cc:740] T 7a0a20f95416425489af94408ac1e97c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.104544 1168 consensus_queue.cc:260] T 7a0a20f95416425489af94408ac1e97c P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.105090 1168 ts_tablet_manager.cc:1434] T 7a0a20f95416425489af94408ac1e97c P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.022s user 0.001s sys 0.001s
W20251211 20:52:49.110769 711 consensus_peers.cc:597] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:49.112113 1160 tablet_bootstrap.cc:654] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:49.114624 711 consensus_peers.cc:597] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:49.106676 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet a295a24e81f14e5fa126a4a58ffd08cf with delete_type TABLET_DATA_DELETED (replaced by tablet 77bd176108f24e79a632c798566e9e31) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.106705 3142 tablet.cc:1722] T f2d6c9781b4d4c37bc08703347963d5d P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.107240 1024 tablet_service.cc:1505] Processing CreateTablet for tablet 2f81a30cd47c4af08761c9de9fff97f8 (DEFAULT_TABLE table=default.Table_bffc8b16e4414214b9e8b77194575e67 [id=479500af29b14703b8b6fc14b4540124]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:49.116151 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f81a30cd47c4af08761c9de9fff97f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:49.107931 1015 tablet_service.cc:1552] Processing DeleteTablet for tablet 6d4e6528352f43b6a1b47d7a4b6b3689 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.108276 1021 tablet_service.cc:1552] Processing DeleteTablet for tablet 1944267def4e46949fd680ab0a369be9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.108783 1019 tablet_service.cc:1552] Processing DeleteTablet for tablet c26c3d0170f64a41ae8944c6668cc5f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.119563 3110 tablet.cc:1722] T 645b08b33676420c96e880de3d460829 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:49.108939 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 7b06cd366de5494780b5ac81abbf0702 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.109030 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 5bcc94ec9630484cb7a3e8bf6d4f378a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.109103 1016 tablet_service.cc:1552] Processing DeleteTablet for tablet 3dde7ed495e64909bd9784dc5eefb04c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.109539 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 58e35de0792a464a818170886ca8d8c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.111716 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 215f8dc74a71482daa2ca80e15010f51 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.111856 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet f3855ffb51a74101b26fd28429e65bc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:36220
W20251211 20:52:49.089152 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 16d2ff8925e342729a71c93db378d031 (table default.Table_34e6ed4f13da437ea50d349b92e83bd6 [id=4a74b9a60cbb4198b55cee630b2f4bef], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 16d2ff8925e342729a71c93db378d031 consensus state has no leader
I20251211 20:52:49.122850 1168 tablet_bootstrap.cc:492] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:49.123929 742 tablet_service.cc:1505] Processing CreateTablet for tablet 18725296eff744ff82ce7e1c4403bd97 (DEFAULT_TABLE table=default.Table_18f6559b16734cbaa5659427c4276f5d [id=f32f878e54f34bc6b54ed80c4d9385e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:49.124336 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18725296eff744ff82ce7e1c4403bd97. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:49.124689 1168 tablet_bootstrap.cc:654] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.130237 2962 tablet_replica.cc:333] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
W20251211 20:52:49.130427 711 consensus_peers.cc:597] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:49.130831 2962 raft_consensus.cc:2243] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:49.131079 2962 raft_consensus.cc:2272] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:49.131749 2962 ts_tablet_manager.cc:1916] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:49.133531 1795 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7ea32bac879c48dd90e4a2b54f38a61a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:52:49.134038 3132 tablet.cc:1722] T f2d6c9781b4d4c37bc08703347963d5d P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.134887 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet efe4d6bb07374294b8ab50a9fe605f87 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.135360 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 32f8d104f6a948d1a97a83769ea82449 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.135848 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 940dfd8781de444a9caa78dd363c7451 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.136219 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet d69fb1f6d9594afcbfb844860b41a377 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.136837 1022 tablet_service.cc:1505] Processing CreateTablet for tablet f5818d01760546c7a7dbc2aba74af67b (DEFAULT_TABLE table=default.Table_a1e4b060f2ff4ebab9bc8d88d25aa4c8 [id=c8402eee95af4440bef63c757fcf341f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:49.137212 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5818d01760546c7a7dbc2aba74af67b. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:49.139815 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 5773c13034714a008c8d7dbe777b4828 (table default.Table_66eb1b021eed464f82f9cd223c88297b [id=94ff09564cd649c7a2d0e94b0f3c7b3a], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 5773c13034714a008c8d7dbe777b4828 consensus state has no leader
W20251211 20:52:49.140096 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 78f77d8916d84edabbe95ce17d8df2c1 (table default.Table_e2af0e2fbd3d46d595136c7a7bf24e4c [id=ddc12ae6d8ed4587b5ef4da3ef81a055], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 78f77d8916d84edabbe95ce17d8df2c1 consensus state has no leader
I20251211 20:52:49.145320 886 tablet_service.cc:1505] Processing CreateTablet for tablet 9ae2a68a1d0c45ae93d42472221bdc65 (DEFAULT_TABLE table=default.Table_43e9cb840d48444b9360e2fa8671dfb8 [id=f6cab1da32324fe6afc1df2007f0bdbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:49.145939 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ae2a68a1d0c45ae93d42472221bdc65. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:49.152792 3142 tablet.cc:1722] T 5453f68789fd4e89b69e98860050865b P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20251211 20:52:49.153780 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet e3f711f02af843adb4f5c3c750924e33 (table default.Table_34bd5155aa3b4ef7818022ba9346c074 [id=83a61d23e80946439a14096ab3ab1bd9], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet e3f711f02af843adb4f5c3c750924e33 consensus state has no leader
I20251211 20:52:49.154271 906 raft_consensus.cc:3060] T 215f8dc74a71482daa2ca80e15010f51 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:49.154424 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet b6cc819332e1445c96ce3ffac4861454 (table default.Table_db4c0f706e0643b39d39ffa84352826d [id=4d459c42bf584068ac3851227d80331a], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet b6cc819332e1445c96ce3ffac4861454 consensus state has no leader
W20251211 20:52:49.155074 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet e89d4d76bf64453892a4d0b55fcc3454 (table default.Table_fba99384e9f14160b3e3504f8d1d0b07 [id=01bba75f854d4c338d6c09a446dd7499], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet e89d4d76bf64453892a4d0b55fcc3454 consensus state has no leader
W20251211 20:52:49.155468 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 7ca98b43863a47c091baae0a6ce65630 (table default.Table_f0a70914acc94231a4e84155c3e4d13f [id=56c070ad80624c698ebe2e01b4929b6e], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 7ca98b43863a47c091baae0a6ce65630 consensus state has no leader
W20251211 20:52:49.155992 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet d6923cfd826d495291d9ac8e0b3b44c6 (table default.Table_68ccb2052d824ccf92f7162d2961fcb1 [id=9ac9591d4f04482d926710fdf6e797c6], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet d6923cfd826d495291d9ac8e0b3b44c6 consensus state has no leader
W20251211 20:52:49.156632 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 33fd09ea06ee47908b85911e4ee79708 (table default.Table_3c3cd48c8fe5411e96cdf811d492925d [id=30620a01c27f456bac05fa7c02a2cb4d], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 33fd09ea06ee47908b85911e4ee79708 consensus state has no leader
W20251211 20:52:49.156797 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 6212b3b4425543168837946a54d60834 (table default.Table_8c30c4151d484db0bb9e2b9ca262248d [id=2d489cd7d73e403d84bf10960e6a2dfb], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 6212b3b4425543168837946a54d60834 consensus state has no leader
I20251211 20:52:49.161994 3124 tablet.cc:1722] T 5453f68789fd4e89b69e98860050865b P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.167398 757 tablet_service.cc:1505] Processing CreateTablet for tablet 200fbba33f8947cca99ddc81e37dfc28 (DEFAULT_TABLE table=default.Table_6d9aa092c758405fa974ff1675c60a8a [id=78f84aeb9f40475f91cdf41fd040b400]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:49.169669 887 tablet_service.cc:1505] Processing CreateTablet for tablet 80486aaeb28849c58c855475871853c0 (DEFAULT_TABLE table=default.Table_f2eaa62388894072a390cc4c167f3385 [id=ac6774893a6a4f82a7c9630bbe502fe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:49.170104 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80486aaeb28849c58c855475871853c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:49.171468 742 tablet_service.cc:1505] Processing CreateTablet for tablet 9ae2a68a1d0c45ae93d42472221bdc65 (DEFAULT_TABLE table=default.Table_43e9cb840d48444b9360e2fa8671dfb8 [id=f6cab1da32324fe6afc1df2007f0bdbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:49.171519 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet ca89f09cc0954389b679e8bb8b5e4dfb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.171975 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 01104975fd8b470aa026578b97ff6b8d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.172477 1022 tablet_service.cc:1505] Processing CreateTablet for tablet 18725296eff744ff82ce7e1c4403bd97 (DEFAULT_TABLE table=default.Table_18f6559b16734cbaa5659427c4276f5d [id=f32f878e54f34bc6b54ed80c4d9385e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:49.172874 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18725296eff744ff82ce7e1c4403bd97. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:49.176584 1160 tablet_bootstrap.cc:492] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:49.176826 1160 ts_tablet_manager.cc:1403] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.091s user 0.004s sys 0.004s
I20251211 20:52:49.177678 1160 raft_consensus.cc:359] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.177947 1160 raft_consensus.cc:385] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.178040 1160 raft_consensus.cc:740] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.178269 1160 consensus_queue.cc:260] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.178860 1160 ts_tablet_manager.cc:1434] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:49.178294 880 tablet_service.cc:1505] Processing CreateTablet for tablet 200fbba33f8947cca99ddc81e37dfc28 (DEFAULT_TABLE table=default.Table_6d9aa092c758405fa974ff1675c60a8a [id=78f84aeb9f40475f91cdf41fd040b400]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:49.179473 880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 200fbba33f8947cca99ddc81e37dfc28. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:49.179724 756 tablet_service.cc:1505] Processing CreateTablet for tablet 80486aaeb28849c58c855475871853c0 (DEFAULT_TABLE table=default.Table_f2eaa62388894072a390cc4c167f3385 [id=ac6774893a6a4f82a7c9630bbe502fe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:49.181572 1795 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46568 (request call id 97) took 1241 ms. Trace:
I20251211 20:52:49.182106 1795 rpcz_store.cc:276] 1211 20:52:47.940139 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.097587 (+1157448us) service_pool.cc:225] Handling call
1211 20:52:49.133662 (+ 36075us) catalog_manager.cc:2101] Acquired catalog manager lock
1211 20:52:49.134182 (+ 520us) catalog_manager.cc:2147] Created new table and tablet info
1211 20:52:49.179652 (+ 45470us) catalog_manager.cc:2207] Wrote table and tablets to system table
1211 20:52:49.179767 (+ 115us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
1211 20:52:49.181528 (+ 1761us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:49.162976 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:49.163068 (+ 92us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:49.163074 (+ 6us) write_op.cc:435] Acquired schema lock
1211 20:52:49.163077 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:49.163171 (+ 94us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:49.163185 (+ 14us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:49.163190 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:49.163269 (+ 79us) tablet.cc:699] Row locks acquired
1211 20:52:49.163278 (+ 9us) write_op.cc:260] PREPARE: finished
1211 20:52:49.163315 (+ 37us) write_op.cc:270] Start()
1211 20:52:49.163351 (+ 36us) write_op.cc:276] Timestamp: P: 1765486369162988 usec, L: 0
1211 20:52:49.163355 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:49.163536 (+ 181us) log.cc:844] Serialized 433 byte log entry
1211 20:52:49.167092 (+ 3556us) op_driver.cc:464] REPLICATION: finished
1211 20:52:49.167240 (+ 148us) write_op.cc:301] APPLY: starting
1211 20:52:49.167256 (+ 16us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:49.167305 (+ 49us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:49.167351 (+ 46us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:49.167466 (+ 115us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:49.167503 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1211 20:52:49.167510 (+ 7us) write_op.cc:312] APPLY: finished
1211 20:52:49.167760 (+ 250us) log.cc:844] Serialized 29 byte log entry
1211 20:52:49.167827 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:49.167859 (+ 32us) write_op.cc:454] Released schema lock
1211 20:52:49.167866 (+ 7us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:49.167940 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:49.167943 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":90,"apply.run_cpu_time_us":613,"apply.run_wall_time_us":699,"num_ops":2,"prepare.queue_time_us":52,"prepare.run_cpu_time_us":710,"prepare.run_wall_time_us":710,"raft.queue_time_us":741,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":220,"replication_time_us":3707}]]}
I20251211 20:52:49.186189 1795 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_a8ebfd7d7a7f4f6cae8b6b02b99b80b0" } new_table_name: "default.Table_7833c67862334235b0e752bfcf44ff81" modify_external_catalogs: true
I20251211 20:52:49.191350 2973 raft_consensus.cc:493] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:49.191509 2973 raft_consensus.cc:515] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.193267 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet 76cfb26762d54593a150ec98125c6592 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.193374 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet d53b074865f4490693152a247b0fffdc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.193549 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet b30d2054316b49fa8096345527a45d87 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.193653 2973 leader_election.cc:290] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:49.193756 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 4a39870c7eea4a379b284ea2e5ddf760 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.193730 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet f9a3fdf17f274c15ac0bdfa9c174f649 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.194010 1016 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a206abd6ea2420089cb07714e0d19c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.194054 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet 47779034ad89466e9e1801987128b4dd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.194203 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 50f9f29c064c4f938a0f20f302ab5e8a with delete_type TABLET_DATA_DELETED (replaced by tablet da81bf8854a24954ba4548185812295d) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.194617 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 7bc03ed8c35740f58fe48ddbaa120ea0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.194823 1016 tablet_service.cc:1552] Processing DeleteTablet for tablet a4ccf9a4ecbc4ca6978a189cf3a15893 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.194286 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet c7147571483e41ad970e573a08c368ca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.195050 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 5262c2241e524395a73c0eb368917169 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.195168 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 098054406a1d459b824a094f50e71e7e with delete_type TABLET_DATA_DELETED (replaced by tablet aa50ddb2789b4be3bfa9aedef9287109) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.195617 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 200fbba33f8947cca99ddc81e37dfc28. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:49.195798 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet 2e29df291d8d40daba83e1d9f97f902b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.196310 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet 6ad6ab53aad14cc2b6d201acfecf6723 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.196543 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 80df719919a44aba9b4e3c62d6830cf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.199172 1010 tablet_service.cc:1505] Processing CreateTablet for tablet 80486aaeb28849c58c855475871853c0 (DEFAULT_TABLE table=default.Table_f2eaa62388894072a390cc4c167f3385 [id=ac6774893a6a4f82a7c9630bbe502fe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:49.199559 1160 tablet_bootstrap.cc:492] T 3562b5effa124378beac7ccf0d27ee7e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:49.199661 1010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80486aaeb28849c58c855475871853c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:49.202273 1160 tablet_bootstrap.cc:654] T 3562b5effa124378beac7ccf0d27ee7e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.203879 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet 41c30bded6504bbc9f0af8f34dc81e8d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.204429 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 71dc8d20d93243e289330d7b8c635511 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.204715 1016 tablet_service.cc:1552] Processing DeleteTablet for tablet 7d1cc9809ab843e693a179a35c53ce25 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.206454 3000 raft_consensus.cc:493] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:49.206589 3000 raft_consensus.cc:515] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.207314 3000 leader_election.cc:290] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:49.207645 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 9ae2a68a1d0c45ae93d42472221bdc65 (DEFAULT_TABLE table=default.Table_43e9cb840d48444b9360e2fa8671dfb8 [id=f6cab1da32324fe6afc1df2007f0bdbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:49.208132 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ae2a68a1d0c45ae93d42472221bdc65. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:49.208235 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "32f8d104f6a948d1a97a83769ea82449" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:49.209152 711 leader_election.cc:343] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:49.211910 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 200fbba33f8947cca99ddc81e37dfc28 (DEFAULT_TABLE table=default.Table_6d9aa092c758405fa974ff1675c60a8a [id=78f84aeb9f40475f91cdf41fd040b400]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:49.212306 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 200fbba33f8947cca99ddc81e37dfc28. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:49.212556 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 7be44df39a8f4d6da793479d425fa684 (table default.Table_f6a10556ff104db5bd5ecc9fc380e228 [id=f75813add54e46189ff1bbed82d83ba7], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 7be44df39a8f4d6da793479d425fa684 consensus state has no leader
W20251211 20:52:49.212754 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 497c10742f01476db885bd70c4d2c491 (table default.Table_18b21232dfa74378b27defe672bb2574 [id=602d7eee703948d99a0edcb7cac0651b], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 497c10742f01476db885bd70c4d2c491 consensus state has no leader
W20251211 20:52:49.212931 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 3003ef71a3554c71a3b24cdfb9c293bc (table default.Table_92efd469dd324f7d886e2cbc90ece61c [id=6e7fb3271e0c48c0bb9d56fe4fccee3a], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 3003ef71a3554c71a3b24cdfb9c293bc consensus state has no leader
W20251211 20:52:49.213649 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet fd43a27db69044b68826cda87e16a086 (table default.Table_a8ebfd7d7a7f4f6cae8b6b02b99b80b0 [id=f1d32002c612403a82bdff5baf209b9e], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet fd43a27db69044b68826cda87e16a086 consensus state has no leader
W20251211 20:52:49.214349 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 94658fcf512a48d0a992a7e2e0f7c176 (table default.Table_c9d86e82aebb4ef0bc83789720352e38 [id=97377a8ef56c4cdebe95d15955207404], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 94658fcf512a48d0a992a7e2e0f7c176 consensus state has no leader
W20251211 20:52:49.214546 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet fb29f983c4524a49bdc5c0c1144ad708 (table default.Table_7809ac85fdfe4c1d9c58afb54edecb27 [id=c4b0bfff99f64888beb70d9154d2f156], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet fb29f983c4524a49bdc5c0c1144ad708 consensus state has no leader
I20251211 20:52:49.217041 961 heartbeater.cc:499] Master 127.0.104.61:40749 was elected leader, sending a full tablet report...
W20251211 20:52:49.219159 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 4fe5c534b572491c911aae01eaa79bae (table default.Table_c1f9dca5c2144d50b67681dcc6800e77 [id=14fa8debfe5443ce81587f02dfa15847], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 4fe5c534b572491c911aae01eaa79bae consensus state has no leader
I20251211 20:52:49.220026 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80486aaeb28849c58c855475871853c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:49.221931 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet 95866de91f9d437d80f40b7962253610 with delete_type TABLET_DATA_DELETED (replaced by tablet 51d12723f5194afb97d2c05fcbaf3776) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.222991 710 leader_election.cc:304] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:49.223404 2959 raft_consensus.cc:2804] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:49.223533 2959 raft_consensus.cc:697] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:49.223871 2959 consensus_queue.cc:237] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.231855 742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ae2a68a1d0c45ae93d42472221bdc65. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:49.234727 1166 tablet_bootstrap.cc:492] T 3f1840e4b5214ddb878935a79b214eae P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
W20251211 20:52:49.236251 711 consensus_peers.cc:597] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:49.238425 710 leader_election.cc:304] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:49.238693 3000 raft_consensus.cc:2804] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:49.239394 3000 raft_consensus.cc:697] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:49.239735 3000 consensus_queue.cc:237] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:49.240638 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 516f7181aaac48cdadf9fbc08d0d3f57 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 516f7181aaac48cdadf9fbc08d0d3f57 already in progress: creating tablet
I20251211 20:52:49.242475 1166 ts_tablet_manager.cc:1403] T 3f1840e4b5214ddb878935a79b214eae P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.181s user 0.002s sys 0.005s
I20251211 20:52:49.243526 1166 raft_consensus.cc:359] T 3f1840e4b5214ddb878935a79b214eae P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.243954 1166 raft_consensus.cc:385] T 3f1840e4b5214ddb878935a79b214eae P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20251211 20:52:49.244012 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet e95c63b4d99a4dd0b2c0fa384ca9d552 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e95c63b4d99a4dd0b2c0fa384ca9d552 already in progress: creating tablet
I20251211 20:52:49.244086 1166 raft_consensus.cc:740] T 3f1840e4b5214ddb878935a79b214eae P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.244468 1166 consensus_queue.cc:260] T 3f1840e4b5214ddb878935a79b214eae P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:49.245025 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 51d12723f5194afb97d2c05fcbaf3776 (table default.Table_823505200e3142dbbe8cd56ac78daf55 [id=ec0bb10933fa4529bef3580bee6409e7], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 51d12723f5194afb97d2c05fcbaf3776 consensus state has no leader
I20251211 20:52:49.248201 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 6029c1fd563d4023a8cf7cfb8ffa0876 (table default.Table_0ec526100c24487b808171d9734bcfb7 [id=8f7b8c70bd6f42b8817de464b5b7aac3]) successfully deleted
I20251211 20:52:49.248401 710 leader_election.cc:304] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:49.248908 1780 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 2e33606930774fe4aafc6e3819618af8 because the tablet was not found. No further retry: Not found: Tablet not found: 2e33606930774fe4aafc6e3819618af8
I20251211 20:52:49.249199 2429 raft_consensus.cc:2804] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:49.249333 2429 raft_consensus.cc:697] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:49.249570 2429 consensus_queue.cc:237] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:49.252219 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 2e800d8cc5ea4535a065d7d9243326b4 (table default.Table_083c478bedec45a5bf5203e4ed333860 [id=8baf0ea87d734d58896adbd0d6ef99c2], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 2e800d8cc5ea4535a065d7d9243326b4 consensus state has no leader
W20251211 20:52:49.254458 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 8675e2a18f0f4a0e95a87005cf82d8b4 (table default.Table_bd9633c55bd242c18aac3a7f2c18a329 [id=767f50bee5464316a18f73e64bab817a], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 8675e2a18f0f4a0e95a87005cf82d8b4 consensus state has no leader
I20251211 20:52:49.257300 710 leader_election.cc:304] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:49.257778 3029 tablet.cc:1722] T ee7b93b0b87541d9af6eb9e68b4cea41 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:49.258292 3054 raft_consensus.cc:2804] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:49.258368 3054 raft_consensus.cc:493] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:49.258422 3054 raft_consensus.cc:3060] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:49.259172 3133 tablet.cc:1722] T ce1799e6604f4ed7875ddedecab6f327 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.257834 710 leader_election.cc:304] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:49.260619 2494 raft_consensus.cc:2804] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:49.260694 2494 raft_consensus.cc:493] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:49.260743 2494 raft_consensus.cc:3060] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:49.262024 1166 ts_tablet_manager.cc:1434] T 3f1840e4b5214ddb878935a79b214eae P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.019s user 0.002s sys 0.000s
I20251211 20:52:49.263204 1166 tablet_bootstrap.cc:492] T 4b07f110f25645799ed0c89737cd8279 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:49.263796 710 leader_election.cc:304] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:49.264019 2959 raft_consensus.cc:2804] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:49.264240 710 leader_election.cc:304] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:49.264626 3000 raft_consensus.cc:2804] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:49.264696 3000 raft_consensus.cc:493] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:49.264750 3000 raft_consensus.cc:3060] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:49.264804 847 leader_election.cc:343] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 0b267c7feaac43f6aacb7229759c7ec3
I20251211 20:52:49.266774 1168 tablet_bootstrap.cc:492] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:49.266939 1168 ts_tablet_manager.cc:1403] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.144s user 0.003s sys 0.000s
I20251211 20:52:49.268396 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 6a61cf8f5de14adaa4a1de5bddde9a2f (table default.Table_b06f9587f78645bb9712c3b080f3c0d4 [id=ea12bdaa894f446da22635f95d97ac2a]) successfully deleted
I20251211 20:52:49.268918 770 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f91487dd7e0f4c8b8483c2b83945da3b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:49.269084 771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "834b84a557b04e78b5d2c8c84520a644" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:49.264142 2959 raft_consensus.cc:493] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:49.270004 2959 raft_consensus.cc:3060] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:49.272447 2974 raft_consensus.cc:493] T c081396b492049b996ac4661e1318388 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:49.272608 2974 raft_consensus.cc:515] T c081396b492049b996ac4661e1318388 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.273252 2974 leader_election.cc:290] T c081396b492049b996ac4661e1318388 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:49.273355 2494 raft_consensus.cc:515] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:49.273808 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet f26ba8f4bd3a4a1f8fa2ba6e80d543f8 (table default.Table_71e9c158dc22435a8fbb02aecca8919d [id=b966d0c0b49d4023894950c5dc06f678], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet f26ba8f4bd3a4a1f8fa2ba6e80d543f8 consensus state has no leader
I20251211 20:52:49.273926 2494 leader_election.cc:290] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:49.277110 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 1c4b2c22bd4c456eb77af45576de466f (table default.Table_e3a99b043e4f420a93ebfee6aa87f2ec [id=3e7d6cefcb7d412191c6540e9ad77c2d], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 1c4b2c22bd4c456eb77af45576de466f consensus state has no leader
I20251211 20:52:49.279031 903 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c081396b492049b996ac4661e1318388" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:49.259547 824 heartbeater.cc:499] Master 127.0.104.61:40749 was elected leader, sending a full tablet report...
I20251211 20:52:49.280869 3000 raft_consensus.cc:515] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.281611 3000 leader_election.cc:290] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:49.285596 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet bc4e7362e6d045da81bf39f4135f500e (table default.Table_328b7dd76e9e407c83b47a0f51cfcce5 [id=9e7c08f833e14a90b17a9b9f60b702de], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet bc4e7362e6d045da81bf39f4135f500e consensus state has no leader
I20251211 20:52:49.288343 3054 raft_consensus.cc:515] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.288519 1160 tablet_bootstrap.cc:492] T 3562b5effa124378beac7ccf0d27ee7e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:49.288679 1160 ts_tablet_manager.cc:1403] T 3562b5effa124378beac7ccf0d27ee7e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.089s user 0.003s sys 0.011s
I20251211 20:52:49.289438 1160 raft_consensus.cc:359] T 3562b5effa124378beac7ccf0d27ee7e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.289603 1160 raft_consensus.cc:385] T 3562b5effa124378beac7ccf0d27ee7e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.289650 1160 raft_consensus.cc:740] T 3562b5effa124378beac7ccf0d27ee7e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.289849 1160 consensus_queue.cc:260] T 3562b5effa124378beac7ccf0d27ee7e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.290289 1160 ts_tablet_manager.cc:1434] T 3562b5effa124378beac7ccf0d27ee7e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:49.290585 1160 tablet_bootstrap.cc:492] T fa79b67b9eee45e2b41c4a90967606cc P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:49.290930 3054 leader_election.cc:290] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:49.294217 1160 tablet_bootstrap.cc:654] T fa79b67b9eee45e2b41c4a90967606cc P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:49.295444 983 leader_election.cc:343] T c081396b492049b996ac4661e1318388 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:49.295845 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 3c5819ae1abb48fe8a83af9067726d02 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.299165 1168 raft_consensus.cc:359] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.299341 1168 raft_consensus.cc:385] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.299384 1168 raft_consensus.cc:740] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.299566 1168 consensus_queue.cc:260] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.301810 1160 tablet_bootstrap.cc:492] T fa79b67b9eee45e2b41c4a90967606cc P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:49.301968 1160 ts_tablet_manager.cc:1403] T fa79b67b9eee45e2b41c4a90967606cc P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20251211 20:52:49.302820 1160 raft_consensus.cc:359] T fa79b67b9eee45e2b41c4a90967606cc P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.303010 1160 raft_consensus.cc:385] T fa79b67b9eee45e2b41c4a90967606cc P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.303056 1160 raft_consensus.cc:740] T fa79b67b9eee45e2b41c4a90967606cc P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.303285 1160 consensus_queue.cc:260] T fa79b67b9eee45e2b41c4a90967606cc P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.304198 1160 ts_tablet_manager.cc:1434] T fa79b67b9eee45e2b41c4a90967606cc P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:49.304461 1160 tablet_bootstrap.cc:492] T 0aac6fb3972e4aba8950828c9e49d507 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:49.306890 1160 tablet_bootstrap.cc:654] T 0aac6fb3972e4aba8950828c9e49d507 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.307822 1168 ts_tablet_manager.cc:1434] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.041s user 0.000s sys 0.002s
I20251211 20:52:49.309240 2959 raft_consensus.cc:515] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:49.310828 982 leader_election.cc:343] T f9a3fdf17f274c15ac0bdfa9c174f649 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:49.310953 982 leader_election.cc:304] T f9a3fdf17f274c15ac0bdfa9c174f649 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:49.311154 2974 raft_consensus.cc:2749] T f9a3fdf17f274c15ac0bdfa9c174f649 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:49.312587 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 48050.7us
W20251211 20:52:49.313164 982 leader_election.cc:343] T f91487dd7e0f4c8b8483c2b83945da3b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:49.313277 982 leader_election.cc:304] T f91487dd7e0f4c8b8483c2b83945da3b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:49.313465 2974 raft_consensus.cc:2749] T f91487dd7e0f4c8b8483c2b83945da3b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:49.313628 982 leader_election.cc:343] T 834b84a557b04e78b5d2c8c84520a644 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:49.313704 982 leader_election.cc:304] T 834b84a557b04e78b5d2c8c84520a644 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:49.313880 2974 raft_consensus.cc:2749] T 834b84a557b04e78b5d2c8c84520a644 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:49.327242 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "962b80ae747b4470abd23643e711fad9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:49.327448 1042 raft_consensus.cc:2468] T 962b80ae747b4470abd23643e711fad9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:49.312494 2959 leader_election.cc:290] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:49.335981 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 4cf0cde801bf44449e539d2799b895a7 (table default.Table_907c596bdddc4edca25d14cbacebb23f [id=321235280d1d4fa0bae5cc0333860f85], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 4cf0cde801bf44449e539d2799b895a7 consensus state has no leader
I20251211 20:52:49.337244 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b52377c8ffd741a598e5328a00d876fe (table default.Table_686a26915b42421380037144900dd97c [id=8a1b0c9d641b4a3c83d1ca48576d110d]). Expected version 1 got 0
I20251211 20:52:49.343451 1168 tablet_bootstrap.cc:492] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:49.345284 1168 tablet_bootstrap.cc:654] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.345376 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "32f8d104f6a948d1a97a83769ea82449" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:49.345573 1042 raft_consensus.cc:2468] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:49.351655 3142 tablet.cc:1722] T ce1799e6604f4ed7875ddedecab6f327 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.352747 1040 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "be876471f8bf453aa4467c91891cbbde" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:49.352945 1040 raft_consensus.cc:3060] T be876471f8bf453aa4467c91891cbbde P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:49.355996 1040 raft_consensus.cc:2468] T be876471f8bf453aa4467c91891cbbde P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:49.356045 3166 tablet.cc:1722] T 9f8aa28c1c9b4fc58dfc128952c180ba P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20251211 20:52:49.358279 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 72af55c8045b4f8f8e116a5782794d81 (table default.Table_2cff6fab088c4d939b13b504c133cc12 [id=b6c4b72063b3428593d5650a160dc66b], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 72af55c8045b4f8f8e116a5782794d81 consensus state has no leader
I20251211 20:52:49.358234 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet c7a2610f20df4c1ca81eae2fc7dba6bb (table default.Table_59072235d93b41c0b5cd5e3debcb8ddf [id=9a15a8a075af4969ae2c8b3be1bfed78]). Expected version 1 got 0
W20251211 20:52:49.363108 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet b81a4cef27164824b52695039744db15 (table default.Table_defdeab28749411590cf75b5251f604b [id=11352566d1744193a40a8bb605aa3912], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet b81a4cef27164824b52695039744db15 consensus state has no leader
W20251211 20:52:49.365466 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 7ffa06c4a7ee48b0b337a27aff090592 (table default.Table_647731023b044ebb8954f7fdcc3faf97 [id=4b91abd6e9e94a8a941b33e62e23c083], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 7ffa06c4a7ee48b0b337a27aff090592 consensus state has no leader
I20251211 20:52:49.366334 1040 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01104975fd8b470aa026578b97ff6b8d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:49.365734 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 0302cc8613934c43a0650f081cf98797 (table default.Table_bf827b216f1b43c196923dc1828ff0d0 [id=ed371e073250439fa44330c7101e9387]). Expected version 1 got 0
I20251211 20:52:49.366657 1040 raft_consensus.cc:3060] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:49.369335 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 5b3e8ec3faf445ac9b746e65dc4dbfe0 (table default.Table_7fe80ff5483a4b0292091119980f6639 [id=3c3462a0c09748cea0d2ffb544ae2f13], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 5b3e8ec3faf445ac9b746e65dc4dbfe0 consensus state has no leader
I20251211 20:52:49.370188 1040 raft_consensus.cc:2468] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:49.369668 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 1f4eb1a174c648c897ae2b7612e44585 (table default.Table_e70873b71ba24d7eae548e2acfd243b3 [id=87adcc552a704d56bd2fe812fb3b791f]). Expected version 1 got 0
I20251211 20:52:49.374200 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "be876471f8bf453aa4467c91891cbbde" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:49.374537 1046 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5df87ea7d28340da84ef141c92efc710" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:49.375085 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01104975fd8b470aa026578b97ff6b8d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:49.374864 1046 raft_consensus.cc:3060] T 5df87ea7d28340da84ef141c92efc710 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:49.375456 903 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5df87ea7d28340da84ef141c92efc710" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:49.376070 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "98d575ac680d49d0990877666638f013" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:49.378651 1046 raft_consensus.cc:2468] T 5df87ea7d28340da84ef141c92efc710 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:49.374202 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet a7afcce522494c9ca1d80e6eea913e6e (table default.Table_cb6a4bac7b694f33a37de2ee78c91ca0 [id=8940f18527c249a9baefc170ed82f45a]). Expected version 1 got 0
I20251211 20:52:49.374794 3169 tablet.cc:1722] T 8b793241ee3b42b1a38e9f2f05e1173b P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:49.382637 1040 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "98d575ac680d49d0990877666638f013" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:49.382934 1040 raft_consensus.cc:3060] T 98d575ac680d49d0990877666638f013 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:49.383916 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 84bb3583857848459e0e865283de623d (table default.Table_92281c5fcfee4355a79215ffedeee588 [id=6e5fe0a78e4b48f98dac028802f1ddf4]). Expected version 4 got 0
I20251211 20:52:49.386180 1168 tablet_bootstrap.cc:492] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:49.386467 1168 ts_tablet_manager.cc:1403] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.043s user 0.002s sys 0.007s
I20251211 20:52:49.387861 1040 raft_consensus.cc:2468] T 98d575ac680d49d0990877666638f013 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:49.382632 3167 tablet.cc:1722] T ee7b93b0b87541d9af6eb9e68b4cea41 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:49.388557 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 7a299ee03d0e48aba7c4f71e3e0f381a (table default.Table_6402a801d48e4a53827b1b62b6cdaba1 [id=6876565d3a9c48ce841b830cf75139ff]). Expected version 3 got 0
I20251211 20:52:49.389379 1168 raft_consensus.cc:359] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.389889 1168 raft_consensus.cc:385] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.390079 1168 raft_consensus.cc:740] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.390497 1168 consensus_queue.cc:260] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.392494 1168 ts_tablet_manager.cc:1434] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20251211 20:52:49.393931 1168 tablet_bootstrap.cc:492] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:49.394690 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 84bb3583857848459e0e865283de623d (table default.Table_92281c5fcfee4355a79215ffedeee588 [id=6e5fe0a78e4b48f98dac028802f1ddf4], current schema version=4): Not found: failed to reset TS proxy: No leader found: Tablet 84bb3583857848459e0e865283de623d consensus state has no leader
I20251211 20:52:49.396847 1160 tablet_bootstrap.cc:492] T 0aac6fb3972e4aba8950828c9e49d507 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:49.397064 1160 ts_tablet_manager.cc:1403] T 0aac6fb3972e4aba8950828c9e49d507 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.093s user 0.018s sys 0.002s
I20251211 20:52:49.398294 1160 raft_consensus.cc:359] T 0aac6fb3972e4aba8950828c9e49d507 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.398548 1160 raft_consensus.cc:385] T 0aac6fb3972e4aba8950828c9e49d507 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.398717 1160 raft_consensus.cc:740] T 0aac6fb3972e4aba8950828c9e49d507 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.398905 903 raft_consensus.cc:3060] T 3562b5effa124378beac7ccf0d27ee7e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:49.398612 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet a7afcce522494c9ca1d80e6eea913e6e (table default.Table_cb6a4bac7b694f33a37de2ee78c91ca0 [id=8940f18527c249a9baefc170ed82f45a], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet a7afcce522494c9ca1d80e6eea913e6e consensus state has no leader
I20251211 20:52:49.398124 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 5b3e8ec3faf445ac9b746e65dc4dbfe0 (table default.Table_7fe80ff5483a4b0292091119980f6639 [id=3c3462a0c09748cea0d2ffb544ae2f13]). Expected version 1 got 0
I20251211 20:52:49.399147 1160 consensus_queue.cc:260] T 0aac6fb3972e4aba8950828c9e49d507 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.404153 1160 ts_tablet_manager.cc:1434] T 0aac6fb3972e4aba8950828c9e49d507 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20251211 20:52:49.402818 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 7ffa06c4a7ee48b0b337a27aff090592 (table default.Table_647731023b044ebb8954f7fdcc3faf97 [id=4b91abd6e9e94a8a941b33e62e23c083]). Expected version 1 got 0
W20251211 20:52:49.404529 1796 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 1f4eb1a174c648c897ae2b7612e44585 (table default.Table_e70873b71ba24d7eae548e2acfd243b3 [id=87adcc552a704d56bd2fe812fb3b791f], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 1f4eb1a174c648c897ae2b7612e44585 consensus state has no leader
W20251211 20:52:49.404955 711 leader_election.cc:343] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:49.406411 1160 tablet_bootstrap.cc:492] T 4a39870c7eea4a379b284ea2e5ddf760 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:49.406317 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 81fe26e81790448fb436929c03e3c284 (table default.Table_efc22dc69ad0405ebf59fda961e23330 [id=d8a0839df7524fb7b95bfc05fd4ada25]). Expected version 1 got 0
I20251211 20:52:49.407717 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b81a4cef27164824b52695039744db15 (table default.Table_defdeab28749411590cf75b5251f604b [id=11352566d1744193a40a8bb605aa3912]). Expected version 1 got 0
W20251211 20:52:49.407423 711 leader_election.cc:343] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:49.410092 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 7023afec769c4ca3b2242a225004d482 (table default.Table_d5276e3cc46d4331b36a8878680fc55e [id=3811f8235b2f44cca283401302d613d4]). Expected version 1 got 0
W20251211 20:52:49.410395 711 leader_election.cc:343] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:49.412534 711 leader_election.cc:343] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:49.412318 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 72af55c8045b4f8f8e116a5782794d81 (table default.Table_2cff6fab088c4d939b13b504c133cc12 [id=b6c4b72063b3428593d5650a160dc66b]). Expected version 1 got 0
I20251211 20:52:49.413359 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c081396b492049b996ac4661e1318388" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:49.412830 1168 tablet_bootstrap.cc:654] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.416846 1160 tablet_bootstrap.cc:654] T 4a39870c7eea4a379b284ea2e5ddf760 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.416810 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 4cf0cde801bf44449e539d2799b895a7 (table default.Table_907c596bdddc4edca25d14cbacebb23f [id=321235280d1d4fa0bae5cc0333860f85]). Expected version 1 got 0
W20251211 20:52:49.417533 847 leader_election.cc:343] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 4f807596fa6a4e66b89566b5d5dd1080
I20251211 20:52:49.417651 847 leader_election.cc:304] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
W20251211 20:52:49.418304 847 leader_election.cc:343] T eb9da081cb9d43fd9a35c4bc51ea133e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: eb9da081cb9d43fd9a35c4bc51ea133e
I20251211 20:52:49.418390 847 leader_election.cc:304] T eb9da081cb9d43fd9a35c4bc51ea133e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:49.418148 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 7b9f569554fb4836ac2b62542c3ecf7e (table default.Table_1a73d567e8914a83974e3e8f5f58be0b [id=6ef691749df24251a7a7c5919c219754]). Expected version 1 got 0
W20251211 20:52:49.418920 847 leader_election.cc:343] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 0b267c7feaac43f6aacb7229759c7ec3
I20251211 20:52:49.419003 847 leader_election.cc:304] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:49.419319 2966 raft_consensus.cc:2749] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:52:49.419662 2973 raft_consensus.cc:2749] T eb9da081cb9d43fd9a35c4bc51ea133e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:49.419936 2973 raft_consensus.cc:2749] T 0b267c7feaac43f6aacb7229759c7ec3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:49.419780 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 3b2c6064c5eb4c0f8979c485f5d85a75 (table default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f]). Expected version 1 got 0
I20251211 20:52:49.421582 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 938a0d8a090f490b816fdf2b8f1a51b8 (table default.Table_5ca4ab25a21d456c95a0ff68c9d73330 [id=9ffcb594267a4b0eb8d792dbe46d585f]). Expected version 1 got 0
W20251211 20:52:49.421648 1796 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.3:60211 (request call id 156) took 4568 ms (client timeout 1000 ms). Trace:
I20251211 20:52:49.423103 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 59acb8474e114bf7ae7f1f043d611459 (table default.Table_c7ed8c0c309c40a3a371ffa130b47cd2 [id=ed1d0670426945aba888b818ad657225]). Expected version 1 got 0
I20251211 20:52:49.425323 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet bbe6e6220eb14e59a821805c438230c9 (table default.Table_16a12ed370b64a38b5e145c4eb024349 [id=b7429ea93123473ab5d795d5f3d5a5f8]). Expected version 1 got 0
I20251211 20:52:49.428882 2966 raft_consensus.cc:493] T 0aac6fb3972e4aba8950828c9e49d507 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251211 20:52:49.422505 1796 rpcz_store.cc:269] 1211 20:52:44.853368 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:44.853878 (+ 510us) service_pool.cc:225] Handling call
1211 20:52:49.421434 (+4567556us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:46.930555 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:46.930700 (+ 145us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:46.930708 (+ 8us) write_op.cc:435] Acquired schema lock
1211 20:52:46.930710 (+ 2us) tablet.cc:660] Decoding operations
1211 20:52:46.936924 (+ 6214us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:46.936940 (+ 16us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:46.936946 (+ 6us) tablet.cc:683] Acquiring locks for 10 operations
1211 20:52:46.937154 (+ 208us) tablet.cc:699] Row locks acquired
1211 20:52:46.937157 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:46.937204 (+ 47us) write_op.cc:270] Start()
1211 20:52:46.937239 (+ 35us) write_op.cc:276] Timestamp: P: 1765486366937185 usec, L: 0
1211 20:52:46.937243 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:46.937462 (+ 219us) log.cc:844] Serialized 3894 byte log entry
1211 20:52:46.994347 (+ 56885us) op_driver.cc:464] REPLICATION: finished
1211 20:52:47.024990 (+ 30643us) write_op.cc:301] APPLY: starting
1211 20:52:47.025010 (+ 20us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:47.025086 (+ 76us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:47.025165 (+ 79us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:47.025356 (+ 191us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:47.025399 (+ 43us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
1211 20:52:47.025407 (+ 8us) write_op.cc:312] APPLY: finished
1211 20:52:47.025543 (+ 136us) log.cc:844] Serialized 77 byte log entry
1211 20:52:47.025622 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.025658 (+ 36us) write_op.cc:454] Released schema lock
1211 20:52:47.025666 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
1211 20:52:47.031777 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:47.031912 (+ 135us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:47.031921 (+ 9us) write_op.cc:435] Acquired schema lock
1211 20:52:47.031924 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:47.032043 (+ 119us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:47.032060 (+ 17us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:47.032066 (+ 6us) tablet.cc:683] Acquiring locks for 1 operations
1211 20:52:47.032141 (+ 75us) tablet.cc:699] Row locks acquired
1211 20:52:47.032143 (+ 2us) write_op.cc:260] PREPARE: finished
1211 20:52:47.032189 (+ 46us) write_op.cc:270] Start()
1211 20:52:47.032227 (+ 38us) write_op.cc:276] Timestamp: P: 1765486367032158 usec, L: 0
1211 20:52:47.032231 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:47.032448 (+ 217us) log.cc:844] Serialized 356 byte log entry
1211 20:52:47.048293 (+ 15845us) op_driver.cc:464] REPLICATION: finished
1211 20:52:47.049885 (+ 1592us) write_op.cc:301] APPLY: starting
1211 20:52:47.049908 (+ 23us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:47.049957 (+ 49us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:47.049992 (+ 35us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:47.050058 (+ 66us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:47.050095 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1211 20:52:47.050101 (+ 6us) write_op.cc:312] APPLY: finished
1211 20:52:47.050192 (+ 91us) log.cc:844] Serialized 23 byte log entry
1211 20:52:47.050461 (+ 269us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.050486 (+ 25us) write_op.cc:454] Released schema lock
1211 20:52:47.050494 (+ 8us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:47.050652 (+ 158us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.050656 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":353,"child_traces":[["op",{"apply.queue_time_us":30544,"apply.run_cpu_time_us":762,"apply.run_wall_time_us":763,"num_ops":10,"prepare.queue_time_us":43097,"prepare.run_cpu_time_us":6338,"prepare.run_wall_time_us":56059,"raft.queue_time_us":719,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":317,"replication_time_us":57048,"thread_start_us":141973,"threads_started":4,"wal-append.queue_time_us":16546}],["op",{"apply.queue_time_us":1509,"apply.run_cpu_time_us":501,"apply.run_wall_time_us":767,"num_ops":1,"prepare.queue_time_us":119,"prepare.run_cpu_time_us":801,"prepare.run_wall_time_us":801,"raft.queue_time_us":510,"raft.run_cpu_time_us":8,"raft.run_wall_time_us":8,"replication_time_us":16027}]]}
I20251211 20:52:49.429032 2966 raft_consensus.cc:515] T 0aac6fb3972e4aba8950828c9e49d507 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.429605 2966 leader_election.cc:290] T 0aac6fb3972e4aba8950828c9e49d507 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:49.429692 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet ce1799e6604f4ed7875ddedecab6f327 (table default.Table_5db1ac98e36d45b1b01889ee275ad83d [id=a6849726434d41d29dce12f47066a685]). Expected version 1 got 0
I20251211 20:52:49.431007 1796 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_e6f3a819faad48aabfad3a235c3c1b95" } modify_external_catalogs: true
I20251211 20:52:49.432332 1796 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_e6f3a819faad48aabfad3a235c3c1b95" } modify_external_catalogs: true
I20251211 20:52:49.432276 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet bc4e7362e6d045da81bf39f4135f500e (table default.Table_328b7dd76e9e407c83b47a0f51cfcce5 [id=9e7c08f833e14a90b17a9b9f60b702de]). Expected version 1 got 0
I20251211 20:52:49.434255 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet dfd194cf00f549c8937cb6ec40987240 (table default.Table_25e38d0576684687890bae04bd81d4f4 [id=d919c6132b3c466e8a1404f981b003b8]). Expected version 2 got 0
W20251211 20:52:49.434028 711 consensus_peers.cc:597] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:49.436812 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet f26ba8f4bd3a4a1f8fa2ba6e80d543f8 (table default.Table_71e9c158dc22435a8fbb02aecca8919d [id=b966d0c0b49d4023894950c5dc06f678]). Expected version 2 got 0
I20251211 20:52:49.438709 3133 tablet.cc:1722] T 3b2c6064c5eb4c0f8979c485f5d85a75 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.438987 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 1c4b2c22bd4c456eb77af45576de466f (table default.Table_e3a99b043e4f420a93ebfee6aa87f2ec [id=3e7d6cefcb7d412191c6540e9ad77c2d]). Expected version 2 got 0
W20251211 20:52:49.439217 711 consensus_peers.cc:597] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:49.442299 1166 tablet_bootstrap.cc:654] T 4b07f110f25645799ed0c89737cd8279 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.446029 2962 ts_tablet_manager.cc:1929] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:49.446249 2962 log.cc:1199] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/5710fc2f24764f818f7e5c86c6fd985e
I20251211 20:52:49.446811 2962 ts_tablet_manager.cc:1950] T 5710fc2f24764f818f7e5c86c6fd985e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:49.448537 3142 tablet.cc:1722] T 3e6f62c2deff4c11b137fc0cba5ee234 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.449339 3169 tablet.cc:1722] T e4647105ab57454d883ebfcde7bde891 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.453100 3142 tablet.cc:1722] T c7a2610f20df4c1ca81eae2fc7dba6bb P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.453778 3154 consensus_queue.cc:1048] T 44e2d2bdecf247baaac26e06bccc94de P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:49.454618 710 leader_election.cc:304] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:49.455013 3154 raft_consensus.cc:2804] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:49.455106 3154 raft_consensus.cc:493] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:49.455185 3154 raft_consensus.cc:3060] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:49.458256 3154 raft_consensus.cc:515] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.459107 3154 leader_election.cc:290] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:49.459160 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 8b793241ee3b42b1a38e9f2f05e1173b (table default.Table_7506ffecc2b64648a9ba736e5298ddf4 [id=279f39d718ba45318f52f95e8aa13652]). Expected version 2 got 0
I20251211 20:52:49.459560 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 645b08b33676420c96e880de3d460829 (table default.Table_e02933dd91ce4fe7a40d1c3b2f9b2d4b [id=6d42f260246e420d8818b3841926302b]). Expected version 2 got 0
I20251211 20:52:49.460125 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet ab4e7a9062134e489d6f9abe6ff26fb4 (table default.Table_4cbc8841b276406cb4434629653a5ce3 [id=b958532650374d609199cab9c07bb64b]). Expected version 2 got 0
I20251211 20:52:49.460191 3154 consensus_queue.cc:1048] T 845b5ac4a5984c0cb4ab188925103dc9 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:49.460572 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 6b7dd8232d974a43b99ec9b28075dc8d (table default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]). Expected version 2 got 0
I20251211 20:52:49.460723 3166 tablet.cc:1722] T 676faec7f3d1488f99b220f8e968e9a0 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:49.461151 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet e3f711f02af843adb4f5c3c750924e33 (table default.Table_34bd5155aa3b4ef7818022ba9346c074 [id=83a61d23e80946439a14096ab3ab1bd9]). Expected version 2 got 0
I20251211 20:52:49.461719 3142 tablet.cc:1722] T 645b08b33676420c96e880de3d460829 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:49.461856 3154 consensus_queue.cc:1048] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:49.462096 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 7ca98b43863a47c091baae0a6ce65630 (table default.Table_f0a70914acc94231a4e84155c3e4d13f [id=56c070ad80624c698ebe2e01b4929b6e]). Expected version 1 got 0
I20251211 20:52:49.464005 710 leader_election.cc:304] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:49.449684 3167 tablet.cc:1722] T a3b1aadf17ea4bf0a896d23cd543fca7 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.464399 3154 raft_consensus.cc:2804] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:49.465255 3166 tablet.cc:1722] T 6b7dd8232d974a43b99ec9b28075dc8d P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:49.465358 3154 raft_consensus.cc:493] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:49.465452 3154 raft_consensus.cc:3060] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:49.465282 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 9f8aa28c1c9b4fc58dfc128952c180ba (table default.Table_bf7d402a36be44cfb6b79041906d26a1 [id=0327e686e7ed4475a3e8cda8721a3a3a]). Expected version 1 got 0
I20251211 20:52:49.466225 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 969a8587019c4202b6e705d126f83d44 (table default.Table_6460313002684bba93b30ad8631f0f49 [id=22347b15537d451d9927a17e8e5a202a]). Expected version 1 got 0
I20251211 20:52:49.466488 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 676faec7f3d1488f99b220f8e968e9a0 (table default.Table_ab826a3eec8e4c4aaa8d4021d87dd382 [id=a2bbfccfe3974c378521d5ef2333df86]). Expected version 2 got 0
I20251211 20:52:49.466784 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet fb29f983c4524a49bdc5c0c1144ad708 (table default.Table_7809ac85fdfe4c1d9c58afb54edecb27 [id=c4b0bfff99f64888beb70d9154d2f156]). Expected version 1 got 0
I20251211 20:52:49.466992 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet d6923cfd826d495291d9ac8e0b3b44c6 (table default.Table_68ccb2052d824ccf92f7162d2961fcb1 [id=9ac9591d4f04482d926710fdf6e797c6]). Expected version 1 got 0
I20251211 20:52:49.467417 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet a3b1aadf17ea4bf0a896d23cd543fca7 (table default.Table_d18401707c88493abdf90d2c713432ae [id=a07662f8dc3f4178be5dfcc1cdc85837]). Expected version 1 got 0
I20251211 20:52:49.467762 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet bc6a1c84501d4e179cd2828b50fddc23 (table default.Table_8bb5b228d60e4d29a41d9ba40c33d45d [id=6ef058fada9941629e0ea4d1906abdac]). Expected version 1 got 0
I20251211 20:52:49.468111 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 16d2ff8925e342729a71c93db378d031 (table default.Table_34e6ed4f13da437ea50d349b92e83bd6 [id=4a74b9a60cbb4198b55cee630b2f4bef]). Expected version 2 got 0
I20251211 20:52:49.468245 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet f94b656e3c1844509fdae36e33b263e8 (table default.Table_ed03ad475bed4804a65abe20efef0625 [id=e23f4305669a474aafa18928f130de3a]). Expected version 2 got 0
I20251211 20:52:49.468386 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 4c0455a5b4f44412803475d1c4852b48 (table default.Table_e00092673b0e4f09b8d7eec63e1ee034 [id=18497e06d7254441a4655ce1cdc5a9bf]). Expected version 1 got 0
I20251211 20:52:49.468575 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet f2d6c9781b4d4c37bc08703347963d5d (table default.Table_2aa7ff9186a04249abf8cd971cc0cb17 [id=b50ab5b032ba4c98938324460512468f]). Expected version 1 got 0
I20251211 20:52:49.474233 3154 raft_consensus.cc:515] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.468843 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet aab26df748724fe5b2bb7fad9e3bfc0c (table default.Table_22bed4b4e9e24a90818b19c6f5c91e1f [id=df6255d66f334ef2b4406f4b845cf980]). Expected version 1 got 0
W20251211 20:52:49.476635 711 consensus_peers.cc:597] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:49.476662 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 497c10742f01476db885bd70c4d2c491 (table default.Table_18b21232dfa74378b27defe672bb2574 [id=602d7eee703948d99a0edcb7cac0651b]). Expected version 1 got 0
I20251211 20:52:49.476857 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 3003ef71a3554c71a3b24cdfb9c293bc (table default.Table_92efd469dd324f7d886e2cbc90ece61c [id=6e7fb3271e0c48c0bb9d56fe4fccee3a]). Expected version 1 got 0
I20251211 20:52:49.477044 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b2974f16034c44cf85ff85b5095cf6a0 (table default.Table_031de0f4cc7744bc9e320261f8b46696 [id=be36dabaa6d44325b4855b9f13bda847]). Expected version 1 got 0
I20251211 20:52:49.477344 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 5e47af616e1b4d2db3780c1d67cdcb47 (table default.Table_296bdf30a11541c389afe3df0b0f667a [id=052edcbde60c41449d468c8a449fc3f1]). Expected version 1 got 0
I20251211 20:52:49.478003 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 03184dcc56dd4e92adc7d0ad22aa8d8b (table default.Table_68261841754847ad8d22e86c29886144 [id=697f23a617f64762bf406b386cde7473]). Expected version 1 got 0
I20251211 20:52:49.478233 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 94658fcf512a48d0a992a7e2e0f7c176 (table default.Table_c9d86e82aebb4ef0bc83789720352e38 [id=97377a8ef56c4cdebe95d15955207404]). Expected version 1 got 0
I20251211 20:52:49.478570 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b85af79c2574443998e847bc19c53b69 (table default.Table_fa3f2f115cd043a9aa6be2b070d320cb [id=15fad5e1cf8d4ea08dae8275a29a11dd]). Expected version 1 got 0
W20251211 20:52:49.478646 711 consensus_peers.cc:597] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:49.478962 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet d37a7d0d662e45079b0cfc02765d2e01 (table default.Table_014b6db9f3294658b965c701384b98bb [id=8a12299efe714617a970fc05c3bb6232]). Expected version 2 got 0
I20251211 20:52:49.478950 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "962b80ae747b4470abd23643e711fad9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:49.479296 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "32f8d104f6a948d1a97a83769ea82449" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:49.479354 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet aa50ddb2789b4be3bfa9aedef9287109 (table default.Table_44f971096d1e41f480e594985ffd5c17 [id=d4e1349374a346a5b8be180f9bdc329a]). Expected version 1 got 0
I20251211 20:52:49.479666 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 13269a20421240d1a98215129443248b (table default.Table_d887c953026046dc883ede774a58fd8e [id=1d8a79f716a0447cb0393f4759178598]). Expected version 1 got 0
I20251211 20:52:49.479847 3110 tablet.cc:1722] T 3b2c6064c5eb4c0f8979c485f5d85a75 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.479888 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 51d12723f5194afb97d2c05fcbaf3776 (table default.Table_823505200e3142dbbe8cd56ac78daf55 [id=ec0bb10933fa4529bef3580bee6409e7]). Expected version 2 got 0
W20251211 20:52:49.480105 711 leader_election.cc:343] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:49.480286 711 leader_election.cc:343] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:49.480289 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 46a2a070cf6a42fe8bc54a42eb802fbc (table default.Table_4e1f03b9d87b4fe59dbcb3a947ee299d [id=8e66ea58499b40e59294a135324aa527]). Expected version 1 got 0
I20251211 20:52:49.480831 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 5773c13034714a008c8d7dbe777b4828 (table default.Table_66eb1b021eed464f82f9cd223c88297b [id=94ff09564cd649c7a2d0e94b0f3c7b3a]). Expected version 1 got 0
I20251211 20:52:49.481113 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 18f125dd9efc401181f0644dad7aeffe (table default.Table_fa6592c2e2624d81ad21abb13ed36045 [id=9b48fc2b39db4fd5a4e5da19e34427dc]). Expected version 1 got 0
I20251211 20:52:49.481283 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 8dc4e730951144d78b77280f6cce6d0e (table default.Table_890c9027415a44ccaf95c0adede51992 [id=7bc8a2b91fe54b91b6594b371fea7bd3]). Expected version 2 got 0
I20251211 20:52:49.481786 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 78f77d8916d84edabbe95ce17d8df2c1 (table default.Table_e2af0e2fbd3d46d595136c7a7bf24e4c [id=ddc12ae6d8ed4587b5ef4da3ef81a055]). Expected version 1 got 0
I20251211 20:52:49.481976 1797 catalog_manager.cc:5654] T d391d4e6a101462d8d1220f1e17d0daa P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:49.476816 3154 leader_election.cc:290] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:49.484162 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet ff4d5ec1f87247e896f9c099c532265a (table default.Table_3ac4746e17264f5a93890f3600505781 [id=475e0a1ec0344d10addbcfa42fe05b68]). Expected version 1 got 0
I20251211 20:52:49.484382 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b6cc819332e1445c96ce3ffac4861454 (table default.Table_db4c0f706e0643b39d39ffa84352826d [id=4d459c42bf584068ac3851227d80331a]). Expected version 1 got 0
I20251211 20:52:49.484724 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 3dfea28c44d84c68850db65a30d52ee6 (table default.Table_c504f8f3796f43b697488bd604268543 [id=2af76b9d32094a4ca656060a4478e08a]). Expected version 1 got 0
I20251211 20:52:49.485109 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 4c7b74f370b042ee8575f821c07dc085 (table default.Table_731826f26f9940b8afe425d694b642d9 [id=17dd9cea9ac844128ff9f82adb3f919f]). Expected version 1 got 0
I20251211 20:52:49.485291 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet f00128ba84c245d9a0e35f26bc842b3e (table default.Table_6f8c133c7a364b93af2c83d8911a0ac3 [id=6a21c38ee7224a02b937df088494f5d8]). Expected version 1 got 0
I20251211 20:52:49.485532 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 6212b3b4425543168837946a54d60834 (table default.Table_8c30c4151d484db0bb9e2b9ca262248d [id=2d489cd7d73e403d84bf10960e6a2dfb]). Expected version 1 got 0
I20251211 20:52:49.485754 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 33fd09ea06ee47908b85911e4ee79708 (table default.Table_3c3cd48c8fe5411e96cdf811d492925d [id=30620a01c27f456bac05fa7c02a2cb4d]). Expected version 2 got 0
I20251211 20:52:49.486073 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 3c865c99e9314f5f91658b9126da4d39 (table default.Table_0d7df3fba38849a49b65d8290812b95c [id=7f966d5cdad3423bb770414802a3cf7b]). Expected version 1 got 0
W20251211 20:52:49.486665 847 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 67741.8us
I20251211 20:52:49.486932 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 8675e2a18f0f4a0e95a87005cf82d8b4 (table default.Table_bd9633c55bd242c18aac3a7f2c18a329 [id=767f50bee5464316a18f73e64bab817a]). Expected version 2 got 0
I20251211 20:52:49.487571 2966 raft_consensus.cc:493] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:49.487747 2966 raft_consensus.cc:515] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.487900 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b115d737d7fc483d9f5a1f040b511bae (table default.Table_6d94053661f142b79292e19752df02df [id=c5be59a2c3934b3789641ceeb2d4d6e2]). Expected version 2 got 0
I20251211 20:52:49.488058 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet da81bf8854a24954ba4548185812295d (table default.Table_18262e22c26d4d66ba4c6134fbde0368 [id=da8cf2b99b42409091484a9513623d6f]). Expected version 1 got 0
I20251211 20:52:49.488278 2966 leader_election.cc:290] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:49.488391 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 3e6f62c2deff4c11b137fc0cba5ee234 (table default.Table_51db250005ed4b3d947f32acd1d62563 [id=fdfaa0b24d2f4736aee695ed228212cf]). Expected version 1 got 0
I20251211 20:52:49.489372 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet e4647105ab57454d883ebfcde7bde891 (table default.Table_c817dec9a7674f699b3366fc302ead57 [id=d555d1fc6aef4a21b4d3f2804264d030]). Expected version 1 got 0
I20251211 20:52:49.489828 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet ee7b93b0b87541d9af6eb9e68b4cea41 (table default.Table_b6e6aea68c88426fb01353146c06b6ac [id=a4f984b4263a4bc38f6248628f643ae3]). Expected version 3 got 0
I20251211 20:52:49.490052 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet df457783b2534b3ea23bb1fd529ce268 (table default.Table_6e9f3ca164384cc8ac4501deb3f0c6aa [id=342c2469d6e74e3e9217236e3aa863d6]). Expected version 1 got 0
I20251211 20:52:49.491144 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 5453f68789fd4e89b69e98860050865b (table default.Table_b7bf8c5ad5304175a1f2f6a03887f725 [id=50d88b663a4d4891b113aee9c5906699]). Expected version 1 got 0
I20251211 20:52:49.491395 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 46cb1d16a0e94b3a833f32c635213fd2 (table default.Table_3482fbfc5b144a7198a7d01aa003cb6b [id=1bdaf5ce0f6b4b99a038f9f9efb828f6]). Expected version 1 got 0
I20251211 20:52:49.491765 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0aac6fb3972e4aba8950828c9e49d507" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:49.492084 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 834b84a557b04e78b5d2c8c84520a644 (table default.Table_909bbe6da1a648d68199fb27424e4342 [id=cc99459a5039424dbc192ff748b6708b]). Expected version 2 got 0
I20251211 20:52:49.492331 1797 catalog_manager.cc:5654] T 92efda2bd0c84d07a0af6b537c0cd882 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:49.492941 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet e89d4d76bf64453892a4d0b55fcc3454 (table default.Table_fba99384e9f14160b3e3504f8d1d0b07 [id=01bba75f854d4c338d6c09a446dd7499]). Expected version 1 got 0
I20251211 20:52:49.493343 1797 catalog_manager.cc:5654] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:49.493705 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 7be44df39a8f4d6da793479d425fa684 (table default.Table_f6a10556ff104db5bd5ecc9fc380e228 [id=f75813add54e46189ff1bbed82d83ba7]). Expected version 1 got 0
I20251211 20:52:49.494051 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet fd43a27db69044b68826cda87e16a086 (table default.Table_a8ebfd7d7a7f4f6cae8b6b02b99b80b0 [id=f1d32002c612403a82bdff5baf209b9e]). Expected version 1 got 0
I20251211 20:52:49.494819 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 4fe5c534b572491c911aae01eaa79bae (table default.Table_c1f9dca5c2144d50b67681dcc6800e77 [id=14fa8debfe5443ce81587f02dfa15847]). Expected version 2 got 0
I20251211 20:52:49.495023 1168 tablet_bootstrap.cc:492] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:49.495237 1168 ts_tablet_manager.cc:1403] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.102s user 0.006s sys 0.002s
I20251211 20:52:49.495827 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 171d40a8a9124f9bb1b38797fef8e5d5 (table default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5]). Expected version 2 got 0
I20251211 20:52:49.496027 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet c081396b492049b996ac4661e1318388 (table default.Table_00ae5c111de94ba0a2680ad22c726e16 [id=95c9ed1ec1834e2dbfd3198887b75369]). Expected version 1 got 0
I20251211 20:52:49.496366 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 2e800d8cc5ea4535a065d7d9243326b4 (table default.Table_083c478bedec45a5bf5203e4ed333860 [id=8baf0ea87d734d58896adbd0d6ef99c2]). Expected version 1 got 0
I20251211 20:52:49.496572 1797 catalog_manager.cc:5654] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } health_report { overall_health: UNKNOWN } } }
I20251211 20:52:49.496975 1797 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet d43210ec61af423ebde624397dea9f2e (table default.Table_af0df4d1ee144455a6041e5a6a0b9b80 [id=7c5b2e33fa1c4b12b3941577360908a8]). Expected version 3 got 0
I20251211 20:52:49.521672 3133 tablet.cc:1722] T 5453f68789fd4e89b69e98860050865b P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.521780 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4d2b7e2a0abe4592bc806d5ef3f894d8" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:49.522395 710 leader_election.cc:304] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:49.522686 3054 raft_consensus.cc:2804] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:49.522820 3054 raft_consensus.cc:697] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:49.523066 3054 consensus_queue.cc:237] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.524161 710 leader_election.cc:304] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:49.525291 2429 raft_consensus.cc:2804] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:49.525384 710 leader_election.cc:304] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:49.525447 2429 raft_consensus.cc:697] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:49.525705 2429 consensus_queue.cc:237] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:49.526078 1784 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 7ffa06c4a7ee48b0b337a27aff090592 (table default.Table_647731023b044ebb8954f7fdcc3faf97 [id=4b91abd6e9e94a8a941b33e62e23c083], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 7ffa06c4a7ee48b0b337a27aff090592 consensus state has no leader
I20251211 20:52:49.525724 3051 raft_consensus.cc:2804] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:49.530766 3029 tablet.cc:1722] T 46cb1d16a0e94b3a833f32c635213fd2 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.531203 3051 raft_consensus.cc:697] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:49.540263 710 leader_election.cc:304] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:49.496312 1168 raft_consensus.cc:359] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.547194 3000 raft_consensus.cc:2804] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:49.548700 3000 raft_consensus.cc:697] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:49.548620 1168 raft_consensus.cc:385] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.549677 3051 consensus_queue.cc:237] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:49.550616 711 consensus_peers.cc:597] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:49.551019 3000 consensus_queue.cc:237] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.551952 1168 raft_consensus.cc:740] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
W20251211 20:52:49.554620 711 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 57659.2us
I20251211 20:52:49.557881 1160 tablet_bootstrap.cc:492] T 4a39870c7eea4a379b284ea2e5ddf760 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:49.558138 1160 ts_tablet_manager.cc:1403] T 4a39870c7eea4a379b284ea2e5ddf760 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.152s user 0.000s sys 0.006s
I20251211 20:52:49.559419 1160 raft_consensus.cc:359] T 4a39870c7eea4a379b284ea2e5ddf760 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.559940 1160 raft_consensus.cc:385] T 4a39870c7eea4a379b284ea2e5ddf760 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.560070 1160 raft_consensus.cc:740] T 4a39870c7eea4a379b284ea2e5ddf760 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.560366 1160 consensus_queue.cc:260] T 4a39870c7eea4a379b284ea2e5ddf760 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.561007 1160 ts_tablet_manager.cc:1434] T 4a39870c7eea4a379b284ea2e5ddf760 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:49.564440 1168 consensus_queue.cc:260] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.566797 1168 ts_tablet_manager.cc:1434] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.071s user 0.002s sys 0.000s
I20251211 20:52:49.567377 3181 tablet.cc:1722] T b85af79c2574443998e847bc19c53b69 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.469863 2962 tablet_replica.cc:333] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:49.570267 2962 raft_consensus.cc:2243] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:49.570446 2962 raft_consensus.cc:2272] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:49.570986 2962 ts_tablet_manager.cc:1916] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:49.571307 1168 tablet_bootstrap.cc:492] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:49.482499 1166 tablet_bootstrap.cc:492] T 4b07f110f25645799ed0c89737cd8279 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:49.492053 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet a7979798e92e45eb969197c8c7ac2580 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.579772 1166 ts_tablet_manager.cc:1403] T 4b07f110f25645799ed0c89737cd8279 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.317s user 0.007s sys 0.005s
I20251211 20:52:49.579834 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 22273287aa9245a2a66131fed2c8861c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.580025 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet bbe3529c301a4a65b58b634a443e1fb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.580144 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 116b21a182474ca191c4a89e9474677d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.580248 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 683d0f2068fb4a6ebc9c169ad7b455ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.580351 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet f25bde14792248b188660036f07ae902 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.580447 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet c91ff29847d54e55b19321652a857e9f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.580538 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 993616e226274c229a5924fbe69911af with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.580638 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet d3b383ace72748d4bf21678894815c4f with delete_type TABLET_DATA_DELETED (replaced by tablet 745ce63314834c8abdecaa367133a068) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.580613 1166 raft_consensus.cc:359] T 4b07f110f25645799ed0c89737cd8279 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.580737 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet b61c4f0cb15940a988c5acb6879d265a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.580775 1166 raft_consensus.cc:385] T 4b07f110f25645799ed0c89737cd8279 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.580816 1166 raft_consensus.cc:740] T 4b07f110f25645799ed0c89737cd8279 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.580837 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 062f980b65504d3baf99ae400e521775 with delete_type TABLET_DATA_DELETED (replaced by tablet 33fd09ea06ee47908b85911e4ee79708) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.580947 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet c549c2bfebd64b87a4fc8ce8210e14f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.581030 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 12aaf618d6474f599458026308d7afdd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.580986 1166 consensus_queue.cc:260] T 4b07f110f25645799ed0c89737cd8279 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.581113 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 10746162da4640e6a2a891467690b770 with delete_type TABLET_DATA_DELETED (replaced by tablet 8dc4e730951144d78b77280f6cce6d0e) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.581202 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 6663f13691a24fb991b48f543d9de76b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.581305 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet a56f1a7bd5464d289bfd018c826ee574 with delete_type TABLET_DATA_DELETED (replaced by tablet 284e34970f634abcb2288c864f73e963) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.581395 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 13ceabb190494e718a52892faaa53986 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.581413 1166 ts_tablet_manager.cc:1434] T 4b07f110f25645799ed0c89737cd8279 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251211 20:52:49.581485 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet fbc2d4aea084492c8270ad1d87096e58 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.581578 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet cc43ee61e78249bfa00b11f3a1fa21a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.581666 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 83067b139f504bc191a515feed0ec771 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.581760 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet fa79b67b9eee45e2b41c4a90967606cc with delete_type TABLET_DATA_DELETED (replaced by tablet 83067b139f504bc191a515feed0ec771) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.581846 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet e8db5a11a69e4554bc689c38ab57b28a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.581970 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet bd2bf1113dfe4b67b263b56f3d3db07d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.582053 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 48820386e41b47129a13bdd5768f021b with delete_type TABLET_DATA_DELETED (replaced by tablet 7a299ee03d0e48aba7c4f71e3e0f381a) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.582127 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 6cfa15533fad4aec84f8d9038227faf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.582217 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 4585e05d06ef42cfb06c4ceb7524fd71 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.582305 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet e709051411354522bdb327c34dbf533a with delete_type TABLET_DATA_DELETED (replaced by tablet dfeb47dfaa6445309e28715b1b8db2ae) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.582404 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 18b977f463d5420186c007b38110e07e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.582513 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 5ddf7c1972564ef88e3c0a0bd1b430e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.582619 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 06ba98a726424dd89450053c5c5793f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.582727 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet ba120d28abc54553ae15532de90db719 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.582836 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 1578887b2f5748cd83a6fbdec2f3fc99 with delete_type TABLET_DATA_DELETED (replaced by tablet bc4e7362e6d045da81bf39f4135f500e) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.582949 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 87c4ec07e82d4bf79b4eeefe732674ff with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.583040 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f7b1bc3dd1f4496b3ac6e71652cf012 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.583132 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 745ce63314834c8abdecaa367133a068 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.583240 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 0add14ccd2af42aeb02fad7a28ed4520 with delete_type TABLET_DATA_DELETED (replaced by tablet 84cca3ee8fc446fcaf9971171235cf55) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.583352 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet f22e4d34cde24b37a025c1b5a1046b1b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.583457 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 0aac6fb3972e4aba8950828c9e49d507 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.492571 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 29463a8f71f943a0927942235af3c2da with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.492846 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet 4d2b7e2a0abe4592bc806d5ef3f894d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.493265 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 845b5ac4a5984c0cb4ab188925103dc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.493587 1016 tablet_service.cc:1552] Processing DeleteTablet for tablet 79cc81e91e1846038621af7a0a9f7a2e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.493747 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 1e13965b6361488eaeb7dd13b467ca65 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.583701 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ef524747034491c99f4f9767acd3c9d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.585042 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 0f8808bfd68841e4ae30818018040cf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.585131 1010 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c5c937cc7db4ed18fe03f4a8a99a66a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.585199 1013 tablet_service.cc:1552] Processing DeleteTablet for tablet 7d4e63f54d8443ae8322d4659a165e76 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.584146 1024 tablet_service.cc:1552] Processing DeleteTablet for tablet 449921ea74f04a9c88eb9e094e8226c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.493850 1020 tablet_service.cc:1552] Processing DeleteTablet for tablet c112714331bf4bd7a1ddcca17f338d4f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.493877 1021 tablet_service.cc:1552] Processing DeleteTablet for tablet de508d79f30f489cb745401f53115165 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.493969 1019 tablet_service.cc:1552] Processing DeleteTablet for tablet 84b2a67c836844ceb3628cc3b7d8ba10 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.494122 1015 tablet_service.cc:1552] Processing DeleteTablet for tablet 79d92736bb1c46d2bfa5678b2d396ec2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.583571 1022 tablet_service.cc:1552] Processing DeleteTablet for tablet 9b99860ee6be45a0b358e686332d3137 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.528137 1018 tablet_service.cc:1552] Processing DeleteTablet for tablet 5f2c14c7a972414e8328e0b388bde0a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.528275 1023 tablet_service.cc:1552] Processing DeleteTablet for tablet f2ae084982f44fdf8bfecdd2bc4c01b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.528416 1012 tablet_service.cc:1552] Processing DeleteTablet for tablet 98d575ac680d49d0990877666638f013 with delete_type TABLET_DATA_DELETED (replaced by tablet 834b84a557b04e78b5d2c8c84520a644) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.528465 1026 tablet_service.cc:1552] Processing DeleteTablet for tablet 75dd170c5828409fa69bd68fdf28a72b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.584929 1016 tablet_service.cc:1552] Processing DeleteTablet for tablet f91487dd7e0f4c8b8483c2b83945da3b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.528519 1014 tablet_service.cc:1552] Processing DeleteTablet for tablet be876471f8bf453aa4467c91891cbbde with delete_type TABLET_DATA_DELETED (replaced by tablet 7d4e63f54d8443ae8322d4659a165e76) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.528683 1028 tablet_service.cc:1552] Processing DeleteTablet for tablet 3f1840e4b5214ddb878935a79b214eae with delete_type TABLET_DATA_DELETED (replaced by tablet 4fc56df5ccb94198b5b0e516eeae6941) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.528908 1011 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b07f110f25645799ed0c89737cd8279 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.529013 1025 tablet_service.cc:1552] Processing DeleteTablet for tablet 757ec913dff840f992a41a38b05f8d41 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.529054 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 66cdc6f297b24cfba3a3a0ae9f45d12d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:36220
W20251211 20:52:49.565353 982 leader_election.cc:343] T c081396b492049b996ac4661e1318388 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:49.584728 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet bb9fccb6981e446a939a28d7fdd38e25 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:49.586341 982 leader_election.cc:304] T c081396b492049b996ac4661e1318388 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:49.586580 2981 raft_consensus.cc:2749] T c081396b492049b996ac4661e1318388 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:49.587385 1031 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "962b80ae747b4470abd23643e711fad9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:49.587882 1031 raft_consensus.cc:3060] T 962b80ae747b4470abd23643e711fad9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:49.588181 2962 ts_tablet_manager.cc:1929] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:49.588279 2962 log.cc:1199] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/e1aeae8c18b6499e9dc1accabdc3f633
I20251211 20:52:49.588526 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 5710fc2f24764f818f7e5c86c6fd985e (table default.Table_491fd0036d9e4f9fa410f3517c2c8286 [id=0e93a84ec95341889e108170806b2525]) successfully deleted
W20251211 20:52:49.588872 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 80c566822fa94ada983aa7266df4bf9d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 80c566822fa94ada983aa7266df4bf9d already in progress: creating tablet
I20251211 20:52:49.589381 1039 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0aac6fb3972e4aba8950828c9e49d507" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:49.589787 1049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4d2b7e2a0abe4592bc806d5ef3f894d8" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:49.589996 2962 ts_tablet_manager.cc:1950] T e1aeae8c18b6499e9dc1accabdc3f633 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:49.590449 1031 raft_consensus.cc:2468] T 962b80ae747b4470abd23643e711fad9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:49.591384 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "32f8d104f6a948d1a97a83769ea82449" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:49.591517 1033 raft_consensus.cc:3060] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:49.594128 1033 raft_consensus.cc:2468] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:49.595950 1166 tablet_bootstrap.cc:492] T c112714331bf4bd7a1ddcca17f338d4f P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:49.597496 1166 tablet_bootstrap.cc:654] T c112714331bf4bd7a1ddcca17f338d4f P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.601284 1168 tablet_bootstrap.cc:654] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.602461 3029 tablet.cc:1722] T f2d6c9781b4d4c37bc08703347963d5d P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.603973 847 leader_election.cc:304] T 0aac6fb3972e4aba8950828c9e49d507 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:49.607839 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet e1aeae8c18b6499e9dc1accabdc3f633 (table default.Table_4a209d1f3ab4458bb6197e926297e60a [id=d67eb639ab9b456195ce69ac993349ae]) successfully deleted
W20251211 20:52:49.608165 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 042f8bf13a7a4712b5bdc04671cd9938 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 042f8bf13a7a4712b5bdc04671cd9938 already in progress: creating tablet
W20251211 20:52:49.608338 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 84ed434868aa437a83455b1605deb26c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84ed434868aa437a83455b1605deb26c already in progress: creating tablet
W20251211 20:52:49.608500 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 4810f8d8b53a45eb939bf3c8bf1b4e77 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4810f8d8b53a45eb939bf3c8bf1b4e77 already in progress: creating tablet
W20251211 20:52:49.608649 1780 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet c7bd218b3ff340b2a589c9517b49a649 because the tablet was not found. No further retry: Not found: Tablet not found: c7bd218b3ff340b2a589c9517b49a649
W20251211 20:52:49.608821 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 07e4336171714da8adc21977ce33ce50 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07e4336171714da8adc21977ce33ce50 already in progress: creating tablet
W20251211 20:52:49.608991 1780 catalog_manager.cc:4952] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet eb9da081cb9d43fd9a35c4bc51ea133e because the tablet was not found. No further retry: Not found: Tablet not found: eb9da081cb9d43fd9a35c4bc51ea133e
I20251211 20:52:49.606108 2966 raft_consensus.cc:493] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6f8d784fd1cd492bbe3809403ec34919)
I20251211 20:52:49.609541 2966 raft_consensus.cc:515] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.610159 2966 leader_election.cc:290] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:49.610852 2966 raft_consensus.cc:2749] T 0aac6fb3972e4aba8950828c9e49d507 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:49.611294 847 leader_election.cc:304] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:49.611974 2966 raft_consensus.cc:2749] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:49.613802 847 leader_election.cc:343] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 14e2650f8ec1482ea9511b989fdc963c
I20251211 20:52:49.619740 2962 tablet_replica.cc:333] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:49.626849 2981 raft_consensus.cc:493] T 3f1840e4b5214ddb878935a79b214eae P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:49.627008 2981 raft_consensus.cc:515] T 3f1840e4b5214ddb878935a79b214eae P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.627683 2981 leader_election.cc:290] T 3f1840e4b5214ddb878935a79b214eae P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
W20251211 20:52:49.628178 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 80851us
I20251211 20:52:49.628346 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3f1840e4b5214ddb878935a79b214eae" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:49.628849 983 leader_election.cc:343] T 3f1840e4b5214ddb878935a79b214eae P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:49.628851 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3f1840e4b5214ddb878935a79b214eae" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:49.629938 1168 tablet_bootstrap.cc:492] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:49.630084 1168 ts_tablet_manager.cc:1403] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.063s user 0.006s sys 0.000s
I20251211 20:52:49.630852 1168 raft_consensus.cc:359] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.631024 1168 raft_consensus.cc:385] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.631086 1168 raft_consensus.cc:740] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.631374 1168 consensus_queue.cc:260] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.631872 1168 ts_tablet_manager.cc:1434] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:49.632164 1168 tablet_bootstrap.cc:492] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:49.631996 982 leader_election.cc:343] T 3f1840e4b5214ddb878935a79b214eae P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:49.632234 982 leader_election.cc:304] T 3f1840e4b5214ddb878935a79b214eae P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:49.632467 2981 raft_consensus.cc:2749] T 3f1840e4b5214ddb878935a79b214eae P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:49.634204 1168 tablet_bootstrap.cc:654] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.639974 2959 consensus_queue.cc:1048] T 497c10742f01476db885bd70c4d2c491 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:49.641719 2959 consensus_queue.cc:1048] T 84b2a67c836844ceb3628cc3b7d8ba10 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:49.649452 2959 consensus_queue.cc:1048] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:49.650249 2959 consensus_queue.cc:1048] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251211 20:52:49.651386 710 consensus_peers.cc:597] T 5710fc2f24764f818f7e5c86c6fd985e P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5710fc2f24764f818f7e5c86c6fd985e. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:49.651674 710 leader_election.cc:304] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:49.651973 2959 raft_consensus.cc:2804] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:49.652061 2959 raft_consensus.cc:697] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:49.652261 2959 consensus_queue.cc:237] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.652560 2962 raft_consensus.cc:2243] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:49.652982 2962 pending_rounds.cc:70] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7: Trying to abort 1 pending ops.
I20251211 20:52:49.653028 2962 pending_rounds.cc:77] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432164312821760 op_type: NO_OP noop_request { }
I20251211 20:52:49.653144 2962 raft_consensus.cc:2889] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:49.653240 2962 raft_consensus.cc:2272] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:49.659606 1166 tablet_bootstrap.cc:492] T c112714331bf4bd7a1ddcca17f338d4f P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:49.659871 1166 ts_tablet_manager.cc:1403] T c112714331bf4bd7a1ddcca17f338d4f P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.064s user 0.006s sys 0.001s
I20251211 20:52:49.660984 1166 raft_consensus.cc:359] T c112714331bf4bd7a1ddcca17f338d4f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.661227 1166 raft_consensus.cc:385] T c112714331bf4bd7a1ddcca17f338d4f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.661309 1166 raft_consensus.cc:740] T c112714331bf4bd7a1ddcca17f338d4f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.661571 1166 consensus_queue.cc:260] T c112714331bf4bd7a1ddcca17f338d4f P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.662781 1166 ts_tablet_manager.cc:1434] T c112714331bf4bd7a1ddcca17f338d4f P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:49.663168 1166 tablet_bootstrap.cc:492] T 6a206abd6ea2420089cb07714e0d19c0 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:49.664925 710 leader_election.cc:304] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:49.665693 1166 tablet_bootstrap.cc:654] T 6a206abd6ea2420089cb07714e0d19c0 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.665683 2970 raft_consensus.cc:2804] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:49.665799 2970 raft_consensus.cc:697] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:49.665864 3174 raft_consensus.cc:493] T fa79b67b9eee45e2b41c4a90967606cc P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:49.666000 3174 raft_consensus.cc:515] T fa79b67b9eee45e2b41c4a90967606cc P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.666549 2970 consensus_queue.cc:237] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.666678 3174 leader_election.cc:290] T fa79b67b9eee45e2b41c4a90967606cc P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:49.672881 1160 tablet_bootstrap.cc:492] T ee7b93b0b87541d9af6eb9e68b4cea41 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:49.674731 1160 tablet_bootstrap.cc:654] T ee7b93b0b87541d9af6eb9e68b4cea41 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.689050 1168 tablet_bootstrap.cc:492] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:49.689311 1168 ts_tablet_manager.cc:1403] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.057s user 0.015s sys 0.009s
I20251211 20:52:49.690160 1168 raft_consensus.cc:359] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.690358 1168 raft_consensus.cc:385] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.690414 1168 raft_consensus.cc:740] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.690640 1168 consensus_queue.cc:260] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.691159 1168 ts_tablet_manager.cc:1434] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:49.692286 1168 tablet_bootstrap.cc:492] T 903c7f4b312d4677bb35be1aaa46ad6b P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:49.694440 1168 tablet_bootstrap.cc:654] T 903c7f4b312d4677bb35be1aaa46ad6b P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.667490 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fa79b67b9eee45e2b41c4a90967606cc" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:49.680868 2962 ts_tablet_manager.cc:1916] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:49.687044 3166 tablet.cc:1722] T 46cb1d16a0e94b3a833f32c635213fd2 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.691787 3142 tablet.cc:1722] T b85af79c2574443998e847bc19c53b69 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.705657 1168 tablet_bootstrap.cc:492] T 903c7f4b312d4677bb35be1aaa46ad6b P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:49.705809 1168 ts_tablet_manager.cc:1403] T 903c7f4b312d4677bb35be1aaa46ad6b P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.005s
I20251211 20:52:49.706542 1168 raft_consensus.cc:359] T 903c7f4b312d4677bb35be1aaa46ad6b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.706703 1168 raft_consensus.cc:385] T 903c7f4b312d4677bb35be1aaa46ad6b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.706753 1168 raft_consensus.cc:740] T 903c7f4b312d4677bb35be1aaa46ad6b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.706950 1168 consensus_queue.cc:260] T 903c7f4b312d4677bb35be1aaa46ad6b P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.708276 1168 ts_tablet_manager.cc:1434] T 903c7f4b312d4677bb35be1aaa46ad6b P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:49.709415 1168 tablet_bootstrap.cc:492] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:49.711066 1168 tablet_bootstrap.cc:654] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.733623 2959 raft_consensus.cc:493] T 7a0a20f95416425489af94408ac1e97c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:49.734838 3142 tablet.cc:1722] T 3b2c6064c5eb4c0f8979c485f5d85a75 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.737926 3174 raft_consensus.cc:493] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:49.738173 3174 raft_consensus.cc:515] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.738895 3174 leader_election.cc:290] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:49.740056 847 leader_election.cc:343] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 2d849c62f7434099a16f62c2a2fc5e85
I20251211 20:52:49.740479 2959 raft_consensus.cc:515] T 7a0a20f95416425489af94408ac1e97c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.741447 1166 tablet_bootstrap.cc:492] T 6a206abd6ea2420089cb07714e0d19c0 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:49.741607 1166 ts_tablet_manager.cc:1403] T 6a206abd6ea2420089cb07714e0d19c0 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.079s user 0.003s sys 0.000s
I20251211 20:52:49.742343 1166 raft_consensus.cc:359] T 6a206abd6ea2420089cb07714e0d19c0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.742523 1166 raft_consensus.cc:385] T 6a206abd6ea2420089cb07714e0d19c0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.742566 1166 raft_consensus.cc:740] T 6a206abd6ea2420089cb07714e0d19c0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.741480 2959 leader_election.cc:290] T 7a0a20f95416425489af94408ac1e97c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:49.748796 1160 tablet_bootstrap.cc:492] T ee7b93b0b87541d9af6eb9e68b4cea41 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:49.749382 1160 ts_tablet_manager.cc:1403] T ee7b93b0b87541d9af6eb9e68b4cea41 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.077s user 0.004s sys 0.000s
I20251211 20:52:49.753743 1160 raft_consensus.cc:359] T ee7b93b0b87541d9af6eb9e68b4cea41 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.754057 1160 raft_consensus.cc:385] T ee7b93b0b87541d9af6eb9e68b4cea41 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.754156 1160 raft_consensus.cc:740] T ee7b93b0b87541d9af6eb9e68b4cea41 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.754568 1160 consensus_queue.cc:260] T ee7b93b0b87541d9af6eb9e68b4cea41 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.755983 1166 consensus_queue.cc:260] T 6a206abd6ea2420089cb07714e0d19c0 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.756690 1166 ts_tablet_manager.cc:1434] T 6a206abd6ea2420089cb07714e0d19c0 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
W20251211 20:52:49.757174 710 consensus_peers.cc:597] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6029c1fd563d4023a8cf7cfb8ffa0876. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:49.757433 1166 tablet_bootstrap.cc:492] T d53b074865f4490693152a247b0fffdc P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:49.759297 1168 tablet_bootstrap.cc:492] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:49.759438 1168 ts_tablet_manager.cc:1403] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.050s user 0.000s sys 0.020s
I20251211 20:52:49.759626 3184 consensus_queue.cc:1048] T 94658fcf512a48d0a992a7e2e0f7c176 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:49.760205 1166 tablet_bootstrap.cc:654] T d53b074865f4490693152a247b0fffdc P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.761350 1168 raft_consensus.cc:359] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.761552 1168 raft_consensus.cc:385] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.761197 2962 ts_tablet_manager.cc:1929] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:49.761610 1168 raft_consensus.cc:740] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.761749 2962 log.cc:1199] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/06b8f8e46e524545aeab6c931efaf116
I20251211 20:52:49.761773 1168 consensus_queue.cc:260] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.762256 2962 ts_tablet_manager.cc:1950] T 06b8f8e46e524545aeab6c931efaf116 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
W20251211 20:52:49.762498 710 consensus_peers.cc:597] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6a61cf8f5de14adaa4a1de5bddde9a2f. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:49.763366 2962 tablet_replica.cc:333] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:49.763454 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 06b8f8e46e524545aeab6c931efaf116 (table default.Table_bf0900ef3fb8447c834a43e509caa105 [id=861b00629ceb42588a1d1be34153b792]) successfully deleted
I20251211 20:52:49.763631 2962 raft_consensus.cc:2243] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:49.763774 2962 raft_consensus.cc:2272] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:49.764874 2962 ts_tablet_manager.cc:1916] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:49.765297 710 consensus_peers.cc:597] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:49.765725 2550 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:52:49.766883 2550 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:52:49.767201 2550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:52:49.768350 3133 tablet.cc:1722] T 7be44df39a8f4d6da793479d425fa684 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.770972 1168 ts_tablet_manager.cc:1434] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20251211 20:52:49.773937 1168 tablet_bootstrap.cc:492] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:49.774652 1160 ts_tablet_manager.cc:1434] T ee7b93b0b87541d9af6eb9e68b4cea41 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.025s user 0.002s sys 0.000s
I20251211 20:52:49.775079 1160 tablet_bootstrap.cc:492] T e484d499af1b4947adb36c32f8ea5704 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:49.776902 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7a0a20f95416425489af94408ac1e97c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:49.777267 1160 tablet_bootstrap.cc:654] T e484d499af1b4947adb36c32f8ea5704 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.783329 1168 tablet_bootstrap.cc:654] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.789736 3166 tablet.cc:1722] T 7be44df39a8f4d6da793479d425fa684 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:49.802273 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fa79b67b9eee45e2b41c4a90967606cc" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:49.802874 1168 tablet_bootstrap.cc:492] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
W20251211 20:52:49.802894 710 leader_election.cc:343] T 7a0a20f95416425489af94408ac1e97c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:49.803035 1168 ts_tablet_manager.cc:1403] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.002s
I20251211 20:52:49.803313 3155 consensus_queue.cc:1048] T 71dc8d20d93243e289330d7b8c635511 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:49.806998 3155 consensus_queue.cc:1048] T de508d79f30f489cb745401f53115165 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:49.810463 1168 raft_consensus.cc:359] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.810670 1168 raft_consensus.cc:385] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.810719 1168 raft_consensus.cc:740] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.810907 1168 consensus_queue.cc:260] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.811388 1168 ts_tablet_manager.cc:1434] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20251211 20:52:49.812994 1168 tablet_bootstrap.cc:492] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:49.814044 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7a0a20f95416425489af94408ac1e97c" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:49.816234 1168 tablet_bootstrap.cc:654] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.822400 2981 raft_consensus.cc:493] T c112714331bf4bd7a1ddcca17f338d4f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:49.822574 2981 raft_consensus.cc:515] T c112714331bf4bd7a1ddcca17f338d4f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.823189 2981 leader_election.cc:290] T c112714331bf4bd7a1ddcca17f338d4f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:49.823699 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c112714331bf4bd7a1ddcca17f338d4f" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:49.824364 903 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c112714331bf4bd7a1ddcca17f338d4f" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:49.825387 982 leader_election.cc:343] T c112714331bf4bd7a1ddcca17f338d4f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:49.827759 983 leader_election.cc:343] T c112714331bf4bd7a1ddcca17f338d4f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:49.827905 983 leader_election.cc:304] T c112714331bf4bd7a1ddcca17f338d4f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:49.828387 2974 raft_consensus.cc:2749] T c112714331bf4bd7a1ddcca17f338d4f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:49.831248 903 raft_consensus.cc:3060] T fa79b67b9eee45e2b41c4a90967606cc P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:49.831317 1096 heartbeater.cc:499] Master 127.0.104.61:40749 was elected leader, sending a full tablet report...
W20251211 20:52:49.844682 847 leader_election.cc:343] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 14e2650f8ec1482ea9511b989fdc963c
I20251211 20:52:49.844820 847 leader_election.cc:304] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:49.845042 3174 raft_consensus.cc:2749] T 14e2650f8ec1482ea9511b989fdc963c P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:52:49.848245 847 leader_election.cc:304] T fa79b67b9eee45e2b41c4a90967606cc P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
W20251211 20:52:49.849812 847 leader_election.cc:343] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 2d849c62f7434099a16f62c2a2fc5e85
I20251211 20:52:49.849977 847 leader_election.cc:304] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:49.850452 2964 raft_consensus.cc:2749] T 2d849c62f7434099a16f62c2a2fc5e85 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:49.851372 3174 raft_consensus.cc:2749] T fa79b67b9eee45e2b41c4a90967606cc P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:49.855412 3155 consensus_queue.cc:1048] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:49.855979 1160 tablet_bootstrap.cc:492] T e484d499af1b4947adb36c32f8ea5704 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:49.856194 1160 ts_tablet_manager.cc:1403] T e484d499af1b4947adb36c32f8ea5704 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.081s user 0.004s sys 0.000s
I20251211 20:52:49.857391 1160 raft_consensus.cc:359] T e484d499af1b4947adb36c32f8ea5704 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.857669 1160 raft_consensus.cc:385] T e484d499af1b4947adb36c32f8ea5704 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.857885 1160 raft_consensus.cc:740] T e484d499af1b4947adb36c32f8ea5704 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.858304 1160 consensus_queue.cc:260] T e484d499af1b4947adb36c32f8ea5704 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.859771 1160 ts_tablet_manager.cc:1434] T e484d499af1b4947adb36c32f8ea5704 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:49.860133 1160 tablet_bootstrap.cc:492] T b85af79c2574443998e847bc19c53b69 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:49.862144 1160 tablet_bootstrap.cc:654] T b85af79c2574443998e847bc19c53b69 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.869869 2962 ts_tablet_manager.cc:1929] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:49.870096 2962 log.cc:1199] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/8b1ce6d2a12a4c7ea044b349412b4bf5
I20251211 20:52:49.870500 2962 ts_tablet_manager.cc:1950] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:49.871224 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 8b1ce6d2a12a4c7ea044b349412b4bf5 (table default.Table_8044d887ac4b408283fbb97b19ac6af6 [id=f964a6ca434049278c880c3089197039]) successfully deleted
I20251211 20:52:49.871431 2962 tablet_replica.cc:333] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:49.871639 2962 raft_consensus.cc:2243] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
W20251211 20:52:49.871551 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet e95b0d8c507a4779a087c8933d464228 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e95b0d8c507a4779a087c8933d464228 already in progress: creating tablet
I20251211 20:52:49.871800 2962 raft_consensus.cc:2272] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:49.871934 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 877aa237fe43452cbdbb4dac46f613ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 877aa237fe43452cbdbb4dac46f613ae already in progress: creating tablet
I20251211 20:52:49.872344 2962 ts_tablet_manager.cc:1916] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:49.874816 3051 raft_consensus.cc:493] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:49.875010 3051 raft_consensus.cc:515] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.876828 3051 leader_election.cc:290] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:49.876948 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "04679dfddf564d6f8c16d1c257e29b41" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:49.877146 1033 raft_consensus.cc:2468] T 04679dfddf564d6f8c16d1c257e29b41 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:49.877884 710 leader_election.cc:304] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:49.878191 3155 raft_consensus.cc:2804] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:49.878270 3155 raft_consensus.cc:493] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:49.878329 3155 raft_consensus.cc:3060] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:49.880965 3155 raft_consensus.cc:515] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.882190 1166 tablet_bootstrap.cc:492] T d53b074865f4490693152a247b0fffdc P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:49.882388 1166 ts_tablet_manager.cc:1403] T d53b074865f4490693152a247b0fffdc P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.125s user 0.028s sys 0.032s
I20251211 20:52:49.882885 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "04679dfddf564d6f8c16d1c257e29b41" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:49.883082 1033 raft_consensus.cc:3060] T 04679dfddf564d6f8c16d1c257e29b41 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:49.883978 1166 raft_consensus.cc:359] T d53b074865f4490693152a247b0fffdc P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.884137 1166 raft_consensus.cc:385] T d53b074865f4490693152a247b0fffdc P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.884184 1166 raft_consensus.cc:740] T d53b074865f4490693152a247b0fffdc P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.882202 3155 leader_election.cc:290] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:49.884378 1166 consensus_queue.cc:260] T d53b074865f4490693152a247b0fffdc P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.885864 1166 ts_tablet_manager.cc:1434] T d53b074865f4490693152a247b0fffdc P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:49.886160 1033 raft_consensus.cc:2468] T 04679dfddf564d6f8c16d1c257e29b41 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:49.886595 710 leader_election.cc:304] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:49.886777 1166 tablet_bootstrap.cc:492] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:49.886922 3155 raft_consensus.cc:2804] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:49.887066 3155 raft_consensus.cc:697] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:49.888408 1166 tablet_bootstrap.cc:654] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.888352 3155 consensus_queue.cc:237] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.894547 3051 raft_consensus.cc:493] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:49.894809 3051 raft_consensus.cc:515] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.895479 3051 leader_election.cc:290] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:49.896548 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f2ae084982f44fdf8bfecdd2bc4c01b9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:49.896800 1048 raft_consensus.cc:2468] T f2ae084982f44fdf8bfecdd2bc4c01b9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:49.897050 1160 tablet_bootstrap.cc:492] T b85af79c2574443998e847bc19c53b69 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:49.897276 1160 ts_tablet_manager.cc:1403] T b85af79c2574443998e847bc19c53b69 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.037s user 0.003s sys 0.001s
I20251211 20:52:49.897315 710 leader_election.cc:304] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:49.897564 3051 raft_consensus.cc:2804] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:49.897675 3051 raft_consensus.cc:493] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:49.897776 3051 raft_consensus.cc:3060] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:49.897964 2966 raft_consensus.cc:493] T d37a7d0d662e45079b0cfc02765d2e01 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:49.898080 2966 raft_consensus.cc:515] T d37a7d0d662e45079b0cfc02765d2e01 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.898623 2966 leader_election.cc:290] T d37a7d0d662e45079b0cfc02765d2e01 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:49.899101 1160 raft_consensus.cc:359] T b85af79c2574443998e847bc19c53b69 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.900071 1160 raft_consensus.cc:385] T b85af79c2574443998e847bc19c53b69 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.900372 1160 raft_consensus.cc:740] T b85af79c2574443998e847bc19c53b69 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.900990 1160 consensus_queue.cc:260] T b85af79c2574443998e847bc19c53b69 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.902499 1160 ts_tablet_manager.cc:1434] T b85af79c2574443998e847bc19c53b69 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251211 20:52:49.903112 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d37a7d0d662e45079b0cfc02765d2e01" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:49.903785 1160 tablet_bootstrap.cc:492] T 0763a475527546bb83cee8fb823893ac P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:49.903436 3051 raft_consensus.cc:515] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.905405 1160 tablet_bootstrap.cc:654] T 0763a475527546bb83cee8fb823893ac P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.906215 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f2ae084982f44fdf8bfecdd2bc4c01b9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:49.906445 1033 raft_consensus.cc:3060] T f2ae084982f44fdf8bfecdd2bc4c01b9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:49.906859 710 consensus_peers.cc:597] T 8b1ce6d2a12a4c7ea044b349412b4bf5 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8b1ce6d2a12a4c7ea044b349412b4bf5. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:49.907610 1166 tablet_bootstrap.cc:492] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:49.907747 1166 ts_tablet_manager.cc:1403] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.022s user 0.005s sys 0.000s
I20251211 20:52:49.908463 1166 raft_consensus.cc:359] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.908622 1166 raft_consensus.cc:385] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.908689 1166 raft_consensus.cc:740] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.908903 1166 consensus_queue.cc:260] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.909816 1033 raft_consensus.cc:2468] T f2ae084982f44fdf8bfecdd2bc4c01b9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:49.910480 710 leader_election.cc:304] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:49.910704 2959 raft_consensus.cc:2804] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:49.910791 2959 raft_consensus.cc:697] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:49.910838 1168 tablet_bootstrap.cc:492] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:49.910969 1168 ts_tablet_manager.cc:1403] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.098s user 0.016s sys 0.016s
I20251211 20:52:49.911000 2959 consensus_queue.cc:237] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.911717 1168 raft_consensus.cc:359] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.911862 1168 raft_consensus.cc:385] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.911916 1168 raft_consensus.cc:740] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.912066 1168 consensus_queue.cc:260] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.912560 1166 ts_tablet_manager.cc:1434] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251211 20:52:49.913146 1168 ts_tablet_manager.cc:1434] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:49.913406 1168 tablet_bootstrap.cc:492] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:49.913542 1166 tablet_bootstrap.cc:492] T 4fe5c534b572491c911aae01eaa79bae P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:49.915079 1168 tablet_bootstrap.cc:654] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.915081 1166 tablet_bootstrap.cc:654] T 4fe5c534b572491c911aae01eaa79bae P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.920157 3051 leader_election.cc:290] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:49.923602 711 leader_election.cc:343] T 7a0a20f95416425489af94408ac1e97c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:49.923723 1166 tablet_bootstrap.cc:492] T 4fe5c534b572491c911aae01eaa79bae P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:49.923857 1166 ts_tablet_manager.cc:1403] T 4fe5c534b572491c911aae01eaa79bae P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.002s
I20251211 20:52:49.923897 711 leader_election.cc:304] T 7a0a20f95416425489af94408ac1e97c P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:49.924574 1166 raft_consensus.cc:359] T 4fe5c534b572491c911aae01eaa79bae P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.924759 1166 raft_consensus.cc:385] T 4fe5c534b572491c911aae01eaa79bae P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.924810 1166 raft_consensus.cc:740] T 4fe5c534b572491c911aae01eaa79bae P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.925930 2970 raft_consensus.cc:2749] T 7a0a20f95416425489af94408ac1e97c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:49.928764 1166 consensus_queue.cc:260] T 4fe5c534b572491c911aae01eaa79bae P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.929814 1166 ts_tablet_manager.cc:1434] T 4fe5c534b572491c911aae01eaa79bae P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20251211 20:52:49.930090 1166 tablet_bootstrap.cc:492] T e59f8afe1834485d9fc64615e7bfe174 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:49.933894 903 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "04679dfddf564d6f8c16d1c257e29b41" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:49.936327 1166 tablet_bootstrap.cc:654] T e59f8afe1834485d9fc64615e7bfe174 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.943485 1166 tablet_bootstrap.cc:492] T e59f8afe1834485d9fc64615e7bfe174 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:49.944193 1166 ts_tablet_manager.cc:1403] T e59f8afe1834485d9fc64615e7bfe174 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.000s
I20251211 20:52:49.944679 903 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "04679dfddf564d6f8c16d1c257e29b41" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:49.945007 1166 raft_consensus.cc:359] T e59f8afe1834485d9fc64615e7bfe174 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.945181 1166 raft_consensus.cc:385] T e59f8afe1834485d9fc64615e7bfe174 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.945241 1166 raft_consensus.cc:740] T e59f8afe1834485d9fc64615e7bfe174 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.945674 903 raft_consensus.cc:3060] T 0aac6fb3972e4aba8950828c9e49d507 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:49.946959 1784 consensus_peers.cc:597] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05 -> Peer 9230f94a6a1a49cead51aeea9f069087 (127.0.104.60:42087): Couldn't send request to peer 9230f94a6a1a49cead51aeea9f069087. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.60:42087: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20251211 20:52:49.948172 907 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f2ae084982f44fdf8bfecdd2bc4c01b9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:49.948514 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f2ae084982f44fdf8bfecdd2bc4c01b9" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:49.949851 1166 consensus_queue.cc:260] T e59f8afe1834485d9fc64615e7bfe174 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.950747 908 raft_consensus.cc:3060] T d37a7d0d662e45079b0cfc02765d2e01 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:49.951737 1168 tablet_bootstrap.cc:492] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:49.951869 1168 ts_tablet_manager.cc:1403] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.039s user 0.003s sys 0.000s
I20251211 20:52:49.951872 1166 ts_tablet_manager.cc:1434] T e59f8afe1834485d9fc64615e7bfe174 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20251211 20:52:49.952698 2970 raft_consensus.cc:493] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:49.952667 1168 raft_consensus.cc:359] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.952836 1168 raft_consensus.cc:385] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.952806 2970 raft_consensus.cc:515] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.952883 1168 raft_consensus.cc:740] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.953088 1168 consensus_queue.cc:260] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.953562 1168 ts_tablet_manager.cc:1434] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:49.953851 1168 tablet_bootstrap.cc:492] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:49.955338 2970 leader_election.cc:290] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:49.957353 1166 tablet_bootstrap.cc:492] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:49.957870 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d37a7d0d662e45079b0cfc02765d2e01" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:49.959092 1166 tablet_bootstrap.cc:654] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.959472 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d53b074865f4490693152a247b0fffdc" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:49.959673 1033 raft_consensus.cc:2468] T d53b074865f4490693152a247b0fffdc P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:49.961009 710 leader_election.cc:304] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:49.961603 2970 raft_consensus.cc:2804] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:49.961728 2970 raft_consensus.cc:493] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:49.961870 1160 tablet_bootstrap.cc:492] T 0763a475527546bb83cee8fb823893ac P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:49.961928 2970 raft_consensus.cc:3060] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:49.962066 1160 ts_tablet_manager.cc:1403] T 0763a475527546bb83cee8fb823893ac P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.058s user 0.007s sys 0.008s
I20251211 20:52:49.962826 1160 raft_consensus.cc:359] T 0763a475527546bb83cee8fb823893ac P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.963055 1160 raft_consensus.cc:385] T 0763a475527546bb83cee8fb823893ac P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.963217 1160 raft_consensus.cc:740] T 0763a475527546bb83cee8fb823893ac P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.963471 1160 consensus_queue.cc:260] T 0763a475527546bb83cee8fb823893ac P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.964136 1160 ts_tablet_manager.cc:1434] T 0763a475527546bb83cee8fb823893ac P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:49.964434 1160 tablet_bootstrap.cc:492] T c26c3d0170f64a41ae8944c6668cc5f3 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:49.965019 3110 tablet.cc:1722] T d37a7d0d662e45079b0cfc02765d2e01 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:49.966051 1160 tablet_bootstrap.cc:654] T c26c3d0170f64a41ae8944c6668cc5f3 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.966936 2962 ts_tablet_manager.cc:1929] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:49.967206 2962 log.cc:1199] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/28ddb8a59a214576a25d613d42c2fef5
I20251211 20:52:49.967674 2962 ts_tablet_manager.cc:1950] T 28ddb8a59a214576a25d613d42c2fef5 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:49.968580 2962 tablet_replica.cc:333] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:49.968772 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 28ddb8a59a214576a25d613d42c2fef5 (table default.Table_e98180a63672423688b0de80af514e8b [id=658bb4c0b37e492c94080811d899c16b]) successfully deleted
I20251211 20:52:49.970096 2970 raft_consensus.cc:515] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.973577 1168 tablet_bootstrap.cc:654] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.980681 1160 tablet_bootstrap.cc:492] T c26c3d0170f64a41ae8944c6668cc5f3 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:49.980929 1160 ts_tablet_manager.cc:1403] T c26c3d0170f64a41ae8944c6668cc5f3 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.000s
I20251211 20:52:49.981199 2970 leader_election.cc:290] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:49.981765 1160 raft_consensus.cc:359] T c26c3d0170f64a41ae8944c6668cc5f3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.982009 1160 raft_consensus.cc:385] T c26c3d0170f64a41ae8944c6668cc5f3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.982097 1160 raft_consensus.cc:740] T c26c3d0170f64a41ae8944c6668cc5f3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.982350 1160 consensus_queue.cc:260] T c26c3d0170f64a41ae8944c6668cc5f3 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.983016 1160 ts_tablet_manager.cc:1434] T c26c3d0170f64a41ae8944c6668cc5f3 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:49.983320 1160 tablet_bootstrap.cc:492] T 1578887b2f5748cd83a6fbdec2f3fc99 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:49.985100 1160 tablet_bootstrap.cc:654] T 1578887b2f5748cd83a6fbdec2f3fc99 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.987175 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d53b074865f4490693152a247b0fffdc" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:49.970758 1166 tablet_bootstrap.cc:492] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:49.987718 1166 ts_tablet_manager.cc:1403] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.031s user 0.004s sys 0.000s
I20251211 20:52:49.988374 3174 raft_consensus.cc:493] T e484d499af1b4947adb36c32f8ea5704 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:49.988489 3174 raft_consensus.cc:515] T e484d499af1b4947adb36c32f8ea5704 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.988513 1166 raft_consensus.cc:359] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.988689 1166 raft_consensus.cc:385] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:49.988763 1166 raft_consensus.cc:740] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:49.988958 1166 consensus_queue.cc:260] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.989053 3174 leader_election.cc:290] T e484d499af1b4947adb36c32f8ea5704 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:49.989427 1166 ts_tablet_manager.cc:1434] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:49.971701 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d53b074865f4490693152a247b0fffdc" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:49.989761 1033 raft_consensus.cc:3060] T d53b074865f4490693152a247b0fffdc P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:49.990526 847 leader_election.cc:304] T d37a7d0d662e45079b0cfc02765d2e01 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:49.991173 3174 raft_consensus.cc:2749] T d37a7d0d662e45079b0cfc02765d2e01 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:49.991707 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e484d499af1b4947adb36c32f8ea5704" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:49.993096 1033 raft_consensus.cc:2468] T d53b074865f4490693152a247b0fffdc P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:49.993511 2962 raft_consensus.cc:2243] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:49.993520 710 leader_election.cc:304] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:49.993705 2962 raft_consensus.cc:2272] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:49.994266 2962 ts_tablet_manager.cc:1916] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:49.994679 2429 raft_consensus.cc:2804] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:49.994808 2429 raft_consensus.cc:697] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:49.995028 2429 consensus_queue.cc:237] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.996192 1166 tablet_bootstrap.cc:492] T 116b21a182474ca191c4a89e9474677d P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:49.998006 1166 tablet_bootstrap.cc:654] T 116b21a182474ca191c4a89e9474677d P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:49.998399 1160 tablet_bootstrap.cc:492] T 1578887b2f5748cd83a6fbdec2f3fc99 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:49.998730 1160 ts_tablet_manager.cc:1403] T 1578887b2f5748cd83a6fbdec2f3fc99 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.001s
I20251211 20:52:49.999607 1160 raft_consensus.cc:359] T 1578887b2f5748cd83a6fbdec2f3fc99 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:49.999926 1160 raft_consensus.cc:385] T 1578887b2f5748cd83a6fbdec2f3fc99 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.000036 1160 raft_consensus.cc:740] T 1578887b2f5748cd83a6fbdec2f3fc99 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.000293 1160 consensus_queue.cc:260] T 1578887b2f5748cd83a6fbdec2f3fc99 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.000957 1160 ts_tablet_manager.cc:1434] T 1578887b2f5748cd83a6fbdec2f3fc99 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:50.001268 1160 tablet_bootstrap.cc:492] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:50.007776 1160 tablet_bootstrap.cc:654] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:50.009016 711 leader_election.cc:343] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.009569 3174 raft_consensus.cc:493] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.009735 3174 raft_consensus.cc:515] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.010133 1166 tablet_bootstrap.cc:492] T 116b21a182474ca191c4a89e9474677d P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:50.010295 1166 ts_tablet_manager.cc:1403] T 116b21a182474ca191c4a89e9474677d P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.003s
I20251211 20:52:50.010360 3174 leader_election.cc:290] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.011137 1166 raft_consensus.cc:359] T 116b21a182474ca191c4a89e9474677d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.011349 1166 raft_consensus.cc:385] T 116b21a182474ca191c4a89e9474677d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.011443 1166 raft_consensus.cc:740] T 116b21a182474ca191c4a89e9474677d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.011730 1166 consensus_queue.cc:260] T 116b21a182474ca191c4a89e9474677d P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:50.012756 711 leader_election.cc:343] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:50.013602 847 leader_election.cc:343] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: e28d66ce4711430cb09a9650e99fbe9e
I20251211 20:52:50.013638 1166 ts_tablet_manager.cc:1434] T 116b21a182474ca191c4a89e9474677d P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251211 20:52:50.014279 1168 tablet_bootstrap.cc:492] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:50.014434 1168 ts_tablet_manager.cc:1403] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.061s user 0.003s sys 0.002s
I20251211 20:52:50.014959 1166 tablet_bootstrap.cc:492] T 062f980b65504d3baf99ae400e521775 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:50.013305 711 leader_election.cc:343] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.015631 1168 raft_consensus.cc:359] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.015842 1168 raft_consensus.cc:385] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.015908 1168 raft_consensus.cc:740] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
W20251211 20:52:50.015951 711 leader_election.cc:343] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.016124 1168 consensus_queue.cc:260] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:50.016379 711 consensus_peers.cc:597] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:50.016860 1166 tablet_bootstrap.cc:654] T 062f980b65504d3baf99ae400e521775 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:50.019304 711 leader_election.cc:343] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.021168 903 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d53b074865f4490693152a247b0fffdc" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:50.022629 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e484d499af1b4947adb36c32f8ea5704" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:50.023100 711 leader_election.cc:343] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.024755 903 raft_consensus.cc:3060] T 4d2b7e2a0abe4592bc806d5ef3f894d8 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:50.025012 847 leader_election.cc:343] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: e28d66ce4711430cb09a9650e99fbe9e
I20251211 20:52:50.025118 847 leader_election.cc:304] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:50.025564 847 leader_election.cc:304] T e484d499af1b4947adb36c32f8ea5704 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:50.025869 3174 raft_consensus.cc:2749] T e28d66ce4711430cb09a9650e99fbe9e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:50.026018 3174 raft_consensus.cc:2749] T e484d499af1b4947adb36c32f8ea5704 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:50.032249 2981 raft_consensus.cc:493] T e59f8afe1834485d9fc64615e7bfe174 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.032451 2981 raft_consensus.cc:515] T e59f8afe1834485d9fc64615e7bfe174 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.033138 2981 leader_election.cc:290] T e59f8afe1834485d9fc64615e7bfe174 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:50.033558 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e59f8afe1834485d9fc64615e7bfe174" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:50.033641 907 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e59f8afe1834485d9fc64615e7bfe174" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:50.034040 983 leader_election.cc:343] T e59f8afe1834485d9fc64615e7bfe174 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:50.034202 982 leader_election.cc:343] T e59f8afe1834485d9fc64615e7bfe174 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.034346 982 leader_election.cc:304] T e59f8afe1834485d9fc64615e7bfe174 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:50.034888 2981 raft_consensus.cc:2749] T e59f8afe1834485d9fc64615e7bfe174 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:50.035240 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet aab26df748724fe5b2bb7fad9e3bfc0c (table default.Table_22bed4b4e9e24a90818b19c6f5c91e1f [id=df6255d66f334ef2b4406f4b845cf980]). Expected version 1 got 0
I20251211 20:52:50.037811 3251 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 11f5f94732f049d0be4c97cebfffcb05)
I20251211 20:52:50.038544 3251 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:50.039633 1829 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1830b0c58cfe495989f3d911dda0409a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1297 } } ignore_live_leader: false dest_uuid: "11f5f94732f049d0be4c97cebfffcb05" is_pre_election: true
I20251211 20:52:50.040143 1168 ts_tablet_manager.cc:1434] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.026s user 0.003s sys 0.000s
I20251211 20:52:50.040518 1168 tablet_bootstrap.cc:492] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:50.042567 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b52377c8ffd741a598e5328a00d876fe (table default.Table_686a26915b42421380037144900dd97c [id=8a1b0c9d641b4a3c83d1ca48576d110d]). Expected version 1 got 0
I20251211 20:52:50.043031 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet c7a2610f20df4c1ca81eae2fc7dba6bb (table default.Table_59072235d93b41c0b5cd5e3debcb8ddf [id=9a15a8a075af4969ae2c8b3be1bfed78]). Expected version 1 got 0
I20251211 20:52:50.043658 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 0302cc8613934c43a0650f081cf98797 (table default.Table_bf827b216f1b43c196923dc1828ff0d0 [id=ed371e073250439fa44330c7101e9387]). Expected version 1 got 0
I20251211 20:52:50.044040 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 1f4eb1a174c648c897ae2b7612e44585 (table default.Table_e70873b71ba24d7eae548e2acfd243b3 [id=87adcc552a704d56bd2fe812fb3b791f]). Expected version 1 got 0
I20251211 20:52:50.044409 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet a7afcce522494c9ca1d80e6eea913e6e (table default.Table_cb6a4bac7b694f33a37de2ee78c91ca0 [id=8940f18527c249a9baefc170ed82f45a]). Expected version 1 got 0
I20251211 20:52:50.044600 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 84bb3583857848459e0e865283de623d (table default.Table_92281c5fcfee4355a79215ffedeee588 [id=6e5fe0a78e4b48f98dac028802f1ddf4]). Expected version 4 got 0
I20251211 20:52:50.044895 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 7a299ee03d0e48aba7c4f71e3e0f381a (table default.Table_6402a801d48e4a53827b1b62b6cdaba1 [id=6876565d3a9c48ce841b830cf75139ff]). Expected version 3 got 0
I20251211 20:52:50.045282 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 5b3e8ec3faf445ac9b746e65dc4dbfe0 (table default.Table_7fe80ff5483a4b0292091119980f6639 [id=3c3462a0c09748cea0d2ffb544ae2f13]). Expected version 1 got 0
I20251211 20:52:50.045615 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 7ffa06c4a7ee48b0b337a27aff090592 (table default.Table_647731023b044ebb8954f7fdcc3faf97 [id=4b91abd6e9e94a8a941b33e62e23c083]). Expected version 1 got 0
I20251211 20:52:50.045771 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 81fe26e81790448fb436929c03e3c284 (table default.Table_efc22dc69ad0405ebf59fda961e23330 [id=d8a0839df7524fb7b95bfc05fd4ada25]). Expected version 1 got 0
I20251211 20:52:50.045941 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b81a4cef27164824b52695039744db15 (table default.Table_defdeab28749411590cf75b5251f604b [id=11352566d1744193a40a8bb605aa3912]). Expected version 1 got 0
I20251211 20:52:50.046123 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 7023afec769c4ca3b2242a225004d482 (table default.Table_d5276e3cc46d4331b36a8878680fc55e [id=3811f8235b2f44cca283401302d613d4]). Expected version 1 got 0
I20251211 20:52:50.046334 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 72af55c8045b4f8f8e116a5782794d81 (table default.Table_2cff6fab088c4d939b13b504c133cc12 [id=b6c4b72063b3428593d5650a160dc66b]). Expected version 1 got 0
I20251211 20:52:50.046373 3051 raft_consensus.cc:493] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.046545 3051 raft_consensus.cc:515] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.046872 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 4cf0cde801bf44449e539d2799b895a7 (table default.Table_907c596bdddc4edca25d14cbacebb23f [id=321235280d1d4fa0bae5cc0333860f85]). Expected version 1 got 0
I20251211 20:52:50.047079 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 7b9f569554fb4836ac2b62542c3ecf7e (table default.Table_1a73d567e8914a83974e3e8f5f58be0b [id=6ef691749df24251a7a7c5919c219754]). Expected version 1 got 0
I20251211 20:52:50.047199 3051 leader_election.cc:290] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.047238 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3b2c6064c5eb4c0f8979c485f5d85a75 (table default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f]). Expected version 1 got 0
I20251211 20:52:50.047467 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 938a0d8a090f490b816fdf2b8f1a51b8 (table default.Table_5ca4ab25a21d456c95a0ff68c9d73330 [id=9ffcb594267a4b0eb8d792dbe46d585f]). Expected version 1 got 0
I20251211 20:52:50.047626 907 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c081396b492049b996ac4661e1318388" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:50.048257 711 leader_election.cc:343] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.048681 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c081396b492049b996ac4661e1318388" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:50.049033 1033 raft_consensus.cc:2468] T c081396b492049b996ac4661e1318388 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:50.049430 710 leader_election.cc:304] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:50.049778 3051 raft_consensus.cc:2804] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:50.049850 3051 raft_consensus.cc:493] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:50.049913 3051 raft_consensus.cc:3060] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.053182 3051 raft_consensus.cc:515] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.054076 3051 leader_election.cc:290] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.054071 907 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c081396b492049b996ac4661e1318388" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:50.054584 711 leader_election.cc:343] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.055769 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c081396b492049b996ac4661e1318388" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:50.056049 1033 raft_consensus.cc:3060] T c081396b492049b996ac4661e1318388 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.059043 1033 raft_consensus.cc:2468] T c081396b492049b996ac4661e1318388 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:50.059633 710 leader_election.cc:304] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:50.060034 3051 raft_consensus.cc:2804] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:50.061108 3051 raft_consensus.cc:697] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:50.061375 3051 consensus_queue.cc:237] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.070873 1168 tablet_bootstrap.cc:654] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.072934 1160 tablet_bootstrap.cc:492] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:50.073093 1160 ts_tablet_manager.cc:1403] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.072s user 0.003s sys 0.001s
I20251211 20:52:50.073905 1160 raft_consensus.cc:359] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.074085 1160 raft_consensus.cc:385] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.074132 1160 raft_consensus.cc:740] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.074380 1160 consensus_queue.cc:260] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.076021 1160 ts_tablet_manager.cc:1434] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:50.077584 3251 leader_election.cc:290] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11f5f94732f049d0be4c97cebfffcb05 (127.0.104.61:40749), 9230f94a6a1a49cead51aeea9f069087 (127.0.104.60:42087)
I20251211 20:52:50.079883 1160 tablet_bootstrap.cc:492] T 7452dd736b0440fcb4626858e794676f P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:50.081019 1295 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.60:42087: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20251211 20:52:50.082625 1160 tablet_bootstrap.cc:654] T 7452dd736b0440fcb4626858e794676f P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.087702 2962 ts_tablet_manager.cc:1929] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:50.088007 2962 log.cc:1199] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/7452dd736b0440fcb4626858e794676f
I20251211 20:52:50.088553 2962 ts_tablet_manager.cc:1950] T 7452dd736b0440fcb4626858e794676f P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:50.089251 907 raft_consensus.cc:3060] T ee7b93b0b87541d9af6eb9e68b4cea41 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.089886 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 134) took 1026 ms. Trace:
I20251211 20:52:50.089996 2962 rpcz_store.cc:276] 1211 20:52:49.063701 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.063740 (+ 39us) service_pool.cc:225] Handling call
1211 20:52:49.968535 (+904795us) ts_tablet_manager.cc:1151] Deleting tablet 7452dd736b0440fcb4626858e794676f
1211 20:52:49.968544 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:50.038809 (+ 70265us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:50.062326 (+ 23517us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:50.089832 (+ 27506us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":904634}
I20251211 20:52:50.090488 2962 tablet_replica.cc:333] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:50.090643 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 7452dd736b0440fcb4626858e794676f (table default.Table_3542335cc1af47b3817527896d07f03a [id=d1e2bcd20b06470bb87adf24e7b8e7fc]) successfully deleted
I20251211 20:52:50.091051 2962 raft_consensus.cc:2243] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
W20251211 20:52:50.091190 710 consensus_peers.cc:597] T e1aeae8c18b6499e9dc1accabdc3f633 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e1aeae8c18b6499e9dc1accabdc3f633. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:50.091280 2962 raft_consensus.cc:2272] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:50.094810 1168 tablet_bootstrap.cc:492] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:50.094971 1168 ts_tablet_manager.cc:1403] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.055s user 0.002s sys 0.003s
I20251211 20:52:50.096627 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 59acb8474e114bf7ae7f1f043d611459 (table default.Table_c7ed8c0c309c40a3a371ffa130b47cd2 [id=ed1d0670426945aba888b818ad657225]). Expected version 1 got 0
I20251211 20:52:50.096998 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet bbe6e6220eb14e59a821805c438230c9 (table default.Table_16a12ed370b64a38b5e145c4eb024349 [id=b7429ea93123473ab5d795d5f3d5a5f8]). Expected version 1 got 0
I20251211 20:52:50.097612 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet ce1799e6604f4ed7875ddedecab6f327 (table default.Table_5db1ac98e36d45b1b01889ee275ad83d [id=a6849726434d41d29dce12f47066a685]). Expected version 1 got 0
I20251211 20:52:50.097892 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet bc4e7362e6d045da81bf39f4135f500e (table default.Table_328b7dd76e9e407c83b47a0f51cfcce5 [id=9e7c08f833e14a90b17a9b9f60b702de]). Expected version 1 got 0
I20251211 20:52:50.098055 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet dfd194cf00f549c8937cb6ec40987240 (table default.Table_25e38d0576684687890bae04bd81d4f4 [id=d919c6132b3c466e8a1404f981b003b8]). Expected version 2 got 0
I20251211 20:52:50.098435 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 8b793241ee3b42b1a38e9f2f05e1173b (table default.Table_7506ffecc2b64648a9ba736e5298ddf4 [id=279f39d718ba45318f52f95e8aa13652]). Expected version 2 got 0
I20251211 20:52:50.098827 1168 raft_consensus.cc:359] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.099108 1168 raft_consensus.cc:385] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.099227 1168 raft_consensus.cc:740] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.099289 1166 tablet_bootstrap.cc:492] T 062f980b65504d3baf99ae400e521775 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:50.099524 1166 ts_tablet_manager.cc:1403] T 062f980b65504d3baf99ae400e521775 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.085s user 0.016s sys 0.017s
W20251211 20:52:50.100582 1295 leader_election.cc:336] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9230f94a6a1a49cead51aeea9f069087 (127.0.104.60:42087): Network error: Client connection negotiation failed: client connection to 127.0.104.60:42087: connect: Connection refused (error 111)
I20251211 20:52:50.100714 1166 raft_consensus.cc:359] T 062f980b65504d3baf99ae400e521775 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.100947 1166 raft_consensus.cc:385] T 062f980b65504d3baf99ae400e521775 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.101044 1166 raft_consensus.cc:740] T 062f980b65504d3baf99ae400e521775 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.101313 1166 consensus_queue.cc:260] T 062f980b65504d3baf99ae400e521775 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.101019 1168 consensus_queue.cc:260] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.102039 1168 ts_tablet_manager.cc:1434] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20251211 20:52:50.103353 1166 ts_tablet_manager.cc:1434] T 062f980b65504d3baf99ae400e521775 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20251211 20:52:50.103667 1295 leader_election.cc:304] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1830b0c58cfe495989f3d911dda0409a; no voters: 11f5f94732f049d0be4c97cebfffcb05, 9230f94a6a1a49cead51aeea9f069087
I20251211 20:52:50.104082 1168 tablet_bootstrap.cc:492] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:50.104266 3251 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20251211 20:52:50.105118 1166 tablet_bootstrap.cc:492] T 29463a8f71f943a0927942235af3c2da P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:50.105942 1168 tablet_bootstrap.cc:654] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.108752 1166 tablet_bootstrap.cc:654] T 29463a8f71f943a0927942235af3c2da P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.110497 3174 raft_consensus.cc:493] T 0763a475527546bb83cee8fb823893ac P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.110678 3174 raft_consensus.cc:515] T 0763a475527546bb83cee8fb823893ac P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.111290 3174 leader_election.cc:290] T 0763a475527546bb83cee8fb823893ac P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.112120 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0763a475527546bb83cee8fb823893ac" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:50.112196 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0763a475527546bb83cee8fb823893ac" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:50.113202 847 leader_election.cc:304] T 0763a475527546bb83cee8fb823893ac P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:50.113736 3174 raft_consensus.cc:2749] T 0763a475527546bb83cee8fb823893ac P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:50.117873 2970 consensus_queue.cc:1048] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:50.118425 903 raft_consensus.cc:3060] T 0763a475527546bb83cee8fb823893ac P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.120213 3051 raft_consensus.cc:493] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.120353 3051 raft_consensus.cc:515] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.122006 3051 leader_election.cc:290] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.122136 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6a206abd6ea2420089cb07714e0d19c0" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:50.122377 1048 raft_consensus.cc:2468] T 6a206abd6ea2420089cb07714e0d19c0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:50.122728 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6a206abd6ea2420089cb07714e0d19c0" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:50.122782 710 leader_election.cc:304] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:50.123174 3051 raft_consensus.cc:2804] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:50.123317 3051 raft_consensus.cc:493] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:50.123450 3051 raft_consensus.cc:3060] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:50.124192 711 leader_election.cc:343] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:50.126784 711 consensus_peers.cc:597] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:50.131139 1160 tablet_bootstrap.cc:492] T 7452dd736b0440fcb4626858e794676f P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:50.132560 1160 ts_tablet_manager.cc:1403] T 7452dd736b0440fcb4626858e794676f P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.053s user 0.001s sys 0.003s
I20251211 20:52:50.132761 3051 raft_consensus.cc:515] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.133846 3051 leader_election.cc:290] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.133844 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6a206abd6ea2420089cb07714e0d19c0" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:50.136466 711 leader_election.cc:343] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.134017 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6a206abd6ea2420089cb07714e0d19c0" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:50.136905 1033 raft_consensus.cc:3060] T 6a206abd6ea2420089cb07714e0d19c0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.140475 2962 ts_tablet_manager.cc:1916] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:50.141345 1160 raft_consensus.cc:359] T 7452dd736b0440fcb4626858e794676f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.145593 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet ab4e7a9062134e489d6f9abe6ff26fb4 (table default.Table_4cbc8841b276406cb4434629653a5ce3 [id=b958532650374d609199cab9c07bb64b]). Expected version 2 got 0
I20251211 20:52:50.146965 2981 raft_consensus.cc:493] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.147125 2981 raft_consensus.cc:515] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.147790 2981 leader_election.cc:290] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:50.148600 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "903c7f4b312d4677bb35be1aaa46ad6b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:50.148793 775 raft_consensus.cc:2468] T 903c7f4b312d4677bb35be1aaa46ad6b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 0.
I20251211 20:52:50.149333 982 leader_election.cc:304] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:50.142956 3110 tablet.cc:1722] T ee7b93b0b87541d9af6eb9e68b4cea41 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:50.148600 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "903c7f4b312d4677bb35be1aaa46ad6b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:50.151394 2981 raft_consensus.cc:2804] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:50.151427 1160 raft_consensus.cc:385] T 7452dd736b0440fcb4626858e794676f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.151506 2981 raft_consensus.cc:493] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:50.151527 1160 raft_consensus.cc:740] T 7452dd736b0440fcb4626858e794676f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.151604 2981 raft_consensus.cc:3060] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.151932 1160 consensus_queue.cc:260] T 7452dd736b0440fcb4626858e794676f P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.152850 1160 ts_tablet_manager.cc:1434] T 7452dd736b0440fcb4626858e794676f P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.020s user 0.002s sys 0.000s
I20251211 20:52:50.153165 1160 tablet_bootstrap.cc:492] T 46cb1d16a0e94b3a833f32c635213fd2 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:50.154371 983 leader_election.cc:343] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.155373 2981 raft_consensus.cc:515] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.156756 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "903c7f4b312d4677bb35be1aaa46ad6b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:50.156946 775 raft_consensus.cc:3060] T 903c7f4b312d4677bb35be1aaa46ad6b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.157166 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 6b7dd8232d974a43b99ec9b28075dc8d (table default.Table_9ef53e1f9a5c405bb08dc1afeb63b47a [id=2bf05710731345a786503ee222ff6f55]). Expected version 2 got 0
I20251211 20:52:50.157544 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 645b08b33676420c96e880de3d460829 (table default.Table_e02933dd91ce4fe7a40d1c3b2f9b2d4b [id=6d42f260246e420d8818b3841926302b]). Expected version 2 got 0
I20251211 20:52:50.157769 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 46cb1d16a0e94b3a833f32c635213fd2 (table default.Table_3482fbfc5b144a7198a7d01aa003cb6b [id=1bdaf5ce0f6b4b99a038f9f9efb828f6]). Expected version 1 got 0
I20251211 20:52:50.158113 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 7ca98b43863a47c091baae0a6ce65630 (table default.Table_f0a70914acc94231a4e84155c3e4d13f [id=56c070ad80624c698ebe2e01b4929b6e]). Expected version 1 got 0
I20251211 20:52:50.158278 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet e3f711f02af843adb4f5c3c750924e33 (table default.Table_34bd5155aa3b4ef7818022ba9346c074 [id=83a61d23e80946439a14096ab3ab1bd9]). Expected version 2 got 0
I20251211 20:52:50.159148 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 969a8587019c4202b6e705d126f83d44 (table default.Table_6460313002684bba93b30ad8631f0f49 [id=22347b15537d451d9927a17e8e5a202a]). Expected version 1 got 0
I20251211 20:52:50.159435 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 676faec7f3d1488f99b220f8e968e9a0 (table default.Table_ab826a3eec8e4c4aaa8d4021d87dd382 [id=a2bbfccfe3974c378521d5ef2333df86]). Expected version 2 got 0
I20251211 20:52:50.159739 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 78f77d8916d84edabbe95ce17d8df2c1 (table default.Table_e2af0e2fbd3d46d595136c7a7bf24e4c [id=ddc12ae6d8ed4587b5ef4da3ef81a055]). Expected version 1 got 0
I20251211 20:52:50.159744 2974 raft_consensus.cc:493] T 4b07f110f25645799ed0c89737cd8279 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.159934 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet df457783b2534b3ea23bb1fd529ce268 (table default.Table_6e9f3ca164384cc8ac4501deb3f0c6aa [id=342c2469d6e74e3e9217236e3aa863d6]). Expected version 1 got 0
I20251211 20:52:50.159914 2974 raft_consensus.cc:515] T 4b07f110f25645799ed0c89737cd8279 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.160285 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 9f8aa28c1c9b4fc58dfc128952c180ba (table default.Table_bf7d402a36be44cfb6b79041906d26a1 [id=0327e686e7ed4475a3e8cda8721a3a3a]). Expected version 1 got 0
I20251211 20:52:50.160579 2974 leader_election.cc:290] T 4b07f110f25645799ed0c89737cd8279 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:50.161144 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet d6923cfd826d495291d9ac8e0b3b44c6 (table default.Table_68ccb2052d824ccf92f7162d2961fcb1 [id=9ac9591d4f04482d926710fdf6e797c6]). Expected version 1 got 0
I20251211 20:52:50.162689 2981 leader_election.cc:290] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:50.162612 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4b07f110f25645799ed0c89737cd8279" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:50.162614 775 raft_consensus.cc:2468] T 903c7f4b312d4677bb35be1aaa46ad6b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 1.
W20251211 20:52:50.163229 982 leader_election.cc:343] T 4b07f110f25645799ed0c89737cd8279 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.164986 982 leader_election.cc:304] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:50.165696 2974 raft_consensus.cc:2804] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:50.178226 1033 raft_consensus.cc:2468] T 6a206abd6ea2420089cb07714e0d19c0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:50.178817 710 leader_election.cc:304] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:50.179214 3000 raft_consensus.cc:2804] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:50.179327 3000 raft_consensus.cc:697] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:50.179598 3000 consensus_queue.cc:237] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.189082 1168 tablet_bootstrap.cc:492] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:50.189267 1168 ts_tablet_manager.cc:1403] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.087s user 0.003s sys 0.000s
I20251211 20:52:50.190084 1168 raft_consensus.cc:359] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.190263 1168 raft_consensus.cc:385] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.190308 1168 raft_consensus.cc:740] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.190548 1168 consensus_queue.cc:260] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.191220 1168 ts_tablet_manager.cc:1434] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:50.191519 1168 tablet_bootstrap.cc:492] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:50.198180 711 consensus_peers.cc:597] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:50.158871 3174 raft_consensus.cc:493] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.200176 3174 raft_consensus.cc:515] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.200881 3174 leader_election.cc:290] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.163282 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4b07f110f25645799ed0c89737cd8279" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:50.199323 903 raft_consensus.cc:3060] T e484d499af1b4947adb36c32f8ea5704 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.156383 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "903c7f4b312d4677bb35be1aaa46ad6b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:50.203037 983 leader_election.cc:343] T 4b07f110f25645799ed0c89737cd8279 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.203198 983 leader_election.cc:304] T 4b07f110f25645799ed0c89737cd8279 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:50.203701 983 leader_election.cc:343] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.205433 2981 raft_consensus.cc:2749] T 4b07f110f25645799ed0c89737cd8279 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:50.205627 847 leader_election.cc:343] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: bb0fc179a337469eae056c5bb3815e6b
W20251211 20:52:50.210184 847 leader_election.cc:343] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: bb0fc179a337469eae056c5bb3815e6b
I20251211 20:52:50.210345 847 leader_election.cc:304] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:50.210935 3174 raft_consensus.cc:2749] T bb0fc179a337469eae056c5bb3815e6b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:50.212164 907 raft_consensus.cc:3060] T c26c3d0170f64a41ae8944c6668cc5f3 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.154942 1160 tablet_bootstrap.cc:654] T 46cb1d16a0e94b3a833f32c635213fd2 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.227041 3174 raft_consensus.cc:493] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.227185 3174 raft_consensus.cc:515] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.227730 3174 leader_election.cc:290] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:50.228741 847 leader_election.cc:343] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 92a71dff4ea8445d8484fc2747471148
I20251211 20:52:50.229173 3174 raft_consensus.cc:493] T b85af79c2574443998e847bc19c53b69 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.229295 3174 raft_consensus.cc:515] T b85af79c2574443998e847bc19c53b69 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:50.229575 847 leader_election.cc:343] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 92a71dff4ea8445d8484fc2747471148
I20251211 20:52:50.229683 847 leader_election.cc:304] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:50.229816 3174 leader_election.cc:290] T b85af79c2574443998e847bc19c53b69 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.229955 3174 raft_consensus.cc:2749] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:50.230541 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b85af79c2574443998e847bc19c53b69" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:50.232095 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b85af79c2574443998e847bc19c53b69" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:50.232661 847 leader_election.cc:304] T b85af79c2574443998e847bc19c53b69 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:50.233114 2964 raft_consensus.cc:2749] T b85af79c2574443998e847bc19c53b69 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:50.236092 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b2974f16034c44cf85ff85b5095cf6a0 (table default.Table_031de0f4cc7744bc9e320261f8b46696 [id=be36dabaa6d44325b4855b9f13bda847]). Expected version 1 got 0
I20251211 20:52:50.236408 1166 tablet_bootstrap.cc:492] T 29463a8f71f943a0927942235af3c2da P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:50.236564 1166 ts_tablet_manager.cc:1403] T 29463a8f71f943a0927942235af3c2da P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.132s user 0.025s sys 0.028s
I20251211 20:52:50.237141 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet f94b656e3c1844509fdae36e33b263e8 (table default.Table_ed03ad475bed4804a65abe20efef0625 [id=e23f4305669a474aafa18928f130de3a]). Expected version 2 got 0
I20251211 20:52:50.237388 1166 raft_consensus.cc:359] T 29463a8f71f943a0927942235af3c2da P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:50.237982 710 consensus_peers.cc:597] T 28ddb8a59a214576a25d613d42c2fef5 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 28ddb8a59a214576a25d613d42c2fef5. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:50.238174 1166 raft_consensus.cc:385] T 29463a8f71f943a0927942235af3c2da P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.238291 1166 raft_consensus.cc:740] T 29463a8f71f943a0927942235af3c2da P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.238592 1166 consensus_queue.cc:260] T 29463a8f71f943a0927942235af3c2da P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.239212 1166 ts_tablet_manager.cc:1434] T 29463a8f71f943a0927942235af3c2da P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:50.240760 1160 tablet_bootstrap.cc:492] T 46cb1d16a0e94b3a833f32c635213fd2 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:50.240994 1160 ts_tablet_manager.cc:1403] T 46cb1d16a0e94b3a833f32c635213fd2 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.088s user 0.002s sys 0.002s
I20251211 20:52:50.241923 1166 tablet_bootstrap.cc:492] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:50.237327 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 4c0455a5b4f44412803475d1c4852b48 (table default.Table_e00092673b0e4f09b8d7eec63e1ee034 [id=18497e06d7254441a4655ce1cdc5a9bf]). Expected version 1 got 0
I20251211 20:52:50.243037 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet aa50ddb2789b4be3bfa9aedef9287109 (table default.Table_44f971096d1e41f480e594985ffd5c17 [id=d4e1349374a346a5b8be180f9bdc329a]). Expected version 1 got 0
I20251211 20:52:50.243410 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 4c7b74f370b042ee8575f821c07dc085 (table default.Table_731826f26f9940b8afe425d694b642d9 [id=17dd9cea9ac844128ff9f82adb3f919f]). Expected version 1 got 0
I20251211 20:52:50.243620 1166 tablet_bootstrap.cc:654] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.243760 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 497c10742f01476db885bd70c4d2c491 (table default.Table_18b21232dfa74378b27defe672bb2574 [id=602d7eee703948d99a0edcb7cac0651b]). Expected version 1 got 0
I20251211 20:52:50.244005 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3003ef71a3554c71a3b24cdfb9c293bc (table default.Table_92efd469dd324f7d886e2cbc90ece61c [id=6e7fb3271e0c48c0bb9d56fe4fccee3a]). Expected version 1 got 0
I20251211 20:52:50.244367 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet f2d6c9781b4d4c37bc08703347963d5d (table default.Table_2aa7ff9186a04249abf8cd971cc0cb17 [id=b50ab5b032ba4c98938324460512468f]). Expected version 1 got 0
I20251211 20:52:50.244750 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 16d2ff8925e342729a71c93db378d031 (table default.Table_34e6ed4f13da437ea50d349b92e83bd6 [id=4a74b9a60cbb4198b55cee630b2f4bef]). Expected version 2 got 0
I20251211 20:52:50.245301 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 5e47af616e1b4d2db3780c1d67cdcb47 (table default.Table_296bdf30a11541c389afe3df0b0f667a [id=052edcbde60c41449d468c8a449fc3f1]). Expected version 1 got 0
I20251211 20:52:50.246201 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 5453f68789fd4e89b69e98860050865b (table default.Table_b7bf8c5ad5304175a1f2f6a03887f725 [id=50d88b663a4d4891b113aee9c5906699]). Expected version 1 got 0
I20251211 20:52:50.246915 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet a3b1aadf17ea4bf0a896d23cd543fca7 (table default.Table_d18401707c88493abdf90d2c713432ae [id=a07662f8dc3f4178be5dfcc1cdc85837]). Expected version 1 got 0
I20251211 20:52:50.247884 2964 raft_consensus.cc:493] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.248042 2964 raft_consensus.cc:515] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.248698 2964 leader_election.cc:290] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.247562 1160 raft_consensus.cc:359] T 46cb1d16a0e94b3a833f32c635213fd2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.249687 1160 raft_consensus.cc:385] T 46cb1d16a0e94b3a833f32c635213fd2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.249800 1160 raft_consensus.cc:740] T 46cb1d16a0e94b3a833f32c635213fd2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
W20251211 20:52:50.250505 847 leader_election.cc:343] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 2197822e0f11484f9679c4858626ac2c
I20251211 20:52:50.250527 2974 raft_consensus.cc:697] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 LEADER]: Becoming Leader. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Running, Role: LEADER
I20251211 20:52:50.250802 2974 consensus_queue.cc:237] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.250102 1160 consensus_queue.cc:260] T 46cb1d16a0e94b3a833f32c635213fd2 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:50.251209 847 leader_election.cc:343] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 2197822e0f11484f9679c4858626ac2c
I20251211 20:52:50.251328 847 leader_election.cc:304] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:50.251860 2964 raft_consensus.cc:2749] T 2197822e0f11484f9679c4858626ac2c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:50.254060 1160 ts_tablet_manager.cc:1434] T 46cb1d16a0e94b3a833f32c635213fd2 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20251211 20:52:50.255442 1160 tablet_bootstrap.cc:492] T 47779034ad89466e9e1801987128b4dd P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:50.238768 2962 ts_tablet_manager.cc:1929] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:50.259851 2962 log.cc:1199] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/3562b5effa124378beac7ccf0d27ee7e
W20251211 20:52:50.260632 710 consensus_peers.cc:597] T 7452dd736b0440fcb4626858e794676f P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7452dd736b0440fcb4626858e794676f. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:50.260893 2962 ts_tablet_manager.cc:1950] T 3562b5effa124378beac7ccf0d27ee7e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:50.261241 1166 tablet_bootstrap.cc:492] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:50.261436 1166 ts_tablet_manager.cc:1403] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.002s
I20251211 20:52:50.262245 1166 raft_consensus.cc:359] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.262562 1166 raft_consensus.cc:385] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.262699 1166 raft_consensus.cc:740] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.262985 1166 consensus_queue.cc:260] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.264406 1166 ts_tablet_manager.cc:1434] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:50.264782 1166 tablet_bootstrap.cc:492] T 4388893262b54ad8be1bdf82516031a5 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:50.266556 1168 tablet_bootstrap.cc:654] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.267118 1166 tablet_bootstrap.cc:654] T 4388893262b54ad8be1bdf82516031a5 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:50.271335 710 consensus_peers.cc:597] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3562b5effa124378beac7ccf0d27ee7e. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:50.273752 2974 raft_consensus.cc:493] T 116b21a182474ca191c4a89e9474677d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.273960 2974 raft_consensus.cc:515] T 116b21a182474ca191c4a89e9474677d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.274554 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 135) took 1210 ms. Trace:
I20251211 20:52:50.275096 903 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "116b21a182474ca191c4a89e9474677d" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:50.275286 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "116b21a182474ca191c4a89e9474677d" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:50.275888 1166 tablet_bootstrap.cc:492] T 4388893262b54ad8be1bdf82516031a5 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
W20251211 20:52:50.275923 982 leader_election.cc:343] T 116b21a182474ca191c4a89e9474677d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.276607 1166 ts_tablet_manager.cc:1403] T 4388893262b54ad8be1bdf82516031a5 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.001s
W20251211 20:52:50.276763 983 leader_election.cc:343] T 116b21a182474ca191c4a89e9474677d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.276850 983 leader_election.cc:304] T 116b21a182474ca191c4a89e9474677d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:50.277099 2981 raft_consensus.cc:2749] T 116b21a182474ca191c4a89e9474677d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:50.277417 1166 raft_consensus.cc:359] T 4388893262b54ad8be1bdf82516031a5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.277786 1166 raft_consensus.cc:385] T 4388893262b54ad8be1bdf82516031a5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.277846 1166 raft_consensus.cc:740] T 4388893262b54ad8be1bdf82516031a5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.278029 1166 consensus_queue.cc:260] T 4388893262b54ad8be1bdf82516031a5 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.277706 2974 leader_election.cc:290] T 116b21a182474ca191c4a89e9474677d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:50.278861 1166 ts_tablet_manager.cc:1434] T 4388893262b54ad8be1bdf82516031a5 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:50.279420 1166 tablet_bootstrap.cc:492] T 59e955aa07c14fbbb347b88eca59a74e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:50.274715 2962 rpcz_store.cc:276] 1211 20:52:49.064062 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.064101 (+ 39us) service_pool.cc:225] Handling call
1211 20:52:50.090406 (+1026305us) ts_tablet_manager.cc:1151] Deleting tablet 3562b5effa124378beac7ccf0d27ee7e
1211 20:52:50.090421 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:50.181039 (+ 90618us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:50.238635 (+ 57596us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:50.274527 (+ 35892us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1026159}
I20251211 20:52:50.283448 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 136) took 1219 ms. Trace:
I20251211 20:52:50.283527 2962 rpcz_store.cc:276] 1211 20:52:49.064363 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.064400 (+ 37us) service_pool.cc:225] Handling call
1211 20:52:50.283334 (+1218934us) ts_tablet_manager.cc:1151] Deleting tablet 14f367adb60744c8a593d3b9d4711823
1211 20:52:50.283346 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:50.283434 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1218765}
I20251211 20:52:50.283820 2962 tablet_replica.cc:333] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:50.284004 2962 raft_consensus.cc:2243] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:50.284143 2962 raft_consensus.cc:2272] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:50.284358 1166 tablet_bootstrap.cc:654] T 59e955aa07c14fbbb347b88eca59a74e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.285087 2962 ts_tablet_manager.cc:1916] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:50.286723 1160 tablet_bootstrap.cc:654] T 47779034ad89466e9e1801987128b4dd P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.283667 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 3562b5effa124378beac7ccf0d27ee7e (table default.Table_7f5df390c11943918af61066a7c67c66 [id=157a0b5022e445e68ac805168ebc586f]) successfully deleted
W20251211 20:52:50.288401 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 14f367adb60744c8a593d3b9d4711823 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14f367adb60744c8a593d3b9d4711823 already in progress: creating tablet
I20251211 20:52:50.288797 1166 tablet_bootstrap.cc:492] T 59e955aa07c14fbbb347b88eca59a74e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:50.288939 1166 ts_tablet_manager.cc:1403] T 59e955aa07c14fbbb347b88eca59a74e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20251211 20:52:50.289669 1166 raft_consensus.cc:359] T 59e955aa07c14fbbb347b88eca59a74e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.289839 1166 raft_consensus.cc:385] T 59e955aa07c14fbbb347b88eca59a74e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.289896 1166 raft_consensus.cc:740] T 59e955aa07c14fbbb347b88eca59a74e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.290087 1166 consensus_queue.cc:260] T 59e955aa07c14fbbb347b88eca59a74e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.290504 1166 ts_tablet_manager.cc:1434] T 59e955aa07c14fbbb347b88eca59a74e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251211 20:52:50.290513 1168 tablet_bootstrap.cc:492] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:50.290777 1166 tablet_bootstrap.cc:492] T 683d0f2068fb4a6ebc9c169ad7b455ce P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:50.290640 1168 ts_tablet_manager.cc:1403] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.099s user 0.003s sys 0.000s
I20251211 20:52:50.291532 1168 raft_consensus.cc:359] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.291738 1168 raft_consensus.cc:385] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.291787 1168 raft_consensus.cc:740] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.292011 1168 consensus_queue.cc:260] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.292109 1166 tablet_bootstrap.cc:654] T 683d0f2068fb4a6ebc9c169ad7b455ce P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.288419 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet bc6a1c84501d4e179cd2828b50fddc23 (table default.Table_8bb5b228d60e4d29a41d9ba40c33d45d [id=6ef058fada9941629e0ea4d1906abdac]). Expected version 1 got 0
I20251211 20:52:50.293073 1168 ts_tablet_manager.cc:1434] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:50.293627 1168 tablet_bootstrap.cc:492] T 51d12723f5194afb97d2c05fcbaf3776 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:50.293058 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 03184dcc56dd4e92adc7d0ad22aa8d8b (table default.Table_68261841754847ad8d22e86c29886144 [id=697f23a617f64762bf406b386cde7473]). Expected version 1 got 0
I20251211 20:52:50.296375 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b85af79c2574443998e847bc19c53b69 (table default.Table_fa3f2f115cd043a9aa6be2b070d320cb [id=15fad5e1cf8d4ea08dae8275a29a11dd]). Expected version 1 got 0
I20251211 20:52:50.297092 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 8dc4e730951144d78b77280f6cce6d0e (table default.Table_890c9027415a44ccaf95c0adede51992 [id=7bc8a2b91fe54b91b6594b371fea7bd3]). Expected version 2 got 0
I20251211 20:52:50.297694 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 13269a20421240d1a98215129443248b (table default.Table_d887c953026046dc883ede774a58fd8e [id=1d8a79f716a0447cb0393f4759178598]). Expected version 1 got 0
I20251211 20:52:50.298032 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 51d12723f5194afb97d2c05fcbaf3776 (table default.Table_823505200e3142dbbe8cd56ac78daf55 [id=ec0bb10933fa4529bef3580bee6409e7]). Expected version 2 got 0
I20251211 20:52:50.298933 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 46a2a070cf6a42fe8bc54a42eb802fbc (table default.Table_4e1f03b9d87b4fe59dbcb3a947ee299d [id=8e66ea58499b40e59294a135324aa527]). Expected version 1 got 0
I20251211 20:52:50.301633 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 5773c13034714a008c8d7dbe777b4828 (table default.Table_66eb1b021eed464f82f9cd223c88297b [id=94ff09564cd649c7a2d0e94b0f3c7b3a]). Expected version 1 got 0
I20251211 20:52:50.301983 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 18f125dd9efc401181f0644dad7aeffe (table default.Table_fa6592c2e2624d81ad21abb13ed36045 [id=9b48fc2b39db4fd5a4e5da19e34427dc]). Expected version 1 got 0
I20251211 20:52:50.302764 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b6cc819332e1445c96ce3ffac4861454 (table default.Table_db4c0f706e0643b39d39ffa84352826d [id=4d459c42bf584068ac3851227d80331a]). Expected version 1 got 0
I20251211 20:52:50.302992 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet ff4d5ec1f87247e896f9c099c532265a (table default.Table_3ac4746e17264f5a93890f3600505781 [id=475e0a1ec0344d10addbcfa42fe05b68]). Expected version 1 got 0
I20251211 20:52:50.303295 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3dfea28c44d84c68850db65a30d52ee6 (table default.Table_c504f8f3796f43b697488bd604268543 [id=2af76b9d32094a4ca656060a4478e08a]). Expected version 1 got 0
I20251211 20:52:50.303623 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet f00128ba84c245d9a0e35f26bc842b3e (table default.Table_6f8c133c7a364b93af2c83d8911a0ac3 [id=6a21c38ee7224a02b937df088494f5d8]). Expected version 1 got 0
I20251211 20:52:50.302759 1168 tablet_bootstrap.cc:654] T 51d12723f5194afb97d2c05fcbaf3776 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.304512 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3c865c99e9314f5f91658b9126da4d39 (table default.Table_0d7df3fba38849a49b65d8290812b95c [id=7f966d5cdad3423bb770414802a3cf7b]). Expected version 1 got 0
I20251211 20:52:50.306569 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet e89d4d76bf64453892a4d0b55fcc3454 (table default.Table_fba99384e9f14160b3e3504f8d1d0b07 [id=01bba75f854d4c338d6c09a446dd7499]). Expected version 1 got 0
I20251211 20:52:50.306842 1160 tablet_bootstrap.cc:492] T 47779034ad89466e9e1801987128b4dd P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:50.307062 1160 ts_tablet_manager.cc:1403] T 47779034ad89466e9e1801987128b4dd P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.052s user 0.003s sys 0.000s
I20251211 20:52:50.307199 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 834b84a557b04e78b5d2c8c84520a644 (table default.Table_909bbe6da1a648d68199fb27424e4342 [id=cc99459a5039424dbc192ff748b6708b]). Expected version 2 got 0
I20251211 20:52:50.307827 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet e4647105ab57454d883ebfcde7bde891 (table default.Table_c817dec9a7674f699b3366fc302ead57 [id=d555d1fc6aef4a21b4d3f2804264d030]). Expected version 1 got 0
I20251211 20:52:50.308063 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet ee7b93b0b87541d9af6eb9e68b4cea41 (table default.Table_b6e6aea68c88426fb01353146c06b6ac [id=a4f984b4263a4bc38f6248628f643ae3]). Expected version 3 got 0
I20251211 20:52:50.308406 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b115d737d7fc483d9f5a1f040b511bae (table default.Table_6d94053661f142b79292e19752df02df [id=c5be59a2c3934b3789641ceeb2d4d6e2]). Expected version 2 got 0
I20251211 20:52:50.308280 1160 raft_consensus.cc:359] T 47779034ad89466e9e1801987128b4dd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.309672 1160 raft_consensus.cc:385] T 47779034ad89466e9e1801987128b4dd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.309773 1160 raft_consensus.cc:740] T 47779034ad89466e9e1801987128b4dd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.310410 1160 consensus_queue.cc:260] T 47779034ad89466e9e1801987128b4dd P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.311808 1160 ts_tablet_manager.cc:1434] T 47779034ad89466e9e1801987128b4dd P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251211 20:52:50.312060 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet da81bf8854a24954ba4548185812295d (table default.Table_18262e22c26d4d66ba4c6134fbde0368 [id=da8cf2b99b42409091484a9513623d6f]). Expected version 1 got 0
I20251211 20:52:50.312433 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3e6f62c2deff4c11b137fc0cba5ee234 (table default.Table_51db250005ed4b3d947f32acd1d62563 [id=fdfaa0b24d2f4736aee695ed228212cf]). Expected version 1 got 0
I20251211 20:52:50.312990 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 33fd09ea06ee47908b85911e4ee79708 (table default.Table_3c3cd48c8fe5411e96cdf811d492925d [id=30620a01c27f456bac05fa7c02a2cb4d]). Expected version 2 got 0
I20251211 20:52:50.313138 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 6212b3b4425543168837946a54d60834 (table default.Table_8c30c4151d484db0bb9e2b9ca262248d [id=2d489cd7d73e403d84bf10960e6a2dfb]). Expected version 1 got 0
I20251211 20:52:50.313330 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 7be44df39a8f4d6da793479d425fa684 (table default.Table_f6a10556ff104db5bd5ecc9fc380e228 [id=f75813add54e46189ff1bbed82d83ba7]). Expected version 1 got 0
I20251211 20:52:50.313515 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet fd43a27db69044b68826cda87e16a086 (table default.Table_a8ebfd7d7a7f4f6cae8b6b02b99b80b0 [id=f1d32002c612403a82bdff5baf209b9e]). Expected version 1 got 0
I20251211 20:52:50.313817 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 94658fcf512a48d0a992a7e2e0f7c176 (table default.Table_c9d86e82aebb4ef0bc83789720352e38 [id=97377a8ef56c4cdebe95d15955207404]). Expected version 1 got 0
I20251211 20:52:50.313957 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet fb29f983c4524a49bdc5c0c1144ad708 (table default.Table_7809ac85fdfe4c1d9c58afb54edecb27 [id=c4b0bfff99f64888beb70d9154d2f156]). Expected version 1 got 0
I20251211 20:52:50.314500 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 171d40a8a9124f9bb1b38797fef8e5d5 (table default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5]). Expected version 2 got 0
I20251211 20:52:50.314684 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet c081396b492049b996ac4661e1318388 (table default.Table_00ae5c111de94ba0a2680ad22c726e16 [id=95c9ed1ec1834e2dbfd3198887b75369]). Expected version 1 got 0
I20251211 20:52:50.314929 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 4fe5c534b572491c911aae01eaa79bae (table default.Table_c1f9dca5c2144d50b67681dcc6800e77 [id=14fa8debfe5443ce81587f02dfa15847]). Expected version 2 got 0
I20251211 20:52:50.315075 1160 tablet_bootstrap.cc:492] T 44e2d2bdecf247baaac26e06bccc94de P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:50.315721 1166 tablet_bootstrap.cc:492] T 683d0f2068fb4a6ebc9c169ad7b455ce P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:50.315865 1166 ts_tablet_manager.cc:1403] T 683d0f2068fb4a6ebc9c169ad7b455ce P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.025s user 0.006s sys 0.000s
I20251211 20:52:50.316179 3000 raft_consensus.cc:493] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.316308 3000 raft_consensus.cc:515] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.317386 907 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f91487dd7e0f4c8b8483c2b83945da3b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:50.318177 711 leader_election.cc:343] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.318750 1160 tablet_bootstrap.cc:654] T 44e2d2bdecf247baaac26e06bccc94de P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.319200 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f91487dd7e0f4c8b8483c2b83945da3b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:50.319386 1033 raft_consensus.cc:2468] T f91487dd7e0f4c8b8483c2b83945da3b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:50.319830 710 leader_election.cc:304] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:50.320094 2429 raft_consensus.cc:2804] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:50.320155 2429 raft_consensus.cc:493] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:50.320207 2429 raft_consensus.cc:3060] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.320820 1166 raft_consensus.cc:359] T 683d0f2068fb4a6ebc9c169ad7b455ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.320963 1166 raft_consensus.cc:385] T 683d0f2068fb4a6ebc9c169ad7b455ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.321004 1166 raft_consensus.cc:740] T 683d0f2068fb4a6ebc9c169ad7b455ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.321215 1166 consensus_queue.cc:260] T 683d0f2068fb4a6ebc9c169ad7b455ce P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.322749 2429 raft_consensus.cc:515] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.323320 2429 leader_election.cc:290] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.325127 2964 raft_consensus.cc:493] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.325322 903 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f91487dd7e0f4c8b8483c2b83945da3b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:50.325304 2964 raft_consensus.cc:515] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.325935 2964 leader_election.cc:290] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.326890 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 2e800d8cc5ea4535a065d7d9243326b4 (table default.Table_083c478bedec45a5bf5203e4ed333860 [id=8baf0ea87d734d58896adbd0d6ef99c2]). Expected version 1 got 0
I20251211 20:52:50.326967 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f91487dd7e0f4c8b8483c2b83945da3b" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:50.327118 1033 raft_consensus.cc:3060] T f91487dd7e0f4c8b8483c2b83945da3b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.327137 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet d43210ec61af423ebde624397dea9f2e (table default.Table_af0df4d1ee144455a6041e5a6a0b9b80 [id=7c5b2e33fa1c4b12b3941577360908a8]). Expected version 3 got 0
I20251211 20:52:50.327314 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 8675e2a18f0f4a0e95a87005cf82d8b4 (table default.Table_bd9633c55bd242c18aac3a7f2c18a329 [id=767f50bee5464316a18f73e64bab817a]). Expected version 2 got 0
I20251211 20:52:50.327857 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet f26ba8f4bd3a4a1f8fa2ba6e80d543f8 (table default.Table_71e9c158dc22435a8fbb02aecca8919d [id=b966d0c0b49d4023894950c5dc06f678]). Expected version 2 got 0
W20251211 20:52:50.328075 711 leader_election.cc:343] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.328188 1799 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 1c4b2c22bd4c456eb77af45576de466f (table default.Table_e3a99b043e4f420a93ebfee6aa87f2ec [id=3e7d6cefcb7d412191c6540e9ad77c2d]). Expected version 2 got 0
I20251211 20:52:50.321735 1166 ts_tablet_manager.cc:1434] T 683d0f2068fb4a6ebc9c169ad7b455ce P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20251211 20:52:50.328647 1796 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Sending DeleteTablet for 3 replicas of tablet 8feee01fc0aa4b32be926ff2b617b894
I20251211 20:52:50.329227 1796 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46568 (request call id 99) took 2077 ms. Trace:
I20251211 20:52:50.329433 1166 tablet_bootstrap.cc:492] T 993616e226274c229a5924fbe69911af P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:50.329329 1796 rpcz_store.cc:276] 1211 20:52:48.251666 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.430050 (+1178384us) service_pool.cc:225] Handling call
1211 20:52:49.433019 (+ 2969us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
1211 20:52:49.434036 (+ 1017us) catalog_manager.cc:2815] Modifying in-memory table state
1211 20:52:49.434537 (+ 501us) catalog_manager.cc:2824] Locking tablets
1211 20:52:49.510301 (+ 75764us) catalog_manager.cc:2838] Removing table and tablets from system table
1211 20:52:49.518470 (+ 8169us) catalog_manager.cc:2856] Removing table from by-name map
1211 20:52:49.518718 (+ 248us) catalog_manager.cc:2872] Committing in-memory state
1211 20:52:50.328583 (+809865us) catalog_manager.cc:2876] Aborting table tasks
1211 20:52:50.329159 (+ 576us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:49.511187 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:49.511324 (+ 137us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:49.511331 (+ 7us) write_op.cc:435] Acquired schema lock
1211 20:52:49.511334 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:49.511458 (+ 124us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:49.511475 (+ 17us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:49.511481 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:49.511613 (+ 132us) tablet.cc:699] Row locks acquired
1211 20:52:49.511616 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:49.511657 (+ 41us) write_op.cc:270] Start()
1211 20:52:49.511690 (+ 33us) write_op.cc:276] Timestamp: P: 1765486369511632 usec, L: 0
1211 20:52:49.511693 (+ 3us) op_driver.cc:348] REPLICATION: starting
1211 20:52:49.511944 (+ 251us) log.cc:844] Serialized 765 byte log entry
1211 20:52:49.516168 (+ 4224us) op_driver.cc:464] REPLICATION: finished
1211 20:52:49.516438 (+ 270us) write_op.cc:301] APPLY: starting
1211 20:52:49.516457 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:49.516513 (+ 56us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:49.516551 (+ 38us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:49.516633 (+ 82us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:49.516673 (+ 40us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
1211 20:52:49.516682 (+ 9us) write_op.cc:312] APPLY: finished
1211 20:52:49.516774 (+ 92us) log.cc:844] Serialized 29 byte log entry
1211 20:52:49.516846 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:49.516871 (+ 25us) write_op.cc:454] Released schema lock
1211 20:52:49.516880 (+ 9us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:49.516969 (+ 89us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:49.516973 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":184,"apply.run_cpu_time_us":522,"apply.run_wall_time_us":522,"num_ops":2,"prepare.queue_time_us":64,"prepare.run_cpu_time_us":876,"prepare.run_wall_time_us":877,"raft.queue_time_us":1009,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":253,"replication_time_us":4441}]]}
I20251211 20:52:50.330837 1166 tablet_bootstrap.cc:654] T 993616e226274c229a5924fbe69911af P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.328667 1810 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: default.Table_819c4206e91e41358ac71e82185b4140 [id=28be836b71ed4107af7adc3fcb948b5e] alter complete (version 1)
I20251211 20:52:50.335183 2962 ts_tablet_manager.cc:1929] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:50.335281 2962 log.cc:1199] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/e9ead34065e240eeb1f946319537d32a
I20251211 20:52:50.335316 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 8feee01fc0aa4b32be926ff2b617b894 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:49 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.335608 2962 ts_tablet_manager.cc:1950] T e9ead34065e240eeb1f946319537d32a P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:50.336167 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 8feee01fc0aa4b32be926ff2b617b894 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:49 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.338202 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 6029c1fd563d4023a8cf7cfb8ffa0876 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.340584 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 516f7181aaac48cdadf9fbc08d0d3f57 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.340584 756 tablet_service.cc:1552] Processing DeleteTablet for tablet d0673c37c2fd47eea8ab10d9e91b8d7a with delete_type TABLET_DATA_DELETED (replaced by tablet 86c5228210954839ae7bf5bf0b5aa027) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.341183 1806 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 11f5f94732f049d0be4c97cebfffcb05: Sending DeleteTablet for 3 replicas of tablet 3fc173c396ff4af5a8589cff18165e85
I20251211 20:52:50.341785 1806 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46568 (request call id 90) took 2772 ms. Trace:
I20251211 20:52:50.341889 1806 rpcz_store.cc:276] 1211 20:52:47.569755 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:48.502576 (+932821us) service_pool.cc:225] Handling call
1211 20:52:48.503106 (+ 530us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
1211 20:52:48.503254 (+ 148us) catalog_manager.cc:2815] Modifying in-memory table state
1211 20:52:48.503334 (+ 80us) catalog_manager.cc:2824] Locking tablets
1211 20:52:48.503469 (+ 135us) catalog_manager.cc:2838] Removing table and tablets from system table
1211 20:52:48.546827 (+ 43358us) catalog_manager.cc:2856] Removing table from by-name map
1211 20:52:48.662075 (+115248us) catalog_manager.cc:2872] Committing in-memory state
1211 20:52:50.341131 (+1679056us) catalog_manager.cc:2876] Aborting table tasks
1211 20:52:50.341709 (+ 578us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:48.539204 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:48.539357 (+ 153us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:48.539365 (+ 8us) write_op.cc:435] Acquired schema lock
1211 20:52:48.539368 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:48.539522 (+ 154us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:48.539537 (+ 15us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:48.539543 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
1211 20:52:48.539691 (+ 148us) tablet.cc:699] Row locks acquired
1211 20:52:48.539694 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:48.539735 (+ 41us) write_op.cc:270] Start()
1211 20:52:48.539774 (+ 39us) write_op.cc:276] Timestamp: P: 1765486368539709 usec, L: 0
1211 20:52:48.539777 (+ 3us) op_driver.cc:348] REPLICATION: starting
1211 20:52:48.540025 (+ 248us) log.cc:844] Serialized 765 byte log entry
1211 20:52:48.545904 (+ 5879us) op_driver.cc:464] REPLICATION: finished
1211 20:52:48.545985 (+ 81us) write_op.cc:301] APPLY: starting
1211 20:52:48.546003 (+ 18us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:48.546064 (+ 61us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:48.546095 (+ 31us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:48.546170 (+ 75us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:48.546208 (+ 38us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
1211 20:52:48.546216 (+ 8us) write_op.cc:312] APPLY: finished
1211 20:52:48.546305 (+ 89us) log.cc:844] Serialized 29 byte log entry
1211 20:52:48.546368 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.546394 (+ 26us) write_op.cc:454] Released schema lock
1211 20:52:48.546402 (+ 8us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:48.560873 (+ 14471us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.560884 (+ 11us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":30,"apply.run_cpu_time_us":482,"apply.run_wall_time_us":481,"num_ops":2,"prepare.queue_time_us":34897,"prepare.run_cpu_time_us":1018,"prepare.run_wall_time_us":21655,"raft.queue_time_us":130,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":704,"replication_time_us":6088}]]}
I20251211 20:52:50.343830 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 137) took 1279 ms. Trace:
I20251211 20:52:50.343919 2962 rpcz_store.cc:276] 1211 20:52:49.064754 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.064793 (+ 39us) service_pool.cc:225] Handling call
1211 20:52:50.283779 (+1218986us) ts_tablet_manager.cc:1151] Deleting tablet e9ead34065e240eeb1f946319537d32a
1211 20:52:50.283788 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:50.322777 (+ 38989us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:50.335070 (+ 12293us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:50.343802 (+ 8732us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1218828}
I20251211 20:52:50.344200 2962 tablet_replica.cc:333] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:50.344372 2962 raft_consensus.cc:2243] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:50.344517 2962 raft_consensus.cc:2272] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:50.345427 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 3fc173c396ff4af5a8589cff18165e85 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:48 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.346097 1160 tablet_bootstrap.cc:492] T 44e2d2bdecf247baaac26e06bccc94de P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:50.346241 1160 ts_tablet_manager.cc:1403] T 44e2d2bdecf247baaac26e06bccc94de P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.031s user 0.006s sys 0.000s
I20251211 20:52:50.346988 1160 raft_consensus.cc:359] T 44e2d2bdecf247baaac26e06bccc94de P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.347168 1160 raft_consensus.cc:385] T 44e2d2bdecf247baaac26e06bccc94de P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.347226 1160 raft_consensus.cc:740] T 44e2d2bdecf247baaac26e06bccc94de P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.347427 1160 consensus_queue.cc:260] T 44e2d2bdecf247baaac26e06bccc94de P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.348726 2962 ts_tablet_manager.cc:1916] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:50.355175 2962 ts_tablet_manager.cc:1929] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:50.355324 2962 log.cc:1199] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/3621735fcb294e3a8c0180ba9ef86442
I20251211 20:52:50.356091 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 3fc173c396ff4af5a8589cff18165e85 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:48 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.357003 1160 ts_tablet_manager.cc:1434] T 44e2d2bdecf247baaac26e06bccc94de P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20251211 20:52:50.357419 1160 tablet_bootstrap.cc:492] T 18b977f463d5420186c007b38110e07e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:50.358392 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet e9ead34065e240eeb1f946319537d32a (table default.Table_3826c483ba654865be6be42ae333a65b [id=6addaf2424e9450e9a11ee71c17f055d]) successfully deleted
I20251211 20:52:50.358776 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 8feee01fc0aa4b32be926ff2b617b894 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:49 UTC) from {username='slave'} at 127.0.0.1:36220
W20251211 20:52:50.358927 847 leader_election.cc:343] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 52ef1707f6f2425aafd454f303af0333
I20251211 20:52:50.359002 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 3fc173c396ff4af5a8589cff18165e85 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:48 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.359094 1160 tablet_bootstrap.cc:654] T 18b977f463d5420186c007b38110e07e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:50.359678 1795 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fd43a27db69044b68826cda87e16a086 consensus state has no leader
I20251211 20:52:50.359859 1795 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46568 (request call id 98) took 2123 ms. Trace:
I20251211 20:52:50.359958 1795 rpcz_store.cc:276] 1211 20:52:48.236864 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.186002 (+949138us) service_pool.cc:225] Handling call
1211 20:52:49.186482 (+ 480us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
1211 20:52:49.186712 (+ 230us) catalog_manager.cc:3708] Apply alter schema
1211 20:52:49.301866 (+115154us) catalog_manager.cc:3740] Acquired catalog manager lock
1211 20:52:49.301951 (+ 85us) catalog_manager.cc:3898] Updating metadata on disk
1211 20:52:49.309577 (+ 7626us) catalog_manager.cc:3929] Committing alterations to in-memory state
1211 20:52:50.359826 (+1050249us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:49.303677 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:49.303821 (+ 144us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:49.303828 (+ 7us) write_op.cc:435] Acquired schema lock
1211 20:52:49.303831 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:49.303957 (+ 126us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:49.303972 (+ 15us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:49.303977 (+ 5us) tablet.cc:683] Acquiring locks for 1 operations
1211 20:52:49.304048 (+ 71us) tablet.cc:699] Row locks acquired
1211 20:52:49.304050 (+ 2us) write_op.cc:260] PREPARE: finished
1211 20:52:49.304091 (+ 41us) write_op.cc:270] Start()
1211 20:52:49.304127 (+ 36us) write_op.cc:276] Timestamp: P: 1765486369303749 usec, L: 0
1211 20:52:49.304131 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:49.304341 (+ 210us) log.cc:844] Serialized 405 byte log entry
1211 20:52:49.308347 (+ 4006us) op_driver.cc:464] REPLICATION: finished
1211 20:52:49.308476 (+ 129us) write_op.cc:301] APPLY: starting
1211 20:52:49.308492 (+ 16us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:49.308535 (+ 43us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:49.308570 (+ 35us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:49.308643 (+ 73us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:49.308676 (+ 33us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1211 20:52:49.308682 (+ 6us) write_op.cc:312] APPLY: finished
1211 20:52:49.308766 (+ 84us) log.cc:844] Serialized 23 byte log entry
1211 20:52:49.308834 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:49.308858 (+ 24us) write_op.cc:454] Released schema lock
1211 20:52:49.308865 (+ 7us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:49.308941 (+ 76us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:49.308945 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":82,"apply.run_cpu_time_us":452,"apply.run_wall_time_us":451,"num_ops":1,"prepare.queue_time_us":848,"prepare.run_cpu_time_us":786,"prepare.run_wall_time_us":785,"raft.queue_time_us":767,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":269,"replication_time_us":4185}]]}
I20251211 20:52:50.362257 2964 raft_consensus.cc:493] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.362401 2964 raft_consensus.cc:515] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.363010 2964 leader_election.cc:290] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:50.364454 847 leader_election.cc:343] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: e4b5a7553110424d901788c7a2ecdce6
I20251211 20:52:50.365902 1160 tablet_bootstrap.cc:492] T 18b977f463d5420186c007b38110e07e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:50.367231 1160 ts_tablet_manager.cc:1403] T 18b977f463d5420186c007b38110e07e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
W20251211 20:52:50.368101 711 consensus_peers.cc:597] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:50.368388 847 leader_election.cc:343] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 52ef1707f6f2425aafd454f303af0333
I20251211 20:52:50.368520 847 leader_election.cc:304] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:50.369498 2964 raft_consensus.cc:2749] T 52ef1707f6f2425aafd454f303af0333 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:50.369937 1160 raft_consensus.cc:359] T 18b977f463d5420186c007b38110e07e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.371002 1160 raft_consensus.cc:385] T 18b977f463d5420186c007b38110e07e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.371251 2962 ts_tablet_manager.cc:1950] T 3621735fcb294e3a8c0180ba9ef86442 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:50.371366 1160 raft_consensus.cc:740] T 18b977f463d5420186c007b38110e07e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.371742 1160 consensus_queue.cc:260] T 18b977f463d5420186c007b38110e07e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.372506 1033 raft_consensus.cc:2468] T f91487dd7e0f4c8b8483c2b83945da3b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:50.373562 1160 ts_tablet_manager.cc:1434] T 18b977f463d5420186c007b38110e07e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20251211 20:52:50.374347 1160 tablet_bootstrap.cc:492] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:50.375648 908 raft_consensus.cc:3060] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.377045 710 leader_election.cc:304] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:50.377701 1160 tablet_bootstrap.cc:654] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.378038 3051 raft_consensus.cc:2804] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:50.378178 3051 raft_consensus.cc:697] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:50.378405 3051 consensus_queue.cc:237] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.386842 1160 tablet_bootstrap.cc:492] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:50.387895 1160 ts_tablet_manager.cc:1403] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20251211 20:52:50.387902 3294 tablet_replica.cc:333] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:50.388125 3294 raft_consensus.cc:2243] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:50.388520 3294 raft_consensus.cc:2272] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:50.388981 1160 raft_consensus.cc:359] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.389154 3294 ts_tablet_manager.cc:1916] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:50.390183 1160 raft_consensus.cc:385] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20251211 20:52:50.390446 847 leader_election.cc:343] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: e4b5a7553110424d901788c7a2ecdce6
I20251211 20:52:50.390444 1160 raft_consensus.cc:740] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.390617 847 leader_election.cc:304] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:50.391098 3174 raft_consensus.cc:2749] T e4b5a7553110424d901788c7a2ecdce6 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:50.391940 3295 tablet_replica.cc:333] T 8feee01fc0aa4b32be926ff2b617b894 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:50.391481 1160 consensus_queue.cc:260] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.392238 3295 raft_consensus.cc:2243] T 8feee01fc0aa4b32be926ff2b617b894 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:50.392503 3174 raft_consensus.cc:493] T 47779034ad89466e9e1801987128b4dd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.392675 3174 raft_consensus.cc:515] T 47779034ad89466e9e1801987128b4dd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.392874 3295 raft_consensus.cc:2272] T 8feee01fc0aa4b32be926ff2b617b894 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:50.393252 3174 leader_election.cc:290] T 47779034ad89466e9e1801987128b4dd P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.393626 3295 ts_tablet_manager.cc:1916] T 8feee01fc0aa4b32be926ff2b617b894 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:50.393724 1160 ts_tablet_manager.cc:1434] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251211 20:52:50.393656 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "47779034ad89466e9e1801987128b4dd" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:50.394845 1160 tablet_bootstrap.cc:492] T 71dc8d20d93243e289330d7b8c635511 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:50.396963 2974 raft_consensus.cc:493] T 4388893262b54ad8be1bdf82516031a5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.397106 2974 raft_consensus.cc:515] T 4388893262b54ad8be1bdf82516031a5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.397507 1166 tablet_bootstrap.cc:492] T 993616e226274c229a5924fbe69911af P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:50.397647 1166 ts_tablet_manager.cc:1403] T 993616e226274c229a5924fbe69911af P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.068s user 0.009s sys 0.012s
I20251211 20:52:50.397696 2974 leader_election.cc:290] T 4388893262b54ad8be1bdf82516031a5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:50.398356 1166 raft_consensus.cc:359] T 993616e226274c229a5924fbe69911af P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.398524 1166 raft_consensus.cc:385] T 993616e226274c229a5924fbe69911af P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.398576 1166 raft_consensus.cc:740] T 993616e226274c229a5924fbe69911af P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.398770 1166 consensus_queue.cc:260] T 993616e226274c229a5924fbe69911af P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.399556 1160 tablet_bootstrap.cc:654] T 71dc8d20d93243e289330d7b8c635511 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.399252 1166 ts_tablet_manager.cc:1434] T 993616e226274c229a5924fbe69911af P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:50.400266 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4388893262b54ad8be1bdf82516031a5" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:50.400947 1166 tablet_bootstrap.cc:492] T 66cdc6f297b24cfba3a3a0ae9f45d12d P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:50.401095 982 leader_election.cc:343] T 4388893262b54ad8be1bdf82516031a5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.402350 1166 tablet_bootstrap.cc:654] T 66cdc6f297b24cfba3a3a0ae9f45d12d P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.402467 908 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4388893262b54ad8be1bdf82516031a5" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:50.408354 983 leader_election.cc:343] T 4388893262b54ad8be1bdf82516031a5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.408504 983 leader_election.cc:304] T 4388893262b54ad8be1bdf82516031a5 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:50.408921 2974 raft_consensus.cc:2749] T 4388893262b54ad8be1bdf82516031a5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:50.412230 1160 tablet_bootstrap.cc:492] T 71dc8d20d93243e289330d7b8c635511 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:50.412317 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "47779034ad89466e9e1801987128b4dd" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:50.412524 1160 ts_tablet_manager.cc:1403] T 71dc8d20d93243e289330d7b8c635511 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.001s
I20251211 20:52:50.412983 847 leader_election.cc:304] T 47779034ad89466e9e1801987128b4dd P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:50.413434 3174 raft_consensus.cc:2749] T 47779034ad89466e9e1801987128b4dd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:50.413357 1160 raft_consensus.cc:359] T 71dc8d20d93243e289330d7b8c635511 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.413587 1160 raft_consensus.cc:385] T 71dc8d20d93243e289330d7b8c635511 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.413652 1160 raft_consensus.cc:740] T 71dc8d20d93243e289330d7b8c635511 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.413887 1160 consensus_queue.cc:260] T 71dc8d20d93243e289330d7b8c635511 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.414485 1160 ts_tablet_manager.cc:1434] T 71dc8d20d93243e289330d7b8c635511 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251211 20:52:50.415143 1160 tablet_bootstrap.cc:492] T 373d3f03d7e0408cbe06955f8f60b16c P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:50.416337 3000 leader_election.cc:290] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.417186 1160 tablet_bootstrap.cc:654] T 373d3f03d7e0408cbe06955f8f60b16c P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.419837 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 4776c456b4d74c90b1fc71d57b62a461 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.423028 903 raft_consensus.cc:3060] T 7452dd736b0440fcb4626858e794676f P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.427110 3000 raft_consensus.cc:493] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.427256 3000 raft_consensus.cc:515] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.427781 1160 tablet_bootstrap.cc:492] T 373d3f03d7e0408cbe06955f8f60b16c P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:50.427846 3000 leader_election.cc:290] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.427940 1160 ts_tablet_manager.cc:1403] T 373d3f03d7e0408cbe06955f8f60b16c P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20251211 20:52:50.428473 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3f1840e4b5214ddb878935a79b214eae" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:50.428663 1033 raft_consensus.cc:2468] T 3f1840e4b5214ddb878935a79b214eae P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:50.428766 1160 raft_consensus.cc:359] T 373d3f03d7e0408cbe06955f8f60b16c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.428934 1160 raft_consensus.cc:385] T 373d3f03d7e0408cbe06955f8f60b16c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.429028 1160 raft_consensus.cc:740] T 373d3f03d7e0408cbe06955f8f60b16c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.429114 710 leader_election.cc:304] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:50.429226 1160 consensus_queue.cc:260] T 373d3f03d7e0408cbe06955f8f60b16c P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.429483 3295 ts_tablet_manager.cc:1929] T 8feee01fc0aa4b32be926ff2b617b894 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20251211 20:52:50.429600 3000 raft_consensus.cc:2804] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:50.429634 3295 log.cc:1199] T 8feee01fc0aa4b32be926ff2b617b894 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/8feee01fc0aa4b32be926ff2b617b894
I20251211 20:52:50.429672 3000 raft_consensus.cc:493] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:50.429730 3000 raft_consensus.cc:3060] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.429849 1160 ts_tablet_manager.cc:1434] T 373d3f03d7e0408cbe06955f8f60b16c P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:50.430018 3295 ts_tablet_manager.cc:1950] T 8feee01fc0aa4b32be926ff2b617b894 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:50.430136 1160 tablet_bootstrap.cc:492] T d391d4e6a101462d8d1220f1e17d0daa P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:50.431715 1160 tablet_bootstrap.cc:654] T d391d4e6a101462d8d1220f1e17d0daa P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.432114 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a61cf8f5de14adaa4a1de5bddde9a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.432156 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c5c937cc7db4ed18fe03f4a8a99a66a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.432229 3000 raft_consensus.cc:515] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.432464 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 0f8808bfd68841e4ae30818018040cf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.432683 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 7d4e63f54d8443ae8322d4659a165e76 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.432857 746 tablet_service.cc:1552] Processing DeleteTablet for tablet bb9fccb6981e446a939a28d7fdd38e25 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.433008 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 449921ea74f04a9c88eb9e094e8226c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.433777 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3f1840e4b5214ddb878935a79b214eae" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:50.433961 1033 raft_consensus.cc:3060] T 3f1840e4b5214ddb878935a79b214eae P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.434741 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3f1840e4b5214ddb878935a79b214eae" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:50.435498 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3f1840e4b5214ddb878935a79b214eae" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:50.436241 711 leader_election.cc:343] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.436483 1033 raft_consensus.cc:2468] T 3f1840e4b5214ddb878935a79b214eae P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
W20251211 20:52:50.436666 711 leader_election.cc:343] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.427549 3294 ts_tablet_manager.cc:1929] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20251211 20:52:50.436918 3294 log.cc:1199] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/8feee01fc0aa4b32be926ff2b617b894
W20251211 20:52:50.437055 1810 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 7a299ee03d0e48aba7c4f71e3e0f381a (table default.Table_6402a801d48e4a53827b1b62b6cdaba1 [id=6876565d3a9c48ce841b830cf75139ff], current schema version=3): Not found: failed to reset TS proxy: No leader found: Tablet 7a299ee03d0e48aba7c4f71e3e0f381a consensus state has no leader
I20251211 20:52:50.437451 710 leader_election.cc:304] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:50.437760 3000 leader_election.cc:290] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.437988 3051 raft_consensus.cc:2804] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:50.438126 3051 raft_consensus.cc:697] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:50.438380 3051 consensus_queue.cc:237] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.438522 3294 ts_tablet_manager.cc:1950] T 8feee01fc0aa4b32be926ff2b617b894 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:50.440248 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 9b99860ee6be45a0b358e686332d3137 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.440418 757 tablet_service.cc:1552] Processing DeleteTablet for tablet f22e4d34cde24b37a025c1b5a1046b1b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.440448 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ef524747034491c99f4f9767acd3c9d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.440713 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 0aac6fb3972e4aba8950828c9e49d507 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.440748 742 tablet_service.cc:1552] Processing DeleteTablet for tablet f91487dd7e0f4c8b8483c2b83945da3b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.443154 908 raft_consensus.cc:3060] T b85af79c2574443998e847bc19c53b69 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.443464 3295 tablet_replica.cc:333] T 3fc173c396ff4af5a8589cff18165e85 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:50.444852 3295 raft_consensus.cc:2243] T 3fc173c396ff4af5a8589cff18165e85 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:50.445079 3295 raft_consensus.cc:2272] T 3fc173c396ff4af5a8589cff18165e85 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:50.446107 3294 tablet_replica.cc:333] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:50.446591 3294 raft_consensus.cc:2243] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:50.447412 3295 ts_tablet_manager.cc:1916] T 3fc173c396ff4af5a8589cff18165e85 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:50.447932 3294 raft_consensus.cc:2272] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:50.448889 3294 ts_tablet_manager.cc:1916] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:50.457708 3051 consensus_queue.cc:1048] T 79cc81e91e1846038621af7a0a9f7a2e P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:50.458702 1160 tablet_bootstrap.cc:492] T d391d4e6a101462d8d1220f1e17d0daa P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:50.458909 1160 ts_tablet_manager.cc:1403] T d391d4e6a101462d8d1220f1e17d0daa P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.029s user 0.000s sys 0.002s
I20251211 20:52:50.459791 1160 raft_consensus.cc:359] T d391d4e6a101462d8d1220f1e17d0daa P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.460000 1160 raft_consensus.cc:385] T d391d4e6a101462d8d1220f1e17d0daa P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.460079 1160 raft_consensus.cc:740] T d391d4e6a101462d8d1220f1e17d0daa P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.460336 1160 consensus_queue.cc:260] T d391d4e6a101462d8d1220f1e17d0daa P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.460878 1160 ts_tablet_manager.cc:1434] T d391d4e6a101462d8d1220f1e17d0daa P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:50.461225 1160 tablet_bootstrap.cc:492] T ce1799e6604f4ed7875ddedecab6f327 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:50.463050 1160 tablet_bootstrap.cc:654] T ce1799e6604f4ed7875ddedecab6f327 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.464365 3000 consensus_queue.cc:1048] T 1e13965b6361488eaeb7dd13b467ca65 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:50.465386 904 raft_consensus.cc:3060] T 4a39870c7eea4a379b284ea2e5ddf760 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.469256 3295 ts_tablet_manager.cc:1929] T 3fc173c396ff4af5a8589cff18165e85 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251211 20:52:50.473404 3294 ts_tablet_manager.cc:1929] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:50.483053 3295 log.cc:1199] T 3fc173c396ff4af5a8589cff18165e85 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/3fc173c396ff4af5a8589cff18165e85
I20251211 20:52:50.484642 3174 raft_consensus.cc:493] T 71dc8d20d93243e289330d7b8c635511 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.484849 3174 raft_consensus.cc:515] T 71dc8d20d93243e289330d7b8c635511 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.485131 3295 ts_tablet_manager.cc:1950] T 3fc173c396ff4af5a8589cff18165e85 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:50.485507 3174 leader_election.cc:290] T 71dc8d20d93243e289330d7b8c635511 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.485913 1160 tablet_bootstrap.cc:492] T ce1799e6604f4ed7875ddedecab6f327 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:50.486061 1160 ts_tablet_manager.cc:1403] T ce1799e6604f4ed7875ddedecab6f327 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.025s user 0.007s sys 0.002s
I20251211 20:52:50.486282 3294 log.cc:1199] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/6029c1fd563d4023a8cf7cfb8ffa0876
I20251211 20:52:50.486452 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71dc8d20d93243e289330d7b8c635511" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:50.486797 1160 raft_consensus.cc:359] T ce1799e6604f4ed7875ddedecab6f327 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.486979 1160 raft_consensus.cc:385] T ce1799e6604f4ed7875ddedecab6f327 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.487071 1160 raft_consensus.cc:740] T ce1799e6604f4ed7875ddedecab6f327 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.487305 1160 consensus_queue.cc:260] T ce1799e6604f4ed7875ddedecab6f327 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.487490 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71dc8d20d93243e289330d7b8c635511" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:50.488095 3294 ts_tablet_manager.cc:1950] T 6029c1fd563d4023a8cf7cfb8ffa0876 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:50.488086 847 leader_election.cc:304] T 71dc8d20d93243e289330d7b8c635511 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:50.488406 1160 ts_tablet_manager.cc:1434] T ce1799e6604f4ed7875ddedecab6f327 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:50.488597 3174 raft_consensus.cc:2749] T 71dc8d20d93243e289330d7b8c635511 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:50.488713 1160 tablet_bootstrap.cc:492] T 92efda2bd0c84d07a0af6b537c0cd882 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:50.490711 2959 consensus_queue.cc:1048] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:50.491321 3294 tablet_replica.cc:333] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:50.491518 3294 raft_consensus.cc:2243] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:50.491825 3294 raft_consensus.cc:2272] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:50.492611 1160 tablet_bootstrap.cc:654] T 92efda2bd0c84d07a0af6b537c0cd882 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.494727 3294 ts_tablet_manager.cc:1916] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:50.502590 711 consensus_peers.cc:597] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:50.506443 3155 consensus_queue.cc:1048] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:50.508203 3110 tablet.cc:1722] T b85af79c2574443998e847bc19c53b69 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:50.517350 3294 ts_tablet_manager.cc:1929] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251211 20:52:50.520727 3294 log.cc:1199] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/3fc173c396ff4af5a8589cff18165e85
I20251211 20:52:50.521332 3294 ts_tablet_manager.cc:1950] T 3fc173c396ff4af5a8589cff18165e85 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
W20251211 20:52:50.524668 711 consensus_peers.cc:597] T be876471f8bf453aa4467c91891cbbde P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:50.525104 3294 tablet_replica.cc:333] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:50.525527 3294 raft_consensus.cc:2243] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:50.528684 3294 raft_consensus.cc:2272] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:50.531308 3294 ts_tablet_manager.cc:1916] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:50.536530 1160 tablet_bootstrap.cc:492] T 92efda2bd0c84d07a0af6b537c0cd882 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:50.536720 1160 ts_tablet_manager.cc:1403] T 92efda2bd0c84d07a0af6b537c0cd882 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.048s user 0.007s sys 0.011s
I20251211 20:52:50.537425 1160 raft_consensus.cc:359] T 92efda2bd0c84d07a0af6b537c0cd882 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.537618 1160 raft_consensus.cc:385] T 92efda2bd0c84d07a0af6b537c0cd882 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.537673 1160 raft_consensus.cc:740] T 92efda2bd0c84d07a0af6b537c0cd882 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.537927 1160 consensus_queue.cc:260] T 92efda2bd0c84d07a0af6b537c0cd882 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.538691 1160 ts_tablet_manager.cc:1434] T 92efda2bd0c84d07a0af6b537c0cd882 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:50.539248 1160 tablet_bootstrap.cc:492] T e1aeae8c18b6499e9dc1accabdc3f633 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:50.540812 1160 tablet_bootstrap.cc:654] T e1aeae8c18b6499e9dc1accabdc3f633 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.548091 1166 tablet_bootstrap.cc:492] T 66cdc6f297b24cfba3a3a0ae9f45d12d P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:50.548348 1166 ts_tablet_manager.cc:1403] T 66cdc6f297b24cfba3a3a0ae9f45d12d P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.148s user 0.010s sys 0.022s
I20251211 20:52:50.549238 1166 raft_consensus.cc:359] T 66cdc6f297b24cfba3a3a0ae9f45d12d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.549500 1166 raft_consensus.cc:385] T 66cdc6f297b24cfba3a3a0ae9f45d12d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.549607 1166 raft_consensus.cc:740] T 66cdc6f297b24cfba3a3a0ae9f45d12d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.550022 1166 consensus_queue.cc:260] T 66cdc6f297b24cfba3a3a0ae9f45d12d P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.551332 1166 ts_tablet_manager.cc:1434] T 66cdc6f297b24cfba3a3a0ae9f45d12d P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:50.552285 1166 tablet_bootstrap.cc:492] T c549c2bfebd64b87a4fc8ce8210e14f6 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:50.553853 1166 tablet_bootstrap.cc:654] T c549c2bfebd64b87a4fc8ce8210e14f6 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.557376 1160 tablet_bootstrap.cc:492] T e1aeae8c18b6499e9dc1accabdc3f633 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:50.557526 1160 ts_tablet_manager.cc:1403] T e1aeae8c18b6499e9dc1accabdc3f633 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.004s
I20251211 20:52:50.558171 1160 raft_consensus.cc:359] T e1aeae8c18b6499e9dc1accabdc3f633 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.558318 1160 raft_consensus.cc:385] T e1aeae8c18b6499e9dc1accabdc3f633 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.558374 1160 raft_consensus.cc:740] T e1aeae8c18b6499e9dc1accabdc3f633 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.558552 1160 consensus_queue.cc:260] T e1aeae8c18b6499e9dc1accabdc3f633 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.559307 1160 ts_tablet_manager.cc:1434] T e1aeae8c18b6499e9dc1accabdc3f633 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:50.559640 1160 tablet_bootstrap.cc:492] T d69fb1f6d9594afcbfb844860b41a377 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:50.561036 1160 tablet_bootstrap.cc:654] T d69fb1f6d9594afcbfb844860b41a377 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.564561 824 heartbeater.cc:499] Master 127.0.104.61:40749 was elected leader, sending a full tablet report...
I20251211 20:52:50.567080 3294 ts_tablet_manager.cc:1929] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:50.567308 3294 log.cc:1199] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/6a61cf8f5de14adaa4a1de5bddde9a2f
I20251211 20:52:50.567768 3294 ts_tablet_manager.cc:1950] T 6a61cf8f5de14adaa4a1de5bddde9a2f P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:50.574721 961 heartbeater.cc:499] Master 127.0.104.61:40749 was elected leader, sending a full tablet report...
I20251211 20:52:50.576799 3294 tablet_replica.cc:333] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:50.577101 3294 raft_consensus.cc:2243] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:50.577267 3174 raft_consensus.cc:493] T 1578887b2f5748cd83a6fbdec2f3fc99 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.577391 3174 raft_consensus.cc:515] T 1578887b2f5748cd83a6fbdec2f3fc99 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.577472 3294 raft_consensus.cc:2272] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:50.577963 3174 leader_election.cc:290] T 1578887b2f5748cd83a6fbdec2f3fc99 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.578027 3294 ts_tablet_manager.cc:1916] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:50.579324 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1578887b2f5748cd83a6fbdec2f3fc99" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:50.583832 2974 raft_consensus.cc:493] T 683d0f2068fb4a6ebc9c169ad7b455ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.583987 2974 raft_consensus.cc:515] T 683d0f2068fb4a6ebc9c169ad7b455ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.584565 2974 leader_election.cc:290] T 683d0f2068fb4a6ebc9c169ad7b455ce P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:50.585263 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "683d0f2068fb4a6ebc9c169ad7b455ce" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:50.372812 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 138) took 1307 ms. Trace:
W20251211 20:52:50.586030 982 leader_election.cc:343] T 683d0f2068fb4a6ebc9c169ad7b455ce P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.586174 3174 raft_consensus.cc:493] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.586295 3174 raft_consensus.cc:515] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.586848 3174 leader_election.cc:290] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.586043 2962 rpcz_store.cc:276] 1211 20:52:49.065142 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.065179 (+ 37us) service_pool.cc:225] Handling call
1211 20:52:50.344153 (+1278974us) ts_tablet_manager.cc:1151] Deleting tablet 3621735fcb294e3a8c0180ba9ef86442
1211 20:52:50.344163 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:50.352034 (+ 7871us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:50.355058 (+ 3024us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:50.372787 (+ 17729us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1278833}
W20251211 20:52:50.587769 847 leader_election.cc:343] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 4201da4241aa48dfb670a713e4453ca5
I20251211 20:52:50.588035 2962 tablet_replica.cc:333] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:50.588277 2962 raft_consensus.cc:2243] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:50.588511 2962 raft_consensus.cc:2272] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:50.589331 2962 ts_tablet_manager.cc:1916] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:50.594907 2959 consensus_queue.cc:1048] T 5df87ea7d28340da84ef141c92efc710 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:50.601598 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "683d0f2068fb4a6ebc9c169ad7b455ce" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:50.602272 983 leader_election.cc:343] T 683d0f2068fb4a6ebc9c169ad7b455ce P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.602396 983 leader_election.cc:304] T 683d0f2068fb4a6ebc9c169ad7b455ce P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:50.602829 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1578887b2f5748cd83a6fbdec2f3fc99" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:50.603502 2974 raft_consensus.cc:2749] T 683d0f2068fb4a6ebc9c169ad7b455ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:50.603965 1783 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 8feee01fc0aa4b32be926ff2b617b894 (table default.Table_e6f3a819faad48aabfad3a235c3c1b95 [id=18f8d722faaa4293a112525ff0c96216]) successfully deleted
W20251211 20:52:50.604162 1783 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 164592us
I20251211 20:52:50.605430 847 leader_election.cc:304] T 1578887b2f5748cd83a6fbdec2f3fc99 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
W20251211 20:52:50.605567 711 consensus_peers.cc:597] T 01104975fd8b470aa026578b97ff6b8d P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:50.605822 847 leader_election.cc:343] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 4201da4241aa48dfb670a713e4453ca5
I20251211 20:52:50.605926 847 leader_election.cc:304] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:50.606495 2966 raft_consensus.cc:2749] T 1578887b2f5748cd83a6fbdec2f3fc99 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:50.606642 2966 raft_consensus.cc:2749] T 4201da4241aa48dfb670a713e4453ca5 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:50.608855 1160 tablet_bootstrap.cc:492] T d69fb1f6d9594afcbfb844860b41a377 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:50.608995 1160 ts_tablet_manager.cc:1403] T d69fb1f6d9594afcbfb844860b41a377 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.049s user 0.003s sys 0.011s
I20251211 20:52:50.609745 1160 raft_consensus.cc:359] T d69fb1f6d9594afcbfb844860b41a377 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.610271 1160 raft_consensus.cc:385] T d69fb1f6d9594afcbfb844860b41a377 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.610323 1160 raft_consensus.cc:740] T d69fb1f6d9594afcbfb844860b41a377 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.610486 1160 consensus_queue.cc:260] T d69fb1f6d9594afcbfb844860b41a377 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.610965 1160 ts_tablet_manager.cc:1434] T d69fb1f6d9594afcbfb844860b41a377 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:50.611217 1160 tablet_bootstrap.cc:492] T d43210ec61af423ebde624397dea9f2e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:50.611817 1783 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 7a299ee03d0e48aba7c4f71e3e0f381a (table default.Table_6402a801d48e4a53827b1b62b6cdaba1 [id=6876565d3a9c48ce841b830cf75139ff], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 7a299ee03d0e48aba7c4f71e3e0f381a consensus state has no leader
I20251211 20:52:50.612142 1783 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 3fc173c396ff4af5a8589cff18165e85 (table default.Table_5e5286ea174048f8ba9abd14daf14bcf [id=a455f6ef64ae44f78972d14b2ea9654c]) successfully deleted
I20251211 20:52:50.612676 1160 tablet_bootstrap.cc:654] T d43210ec61af423ebde624397dea9f2e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.612834 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 8feee01fc0aa4b32be926ff2b617b894 (table default.Table_e6f3a819faad48aabfad3a235c3c1b95 [id=18f8d722faaa4293a112525ff0c96216]) successfully deleted
W20251211 20:52:50.615236 1780 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.104.1:42177, user_credentials={real_user=slave}} blocked reactor thread for 168538us
W20251211 20:52:50.615751 711 consensus_peers.cc:597] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:50.616071 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 3621735fcb294e3a8c0180ba9ef86442 (table default.Table_01868fec1884449da230c4236ab2d506 [id=d303c5c8fd81400ab473845a942ec140]) successfully deleted
I20251211 20:52:50.616309 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 6029c1fd563d4023a8cf7cfb8ffa0876 (table default.Table_0ec526100c24487b808171d9734bcfb7 [id=8f7b8c70bd6f42b8817de464b5b7aac3]) successfully deleted
I20251211 20:52:50.616359 1168 tablet_bootstrap.cc:492] T 51d12723f5194afb97d2c05fcbaf3776 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:50.616478 1168 ts_tablet_manager.cc:1403] T 51d12723f5194afb97d2c05fcbaf3776 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.323s user 0.005s sys 0.002s
W20251211 20:52:50.616488 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 516f7181aaac48cdadf9fbc08d0d3f57 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 516f7181aaac48cdadf9fbc08d0d3f57 already in progress: creating tablet
W20251211 20:52:50.616650 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet d0673c37c2fd47eea8ab10d9e91b8d7a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d0673c37c2fd47eea8ab10d9e91b8d7a already in progress: creating tablet
I20251211 20:52:50.616776 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 3fc173c396ff4af5a8589cff18165e85 (table default.Table_5e5286ea174048f8ba9abd14daf14bcf [id=a455f6ef64ae44f78972d14b2ea9654c]) successfully deleted
W20251211 20:52:50.616981 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 4776c456b4d74c90b1fc71d57b62a461 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4776c456b4d74c90b1fc71d57b62a461 already in progress: creating tablet
I20251211 20:52:50.617110 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 6a61cf8f5de14adaa4a1de5bddde9a2f (table default.Table_b06f9587f78645bb9712c3b080f3c0d4 [id=ea12bdaa894f446da22635f95d97ac2a]) successfully deleted
W20251211 20:52:50.617274 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 9c5c937cc7db4ed18fe03f4a8a99a66a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9c5c937cc7db4ed18fe03f4a8a99a66a already in progress: creating tablet
I20251211 20:52:50.617285 1168 raft_consensus.cc:359] T 51d12723f5194afb97d2c05fcbaf3776 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:50.617419 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 0f8808bfd68841e4ae30818018040cf9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0f8808bfd68841e4ae30818018040cf9 already in progress: creating tablet
I20251211 20:52:50.617442 1168 raft_consensus.cc:385] T 51d12723f5194afb97d2c05fcbaf3776 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20251211 20:52:50.617574 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 7d4e63f54d8443ae8322d4659a165e76 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d4e63f54d8443ae8322d4659a165e76 already in progress: creating tablet
I20251211 20:52:50.617638 1168 raft_consensus.cc:740] T 51d12723f5194afb97d2c05fcbaf3776 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
W20251211 20:52:50.617717 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet bb9fccb6981e446a939a28d7fdd38e25 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb9fccb6981e446a939a28d7fdd38e25 already in progress: creating tablet
W20251211 20:52:50.617888 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 449921ea74f04a9c88eb9e094e8226c8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 449921ea74f04a9c88eb9e094e8226c8 already in progress: creating tablet
I20251211 20:52:50.617846 1168 consensus_queue.cc:260] T 51d12723f5194afb97d2c05fcbaf3776 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:50.618029 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 9b99860ee6be45a0b358e686332d3137 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9b99860ee6be45a0b358e686332d3137 already in progress: creating tablet
I20251211 20:52:50.618261 1168 ts_tablet_manager.cc:1434] T 51d12723f5194afb97d2c05fcbaf3776 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:50.618906 1168 tablet_bootstrap.cc:492] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:50.619031 903 raft_consensus.cc:3060] T e1aeae8c18b6499e9dc1accabdc3f633 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.620363 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 68792ccef54944a09af9b5456e9ff1fe with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.620574 751 tablet_service.cc:1552] Processing DeleteTablet for tablet ba120d28abc54553ae15532de90db719 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.620651 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 0add14ccd2af42aeb02fad7a28ed4520 with delete_type TABLET_DATA_DELETED (replaced by tablet 84cca3ee8fc446fcaf9971171235cf55) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.620784 738 tablet_service.cc:1552] Processing DeleteTablet for tablet 3dde7ed495e64909bd9784dc5eefb04c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.620819 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 745ce63314834c8abdecaa367133a068 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.620967 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f7b1bc3dd1f4496b3ac6e71652cf012 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.620591 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 76cfb26762d54593a150ec98125c6592 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.625914 1160 tablet_bootstrap.cc:492] T d43210ec61af423ebde624397dea9f2e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:50.626065 1160 ts_tablet_manager.cc:1403] T d43210ec61af423ebde624397dea9f2e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.000s
I20251211 20:52:50.626801 1160 raft_consensus.cc:359] T d43210ec61af423ebde624397dea9f2e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.626981 1160 raft_consensus.cc:385] T d43210ec61af423ebde624397dea9f2e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.627038 1160 raft_consensus.cc:740] T d43210ec61af423ebde624397dea9f2e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.627254 1160 consensus_queue.cc:260] T d43210ec61af423ebde624397dea9f2e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.627875 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 5ddf7c1972564ef88e3c0a0bd1b430e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.628029 1160 ts_tablet_manager.cc:1434] T d43210ec61af423ebde624397dea9f2e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:50.628062 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 18b977f463d5420186c007b38110e07e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.628310 1160 tablet_bootstrap.cc:492] T f25bde14792248b188660036f07ae902 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:50.630053 1160 tablet_bootstrap.cc:654] T f25bde14792248b188660036f07ae902 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.632656 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 042f8bf13a7a4712b5bdc04671cd9938 with delete_type TABLET_DATA_DELETED (replaced by tablet 53b4116db0a44dfbb4c1ec5b2acebc67) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.620801 1168 tablet_bootstrap.cc:654] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.641407 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 7d4bba0aa7ea403e8dfcb74cec2e1f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.641898 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 6ad6ab53aad14cc2b6d201acfecf6723 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.642083 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 2e33606930774fe4aafc6e3819618af8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.642210 756 tablet_service.cc:1552] Processing DeleteTablet for tablet e9ead34065e240eeb1f946319537d32a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.642340 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 3621735fcb294e3a8c0180ba9ef86442 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.642665 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 84ed434868aa437a83455b1605deb26c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.643006 757 tablet_service.cc:1552] Processing DeleteTablet for tablet eb9da081cb9d43fd9a35c4bc51ea133e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.643306 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.643615 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 3562b5effa124378beac7ccf0d27ee7e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.648797 1160 tablet_bootstrap.cc:492] T f25bde14792248b188660036f07ae902 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:50.648958 1160 ts_tablet_manager.cc:1403] T f25bde14792248b188660036f07ae902 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.002s
I20251211 20:52:50.649726 1160 raft_consensus.cc:359] T f25bde14792248b188660036f07ae902 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.649922 1160 raft_consensus.cc:385] T f25bde14792248b188660036f07ae902 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.649969 1160 raft_consensus.cc:740] T f25bde14792248b188660036f07ae902 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.650162 1160 consensus_queue.cc:260] T f25bde14792248b188660036f07ae902 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.650627 1160 ts_tablet_manager.cc:1434] T f25bde14792248b188660036f07ae902 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:50.652616 1160 tablet_bootstrap.cc:492] T 5bcc94ec9630484cb7a3e8bf6d4f378a P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:50.654361 1160 tablet_bootstrap.cc:654] T 5bcc94ec9630484cb7a3e8bf6d4f378a P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:50.656551 3321 log.cc:927] Time spent T 5df87ea7d28340da84ef141c92efc710 P e715a811a82446ce8d5bb4aafd5b81c7: Append to log took a long time: real 0.051s user 0.000s sys 0.000s
I20251211 20:52:50.657809 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 06b8f8e46e524545aeab6c931efaf116 with delete_type TABLET_DATA_DELETED (replaced by tablet 7bc03ed8c35740f58fe48ddbaa120ea0) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.659929 3000 consensus_queue.cc:1048] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:50.663209 1166 tablet_bootstrap.cc:492] T c549c2bfebd64b87a4fc8ce8210e14f6 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:50.663362 1166 ts_tablet_manager.cc:1403] T c549c2bfebd64b87a4fc8ce8210e14f6 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.112s user 0.002s sys 0.006s
I20251211 20:52:50.664537 1166 raft_consensus.cc:359] T c549c2bfebd64b87a4fc8ce8210e14f6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.664716 1166 raft_consensus.cc:385] T c549c2bfebd64b87a4fc8ce8210e14f6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.664769 1166 raft_consensus.cc:740] T c549c2bfebd64b87a4fc8ce8210e14f6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.665539 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 47779034ad89466e9e1801987128b4dd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:56136
W20251211 20:52:50.668349 711 consensus_peers.cc:597] T 04679dfddf564d6f8c16d1c257e29b41 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:50.669059 1160 tablet_bootstrap.cc:492] T 5bcc94ec9630484cb7a3e8bf6d4f378a P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:50.669202 1160 ts_tablet_manager.cc:1403] T 5bcc94ec9630484cb7a3e8bf6d4f378a P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.003s
I20251211 20:52:50.669992 1160 raft_consensus.cc:359] T 5bcc94ec9630484cb7a3e8bf6d4f378a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.670181 1160 raft_consensus.cc:385] T 5bcc94ec9630484cb7a3e8bf6d4f378a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.670233 1160 raft_consensus.cc:740] T 5bcc94ec9630484cb7a3e8bf6d4f378a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.670415 1160 consensus_queue.cc:260] T 5bcc94ec9630484cb7a3e8bf6d4f378a P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.670894 1160 ts_tablet_manager.cc:1434] T 5bcc94ec9630484cb7a3e8bf6d4f378a P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:50.672606 1160 tablet_bootstrap.cc:492] T 95866de91f9d437d80f40b7962253610 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:50.674312 1160 tablet_bootstrap.cc:654] T 95866de91f9d437d80f40b7962253610 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.678006 757 tablet_service.cc:1552] Processing DeleteTablet for tablet cce44b80c97948e1af7ac68e77cccfd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.680446 3051 raft_consensus.cc:493] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.680619 3051 raft_consensus.cc:515] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.681291 3051 leader_election.cc:290] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.682231 1036 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f9a3fdf17f274c15ac0bdfa9c174f649" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:50.682477 1036 raft_consensus.cc:2468] T f9a3fdf17f274c15ac0bdfa9c174f649 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:50.683115 710 leader_election.cc:304] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:50.683597 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 80c566822fa94ada983aa7266df4bf9d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.683884 3051 raft_consensus.cc:2804] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:50.683960 3051 raft_consensus.cc:493] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:50.684029 3051 raft_consensus.cc:3060] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.687419 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 516f7181aaac48cdadf9fbc08d0d3f57 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.687757 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a61cf8f5de14adaa4a1de5bddde9a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.688233 3295 tablet_replica.cc:333] T 6a61cf8f5de14adaa4a1de5bddde9a2f P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:50.688427 3295 raft_consensus.cc:2243] T 6a61cf8f5de14adaa4a1de5bddde9a2f P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:50.688596 3295 raft_consensus.cc:2272] T 6a61cf8f5de14adaa4a1de5bddde9a2f P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:50.689182 3295 ts_tablet_manager.cc:1916] T 6a61cf8f5de14adaa4a1de5bddde9a2f P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:50.690230 903 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f9a3fdf17f274c15ac0bdfa9c174f649" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:50.691514 2966 raft_consensus.cc:493] T 44e2d2bdecf247baaac26e06bccc94de P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.691664 2966 raft_consensus.cc:515] T 44e2d2bdecf247baaac26e06bccc94de P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.691745 2974 raft_consensus.cc:493] T 993616e226274c229a5924fbe69911af P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.691877 2974 raft_consensus.cc:515] T 993616e226274c229a5924fbe69911af P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.692222 2966 leader_election.cc:290] T 44e2d2bdecf247baaac26e06bccc94de P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.692430 2974 leader_election.cc:290] T 993616e226274c229a5924fbe69911af P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:50.693455 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 0f8808bfd68841e4ae30818018040cf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.693660 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c5c937cc7db4ed18fe03f4a8a99a66a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.693827 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 7d4e63f54d8443ae8322d4659a165e76 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.695291 3051 raft_consensus.cc:515] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.695873 1036 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "44e2d2bdecf247baaac26e06bccc94de" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:50.697714 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f9a3fdf17f274c15ac0bdfa9c174f649" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:50.697767 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "44e2d2bdecf247baaac26e06bccc94de" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:50.697993 1036 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f9a3fdf17f274c15ac0bdfa9c174f649" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:50.698154 1036 raft_consensus.cc:3060] T f9a3fdf17f274c15ac0bdfa9c174f649 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:50.699478 711 leader_election.cc:343] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:50.699839 711 consensus_peers.cc:597] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:50.700008 711 leader_election.cc:343] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.700307 847 leader_election.cc:304] T 44e2d2bdecf247baaac26e06bccc94de P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:50.700536 903 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "993616e226274c229a5924fbe69911af" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:50.700768 1036 raft_consensus.cc:2468] T f9a3fdf17f274c15ac0bdfa9c174f649 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:50.700913 3174 raft_consensus.cc:2749] T 44e2d2bdecf247baaac26e06bccc94de P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:50.700966 983 leader_election.cc:343] T 993616e226274c229a5924fbe69911af P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.701176 904 raft_consensus.cc:3060] T 46cb1d16a0e94b3a833f32c635213fd2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.701417 710 leader_election.cc:304] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:50.701679 3000 raft_consensus.cc:2804] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:50.701782 3000 raft_consensus.cc:697] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:50.702006 3000 consensus_queue.cc:237] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.703867 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "993616e226274c229a5924fbe69911af" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:50.704118 2429 raft_consensus.cc:493] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.704232 2429 raft_consensus.cc:515] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.704730 2429 leader_election.cc:290] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:50.706456 982 leader_election.cc:343] T 993616e226274c229a5924fbe69911af P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.706575 982 leader_election.cc:304] T 993616e226274c229a5924fbe69911af P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:50.707036 3269 raft_consensus.cc:2749] T 993616e226274c229a5924fbe69911af P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:50.708380 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4fe5c534b572491c911aae01eaa79bae" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:50.708874 711 consensus_peers.cc:597] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:50.709054 711 leader_election.cc:343] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.711150 1036 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4fe5c534b572491c911aae01eaa79bae" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:50.711333 1036 raft_consensus.cc:2468] T 4fe5c534b572491c911aae01eaa79bae P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:50.711726 710 leader_election.cc:304] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:50.712239 2429 raft_consensus.cc:2804] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:50.712352 2429 raft_consensus.cc:493] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:50.712414 2429 raft_consensus.cc:3060] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.714805 2429 raft_consensus.cc:515] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.715440 2429 leader_election.cc:290] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.716395 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4fe5c534b572491c911aae01eaa79bae" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:50.716584 1048 raft_consensus.cc:3060] T 4fe5c534b572491c911aae01eaa79bae P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.718262 3174 raft_consensus.cc:493] T 5bcc94ec9630484cb7a3e8bf6d4f378a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.718400 3174 raft_consensus.cc:515] T 5bcc94ec9630484cb7a3e8bf6d4f378a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.718982 3174 leader_election.cc:290] T 5bcc94ec9630484cb7a3e8bf6d4f378a P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.719182 3051 leader_election.cc:290] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.719240 1048 raft_consensus.cc:2468] T 4fe5c534b572491c911aae01eaa79bae P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:50.719609 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4fe5c534b572491c911aae01eaa79bae" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:50.719718 710 leader_election.cc:304] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:50.720081 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5bcc94ec9630484cb7a3e8bf6d4f378a" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:50.720383 2429 raft_consensus.cc:2804] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:50.720619 2429 raft_consensus.cc:697] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:50.720907 2429 consensus_queue.cc:237] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.721814 2970 raft_consensus.cc:493] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.721938 2970 raft_consensus.cc:515] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.722435 2970 leader_election.cc:290] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.723407 3174 raft_consensus.cc:493] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6f8d784fd1cd492bbe3809403ec34919)
I20251211 20:52:50.723469 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "969a8587019c4202b6e705d126f83d44" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:50.723547 3174 raft_consensus.cc:515] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.723714 1048 raft_consensus.cc:2468] T 969a8587019c4202b6e705d126f83d44 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:50.724083 710 leader_election.cc:304] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:50.724112 3174 leader_election.cc:290] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.724416 2959 raft_consensus.cc:2804] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:50.724537 2959 raft_consensus.cc:493] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:50.724654 2959 raft_consensus.cc:3060] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:50.725013 847 leader_election.cc:343] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: e6ba12f94f5f4d688f4222a4c2fb5436
W20251211 20:52:50.720359 711 leader_election.cc:343] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.725564 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5bcc94ec9630484cb7a3e8bf6d4f378a" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:50.726255 847 leader_election.cc:343] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: e6ba12f94f5f4d688f4222a4c2fb5436
I20251211 20:52:50.726359 847 leader_election.cc:304] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:50.726763 3174 raft_consensus.cc:2749] T e6ba12f94f5f4d688f4222a4c2fb5436 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:52:50.726845 880 tablet_service.cc:1552] Processing DeleteTablet for tablet bb9fccb6981e446a939a28d7fdd38e25 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.727504 847 leader_election.cc:304] T 5bcc94ec9630484cb7a3e8bf6d4f378a P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:50.727773 3051 consensus_queue.cc:1048] T 98d575ac680d49d0990877666638f013 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:50.727893 3174 raft_consensus.cc:2749] T 5bcc94ec9630484cb7a3e8bf6d4f378a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:50.728228 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 516f7181aaac48cdadf9fbc08d0d3f57 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 516f7181aaac48cdadf9fbc08d0d3f57 already in progress: creating tablet
I20251211 20:52:50.728264 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "969a8587019c4202b6e705d126f83d44" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:50.731835 711 leader_election.cc:343] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.734438 3051 raft_consensus.cc:493] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.735133 3051 raft_consensus.cc:515] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.737349 3174 raft_consensus.cc:493] T 373d3f03d7e0408cbe06955f8f60b16c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.737514 3174 raft_consensus.cc:515] T 373d3f03d7e0408cbe06955f8f60b16c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.737633 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "834b84a557b04e78b5d2c8c84520a644" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:50.737823 1048 raft_consensus.cc:2468] T 834b84a557b04e78b5d2c8c84520a644 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:50.738102 3174 leader_election.cc:290] T 373d3f03d7e0408cbe06955f8f60b16c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.738196 710 leader_election.cc:304] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:50.738406 3051 leader_election.cc:290] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.738508 3051 raft_consensus.cc:2804] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:50.738554 3051 raft_consensus.cc:493] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:50.738600 3051 raft_consensus.cc:3060] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:50.739045 847 leader_election.cc:343] T 373d3f03d7e0408cbe06955f8f60b16c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 373d3f03d7e0408cbe06955f8f60b16c
I20251211 20:52:50.741784 3051 raft_consensus.cc:515] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.742884 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "834b84a557b04e78b5d2c8c84520a644" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:50.743053 1048 raft_consensus.cc:3060] T 834b84a557b04e78b5d2c8c84520a644 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.745741 1048 raft_consensus.cc:2468] T 834b84a557b04e78b5d2c8c84520a644 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:50.746362 710 leader_election.cc:304] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:50.746613 2970 raft_consensus.cc:2804] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:50.746997 2970 raft_consensus.cc:697] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:50.747265 2970 consensus_queue.cc:237] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.748183 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "834b84a557b04e78b5d2c8c84520a644" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:50.748427 903 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "834b84a557b04e78b5d2c8c84520a644" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:50.749184 711 leader_election.cc:343] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:50.749521 711 leader_election.cc:343] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:50.749807 847 leader_election.cc:343] T 373d3f03d7e0408cbe06955f8f60b16c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 373d3f03d7e0408cbe06955f8f60b16c
I20251211 20:52:50.749924 847 leader_election.cc:304] T 373d3f03d7e0408cbe06955f8f60b16c P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:50.750159 3174 raft_consensus.cc:2749] T 373d3f03d7e0408cbe06955f8f60b16c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:50.751784 3051 leader_election.cc:290] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.754218 3110 tablet.cc:1722] T 46cb1d16a0e94b3a833f32c635213fd2 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:50.754411 2959 raft_consensus.cc:515] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.755024 2959 leader_election.cc:290] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.756455 3295 ts_tablet_manager.cc:1929] T 6a61cf8f5de14adaa4a1de5bddde9a2f P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:50.756567 3295 log.cc:1199] T 6a61cf8f5de14adaa4a1de5bddde9a2f P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/6a61cf8f5de14adaa4a1de5bddde9a2f
I20251211 20:52:50.756572 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "969a8587019c4202b6e705d126f83d44" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:50.756739 1048 raft_consensus.cc:3060] T 969a8587019c4202b6e705d126f83d44 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.756918 3295 ts_tablet_manager.cc:1950] T 6a61cf8f5de14adaa4a1de5bddde9a2f P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:50.759938 1048 raft_consensus.cc:2468] T 969a8587019c4202b6e705d126f83d44 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:50.760147 1166 consensus_queue.cc:260] T c549c2bfebd64b87a4fc8ce8210e14f6 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.760370 710 leader_election.cc:304] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:50.760859 3155 raft_consensus.cc:2804] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:50.760975 3155 raft_consensus.cc:697] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:50.761205 3155 consensus_queue.cc:237] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.761795 1166 ts_tablet_manager.cc:1434] T c549c2bfebd64b87a4fc8ce8210e14f6 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.098s user 0.001s sys 0.001s
I20251211 20:52:50.762528 3269 raft_consensus.cc:493] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.762668 3269 raft_consensus.cc:515] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.763236 3269 leader_election.cc:290] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:50.763828 903 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7a0a20f95416425489af94408ac1e97c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:50.764165 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7a0a20f95416425489af94408ac1e97c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:50.764353 775 raft_consensus.cc:2468] T 7a0a20f95416425489af94408ac1e97c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 0.
W20251211 20:52:50.764801 983 leader_election.cc:343] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.765214 982 leader_election.cc:304] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:50.765664 3269 raft_consensus.cc:2804] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:50.765743 3269 raft_consensus.cc:493] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:50.765800 3269 raft_consensus.cc:3060] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.768577 3269 raft_consensus.cc:515] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.769732 903 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "969a8587019c4202b6e705d126f83d44" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:50.770234 3269 leader_election.cc:290] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:50.770416 903 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7a0a20f95416425489af94408ac1e97c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:50.770805 983 leader_election.cc:343] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.771399 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7a0a20f95416425489af94408ac1e97c" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:50.771615 775 raft_consensus.cc:3060] T 7a0a20f95416425489af94408ac1e97c P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.774381 775 raft_consensus.cc:2468] T 7a0a20f95416425489af94408ac1e97c P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 1.
I20251211 20:52:50.779292 982 leader_election.cc:304] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:50.780510 3269 raft_consensus.cc:2804] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:50.780983 3269 raft_consensus.cc:697] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 LEADER]: Becoming Leader. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Running, Role: LEADER
I20251211 20:52:50.781361 3269 consensus_queue.cc:237] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.790913 1166 tablet_bootstrap.cc:492] T 12aaf618d6474f599458026308d7afdd P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:50.795497 757 tablet_service.cc:1552] Processing DeleteTablet for tablet bb4004212441473483a209d23c0800ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.796984 1783 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 6a61cf8f5de14adaa4a1de5bddde9a2f (table default.Table_b06f9587f78645bb9712c3b080f3c0d4 [id=ea12bdaa894f446da22635f95d97ac2a]) successfully deleted
W20251211 20:52:50.802852 1783 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 43779.6us
W20251211 20:52:50.803242 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 0f8808bfd68841e4ae30818018040cf9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0f8808bfd68841e4ae30818018040cf9 already in progress: creating tablet
I20251211 20:52:50.804525 757 tablet_service.cc:1552] Processing DeleteTablet for tablet c97d410272a947de8b5d7fb40fe4777d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:56136
W20251211 20:52:50.805392 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 9c5c937cc7db4ed18fe03f4a8a99a66a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9c5c937cc7db4ed18fe03f4a8a99a66a already in progress: creating tablet
W20251211 20:52:50.805645 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 7d4e63f54d8443ae8322d4659a165e76 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d4e63f54d8443ae8322d4659a165e76 already in progress: creating tablet
I20251211 20:52:50.806011 2974 raft_consensus.cc:493] T 66cdc6f297b24cfba3a3a0ae9f45d12d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.806182 2974 raft_consensus.cc:515] T 66cdc6f297b24cfba3a3a0ae9f45d12d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.806785 2974 leader_election.cc:290] T 66cdc6f297b24cfba3a3a0ae9f45d12d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:50.807261 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "66cdc6f297b24cfba3a3a0ae9f45d12d" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:50.807744 983 leader_election.cc:343] T 66cdc6f297b24cfba3a3a0ae9f45d12d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.809088 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "66cdc6f297b24cfba3a3a0ae9f45d12d" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:50.809904 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 59e955aa07c14fbbb347b88eca59a74e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:56136
W20251211 20:52:50.810470 982 leader_election.cc:343] T 66cdc6f297b24cfba3a3a0ae9f45d12d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.810652 982 leader_election.cc:304] T 66cdc6f297b24cfba3a3a0ae9f45d12d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:50.810909 2974 raft_consensus.cc:2749] T 66cdc6f297b24cfba3a3a0ae9f45d12d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:50.811395 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet bb9fccb6981e446a939a28d7fdd38e25 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb9fccb6981e446a939a28d7fdd38e25 already in progress: creating tablet
I20251211 20:52:50.820675 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 449921ea74f04a9c88eb9e094e8226c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.821694 880 tablet_service.cc:1552] Processing DeleteTablet for tablet f91487dd7e0f4c8b8483c2b83945da3b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.824491 2974 raft_consensus.cc:493] T 29463a8f71f943a0927942235af3c2da P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.824677 2974 raft_consensus.cc:515] T 29463a8f71f943a0927942235af3c2da P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.825397 2974 leader_election.cc:290] T 29463a8f71f943a0927942235af3c2da P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:50.834635 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29463a8f71f943a0927942235af3c2da" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:50.834904 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ef524747034491c99f4f9767acd3c9d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:50110
W20251211 20:52:50.835317 983 leader_election.cc:343] T 29463a8f71f943a0927942235af3c2da P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:50.835698 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 63179.5us
I20251211 20:52:50.836287 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29463a8f71f943a0927942235af3c2da" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:50.836771 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 6029c1fd563d4023a8cf7cfb8ffa0876 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.837102 757 tablet_service.cc:1552] Processing DeleteTablet for tablet d0673c37c2fd47eea8ab10d9e91b8d7a with delete_type TABLET_DATA_DELETED (replaced by tablet 86c5228210954839ae7bf5bf0b5aa027) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.837558 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 516f7181aaac48cdadf9fbc08d0d3f57 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.839376 2974 raft_consensus.cc:493] T 062f980b65504d3baf99ae400e521775 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.839545 2974 raft_consensus.cc:515] T 062f980b65504d3baf99ae400e521775 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.840164 2974 leader_election.cc:290] T 062f980b65504d3baf99ae400e521775 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:50.840898 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 4776c456b4d74c90b1fc71d57b62a461 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.841630 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a61cf8f5de14adaa4a1de5bddde9a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.841944 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c5c937cc7db4ed18fe03f4a8a99a66a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.844077 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "062f980b65504d3baf99ae400e521775" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:50.844677 983 leader_election.cc:343] T 062f980b65504d3baf99ae400e521775 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.846958 3051 consensus_queue.cc:1048] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:50.849030 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 0f8808bfd68841e4ae30818018040cf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.849529 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 7d4e63f54d8443ae8322d4659a165e76 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.849891 757 tablet_service.cc:1552] Processing DeleteTablet for tablet bb9fccb6981e446a939a28d7fdd38e25 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.850214 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 449921ea74f04a9c88eb9e094e8226c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.854178 3354 raft_consensus.cc:493] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.854331 3354 raft_consensus.cc:515] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.855038 3354 leader_election.cc:290] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.856618 1166 tablet_bootstrap.cc:654] T 12aaf618d6474f599458026308d7afdd P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.860406 2974 raft_consensus.cc:493] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.860579 2974 raft_consensus.cc:515] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.861225 2974 leader_election.cc:290] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:50.862010 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8214c97ed1640f5965aa5c4b999e2f1" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:50.862620 983 leader_election.cc:343] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:50.865360 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 449921ea74f04a9c88eb9e094e8226c8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 449921ea74f04a9c88eb9e094e8226c8 already in progress: creating tablet
W20251211 20:52:50.865831 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet f91487dd7e0f4c8b8483c2b83945da3b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f91487dd7e0f4c8b8483c2b83945da3b already in progress: creating tablet
W20251211 20:52:50.866039 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 2ef524747034491c99f4f9767acd3c9d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2ef524747034491c99f4f9767acd3c9d already in progress: creating tablet
W20251211 20:52:50.867499 711 leader_election.cc:343] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.714811 3294 ts_tablet_manager.cc:1929] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:50.867882 3294 log.cc:1199] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/f22e4d34cde24b37a025c1b5a1046b1b
W20251211 20:52:50.867971 711 consensus_peers.cc:597] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:50.868270 3294 ts_tablet_manager.cc:1950] T f22e4d34cde24b37a025c1b5a1046b1b P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:50.868858 3294 tablet_replica.cc:333] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:50.869078 3294 raft_consensus.cc:2243] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:50.869354 3294 raft_consensus.cc:2272] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:50.869939 3294 ts_tablet_manager.cc:1916] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:50.869886 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79d92736bb1c46d2bfa5678b2d396ec2" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:50.871690 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet e95c63b4d99a4dd0b2c0fa384ca9d552 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.875360 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 4776c456b4d74c90b1fc71d57b62a461 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.876035 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet d0673c37c2fd47eea8ab10d9e91b8d7a with delete_type TABLET_DATA_DELETED (replaced by tablet 86c5228210954839ae7bf5bf0b5aa027) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.876576 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 516f7181aaac48cdadf9fbc08d0d3f57 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.877107 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 6029c1fd563d4023a8cf7cfb8ffa0876 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.877583 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a61cf8f5de14adaa4a1de5bddde9a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.879169 880 tablet_service.cc:1552] Processing DeleteTablet for tablet f22e4d34cde24b37a025c1b5a1046b1b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.880060 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 9b99860ee6be45a0b358e686332d3137 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.880807 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 0aac6fb3972e4aba8950828c9e49d507 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.881491 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 0add14ccd2af42aeb02fad7a28ed4520 with delete_type TABLET_DATA_DELETED (replaced by tablet 84cca3ee8fc446fcaf9971171235cf55) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.881538 1036 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79d92736bb1c46d2bfa5678b2d396ec2" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:50.882151 1036 raft_consensus.cc:2468] T 79d92736bb1c46d2bfa5678b2d396ec2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:50.882272 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 76cfb26762d54593a150ec98125c6592 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.882928 880 tablet_service.cc:1552] Processing DeleteTablet for tablet b631e6b3048e428f901af1b5b0b1ab3c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.882925 2966 raft_consensus.cc:493] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.883154 2966 raft_consensus.cc:515] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.883775 2966 leader_election.cc:290] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.883898 1166 tablet_bootstrap.cc:492] T 12aaf618d6474f599458026308d7afdd P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:50.884022 1166 ts_tablet_manager.cc:1403] T 12aaf618d6474f599458026308d7afdd P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.093s user 0.002s sys 0.001s
I20251211 20:52:50.884716 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 745ce63314834c8abdecaa367133a068 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.885797 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f7b1bc3dd1f4496b3ac6e71652cf012 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.886590 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 68792ccef54944a09af9b5456e9ff1fe with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:50110
W20251211 20:52:50.886951 847 leader_election.cc:343] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 2dde7f28e61d41f48015ba6e4f3757f2
I20251211 20:52:50.887955 2962 ts_tablet_manager.cc:1929] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:50.888151 2962 log.cc:1199] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/3acfef6f082641a199b80eefcfc772ac
I20251211 20:52:50.888470 710 leader_election.cc:304] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:50.888641 2962 ts_tablet_manager.cc:1950] T 3acfef6f082641a199b80eefcfc772ac P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:50.889418 2959 raft_consensus.cc:2804] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:50.889564 2959 raft_consensus.cc:493] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:50.889689 2959 raft_consensus.cc:3060] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.892256 2959 raft_consensus.cc:515] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.892573 880 tablet_service.cc:1552] Processing DeleteTablet for tablet d0673c37c2fd47eea8ab10d9e91b8d7a with delete_type TABLET_DATA_DELETED (replaced by tablet 86c5228210954839ae7bf5bf0b5aa027) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.893394 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 87c4ec07e82d4bf79b4eeefe732674ff with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.894224 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 1578887b2f5748cd83a6fbdec2f3fc99 with delete_type TABLET_DATA_DELETED (replaced by tablet bc4e7362e6d045da81bf39f4135f500e) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.894809 880 tablet_service.cc:1552] Processing DeleteTablet for tablet ba120d28abc54553ae15532de90db719 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.895390 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 4585e05d06ef42cfb06c4ceb7524fd71 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.896138 1168 tablet_bootstrap.cc:492] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:50.896116 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 3dde7ed495e64909bd9784dc5eefb04c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.896347 1168 ts_tablet_manager.cc:1403] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.278s user 0.011s sys 0.002s
I20251211 20:52:50.896836 880 tablet_service.cc:1552] Processing DeleteTablet for tablet e709051411354522bdb327c34dbf533a with delete_type TABLET_DATA_DELETED (replaced by tablet dfeb47dfaa6445309e28715b1b8db2ae) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.897583 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 6cfa15533fad4aec84f8d9038227faf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.897924 1166 raft_consensus.cc:359] T 12aaf618d6474f599458026308d7afdd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.898207 1166 raft_consensus.cc:385] T 12aaf618d6474f599458026308d7afdd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.898319 1166 raft_consensus.cc:740] T 12aaf618d6474f599458026308d7afdd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.898268 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 48820386e41b47129a13bdd5768f021b with delete_type TABLET_DATA_DELETED (replaced by tablet 7a299ee03d0e48aba7c4f71e3e0f381a) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.898572 3295 tablet_replica.cc:333] T f22e4d34cde24b37a025c1b5a1046b1b P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:50.898694 1166 consensus_queue.cc:260] T 12aaf618d6474f599458026308d7afdd P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.898792 3295 raft_consensus.cc:2243] T f22e4d34cde24b37a025c1b5a1046b1b P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:50.899006 3295 raft_consensus.cc:2272] T f22e4d34cde24b37a025c1b5a1046b1b P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:50.899627 3295 ts_tablet_manager.cc:1916] T f22e4d34cde24b37a025c1b5a1046b1b P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:50.901021 1036 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79d92736bb1c46d2bfa5678b2d396ec2" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:50.901203 1036 raft_consensus.cc:3060] T 79d92736bb1c46d2bfa5678b2d396ec2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.905522 904 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79d92736bb1c46d2bfa5678b2d396ec2" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:50.907881 1036 raft_consensus.cc:2468] T 79d92736bb1c46d2bfa5678b2d396ec2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:50.893417 2959 leader_election.cc:290] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.909162 1160 tablet_bootstrap.cc:492] T 95866de91f9d437d80f40b7962253610 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:50.909309 1160 ts_tablet_manager.cc:1403] T 95866de91f9d437d80f40b7962253610 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.237s user 0.041s sys 0.063s
I20251211 20:52:50.910044 1160 raft_consensus.cc:359] T 95866de91f9d437d80f40b7962253610 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.910221 1160 raft_consensus.cc:385] T 95866de91f9d437d80f40b7962253610 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.910271 1160 raft_consensus.cc:740] T 95866de91f9d437d80f40b7962253610 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.910447 1160 consensus_queue.cc:260] T 95866de91f9d437d80f40b7962253610 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.911458 1168 raft_consensus.cc:359] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.911825 1166 ts_tablet_manager.cc:1434] T 12aaf618d6474f599458026308d7afdd P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.028s user 0.002s sys 0.000s
I20251211 20:52:50.912726 1166 tablet_bootstrap.cc:492] T f26ba8f4bd3a4a1f8fa2ba6e80d543f8 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:50.913479 1168 raft_consensus.cc:385] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.913653 1168 raft_consensus.cc:740] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.912834 710 leader_election.cc:304] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:50.913249 756 tablet_service.cc:1552] Processing DeleteTablet for tablet 41627cb27be8443fa37e514f9b3825a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.913503 747 tablet_service.cc:1552] Processing DeleteTablet for tablet 87c4ec07e82d4bf79b4eeefe732674ff with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.912345 757 tablet_service.cc:1552] Processing DeleteTablet for tablet 9b99860ee6be45a0b358e686332d3137 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.916491 2429 raft_consensus.cc:2804] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:50.916631 2429 raft_consensus.cc:697] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:50.916868 2429 consensus_queue.cc:237] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:50.908746 711 leader_election.cc:343] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.917881 744 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ce495233a5e4f3792faf44680df54ba with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.913791 742 tablet_service.cc:1552] Processing DeleteTablet for tablet 1578887b2f5748cd83a6fbdec2f3fc99 with delete_type TABLET_DATA_DELETED (replaced by tablet bc4e7362e6d045da81bf39f4135f500e) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.914045 738 tablet_service.cc:1552] Processing DeleteTablet for tablet 6cfa15533fad4aec84f8d9038227faf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.914130 746 tablet_service.cc:1552] Processing DeleteTablet for tablet 48820386e41b47129a13bdd5768f021b with delete_type TABLET_DATA_DELETED (replaced by tablet 7a299ee03d0e48aba7c4f71e3e0f381a) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.914677 751 tablet_service.cc:1552] Processing DeleteTablet for tablet bd2bf1113dfe4b67b263b56f3d3db07d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.918104 748 tablet_service.cc:1552] Processing DeleteTablet for tablet fbc2d4aea084492c8270ad1d87096e58 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.914975 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 4585e05d06ef42cfb06c4ceb7524fd71 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.915056 739 tablet_service.cc:1552] Processing DeleteTablet for tablet 06ba98a726424dd89450053c5c5793f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.915143 741 tablet_service.cc:1552] Processing DeleteTablet for tablet fa79b67b9eee45e2b41c4a90967606cc with delete_type TABLET_DATA_DELETED (replaced by tablet 83067b139f504bc191a515feed0ec771) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.915064 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 83067b139f504bc191a515feed0ec771 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.915249 749 tablet_service.cc:1552] Processing DeleteTablet for tablet e8db5a11a69e4554bc689c38ab57b28a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.914866 745 tablet_service.cc:1552] Processing DeleteTablet for tablet e709051411354522bdb327c34dbf533a with delete_type TABLET_DATA_DELETED (replaced by tablet dfeb47dfaa6445309e28715b1b8db2ae) from {username='slave'} at 127.0.0.1:56136
W20251211 20:52:50.918512 711 consensus_peers.cc:597] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:50.918898 1160 ts_tablet_manager.cc:1434] T 95866de91f9d437d80f40b7962253610 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20251211 20:52:50.919217 1160 tablet_bootstrap.cc:492] T 7be44df39a8f4d6da793479d425fa684 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:50.915486 1168 consensus_queue.cc:260] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:50.920773 711 leader_election.cc:343] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.922791 1168 ts_tablet_manager.cc:1434] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.026s user 0.002s sys 0.000s
I20251211 20:52:50.924494 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 042f8bf13a7a4712b5bdc04671cd9938 with delete_type TABLET_DATA_DELETED (replaced by tablet 53b4116db0a44dfbb4c1ec5b2acebc67) from {username='slave'} at 127.0.0.1:36220
W20251211 20:52:50.925323 1797 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 4c0455a5b4f44412803475d1c4852b48 (table default.Table_e00092673b0e4f09b8d7eec63e1ee034 [id=18497e06d7254441a4655ce1cdc5a9bf], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 4c0455a5b4f44412803475d1c4852b48 consensus state has no leader
W20251211 20:52:50.926607 847 leader_election.cc:343] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 2dde7f28e61d41f48015ba6e4f3757f2
I20251211 20:52:50.926757 847 leader_election.cc:304] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:50.927143 2966 raft_consensus.cc:2749] T 2dde7f28e61d41f48015ba6e4f3757f2 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:50.927407 2966 raft_consensus.cc:493] T d43210ec61af423ebde624397dea9f2e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.927523 2966 raft_consensus.cc:515] T d43210ec61af423ebde624397dea9f2e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.927978 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 84ed434868aa437a83455b1605deb26c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.928148 2966 leader_election.cc:290] T d43210ec61af423ebde624397dea9f2e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.928174 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet eb9da081cb9d43fd9a35c4bc51ea133e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.928385 904 raft_consensus.cc:3060] T ce1799e6604f4ed7875ddedecab6f327 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.928632 1160 tablet_bootstrap.cc:654] T 7be44df39a8f4d6da793479d425fa684 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.929111 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 877aa237fe43452cbdbb4dac46f613ae with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.929414 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 28ddb8a59a214576a25d613d42c2fef5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.929860 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet 7452dd736b0440fcb4626858e794676f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.931697 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d43210ec61af423ebde624397dea9f2e" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:50.931988 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 139) took 1866 ms. Trace:
I20251211 20:52:50.932895 2962 rpcz_store.cc:276] 1211 20:52:49.065478 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.065515 (+ 37us) service_pool.cc:225] Handling call
1211 20:52:50.587983 (+1522468us) ts_tablet_manager.cc:1151] Deleting tablet 3acfef6f082641a199b80eefcfc772ac
1211 20:52:50.587994 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:50.592461 (+ 4467us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:50.887831 (+295370us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:50.931966 (+ 44135us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1522315}
I20251211 20:52:50.933610 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 3acfef6f082641a199b80eefcfc772ac (table default.Table_320e1edc81a842849ac04e448aa2e2d4 [id=1c8eceb8a080464192ef7ab198728cd5]) successfully deleted
I20251211 20:52:50.933698 1166 tablet_bootstrap.cc:654] T f26ba8f4bd3a4a1f8fa2ba6e80d543f8 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.933729 2962 tablet_replica.cc:333] T 59e955aa07c14fbbb347b88eca59a74e P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:50.933920 2962 raft_consensus.cc:2243] T 59e955aa07c14fbbb347b88eca59a74e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:50.936709 2962 raft_consensus.cc:2272] T 59e955aa07c14fbbb347b88eca59a74e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:50.937345 1168 tablet_bootstrap.cc:492] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:50.938547 880 tablet_service.cc:1552] Processing DeleteTablet for tablet bd2bf1113dfe4b67b263b56f3d3db07d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.941043 1160 tablet_bootstrap.cc:492] T 7be44df39a8f4d6da793479d425fa684 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:50.941215 1160 ts_tablet_manager.cc:1403] T 7be44df39a8f4d6da793479d425fa684 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.000s
I20251211 20:52:50.941972 1160 raft_consensus.cc:359] T 7be44df39a8f4d6da793479d425fa684 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.942149 1160 raft_consensus.cc:385] T 7be44df39a8f4d6da793479d425fa684 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:50.942199 1160 raft_consensus.cc:740] T 7be44df39a8f4d6da793479d425fa684 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:50.942279 3294 ts_tablet_manager.cc:1929] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:50.942399 3294 log.cc:1199] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/0aac6fb3972e4aba8950828c9e49d507
I20251211 20:52:50.942385 1160 consensus_queue.cc:260] T 7be44df39a8f4d6da793479d425fa684 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.942770 3294 ts_tablet_manager.cc:1950] T 0aac6fb3972e4aba8950828c9e49d507 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:50.944084 3155 consensus_queue.cc:1048] T 962b80ae747b4470abd23643e711fad9 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:50.945115 3294 tablet_replica.cc:333] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:50.945314 3294 raft_consensus.cc:2243] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:50.945570 3294 pending_rounds.cc:70] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919: Trying to abort 1 pending ops.
I20251211 20:52:50.945611 3294 pending_rounds.cc:77] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231432173072297984 op_type: NO_OP noop_request { }
I20251211 20:52:50.945708 3294 raft_consensus.cc:2889] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251211 20:52:50.945799 3294 raft_consensus.cc:2272] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:50.943727 1029 tablet_service.cc:1552] Processing DeleteTablet for tablet e9ead34065e240eeb1f946319537d32a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.944862 1027 tablet_service.cc:1552] Processing DeleteTablet for tablet 3acfef6f082641a199b80eefcfc772ac with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.944759 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 3621735fcb294e3a8c0180ba9ef86442 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.939396 2962 ts_tablet_manager.cc:1916] T 59e955aa07c14fbbb347b88eca59a74e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:50.951479 903 raft_consensus.cc:3060] T 44e2d2bdecf247baaac26e06bccc94de P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.953758 3155 consensus_queue.cc:1048] T c081396b492049b996ac4661e1318388 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:50.954771 2974 raft_consensus.cc:493] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.954792 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 260bdeb73439437cbf234d20f7264ab3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.954913 2974 raft_consensus.cc:515] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.955142 749 tablet_service.cc:1552] Processing DeleteTablet for tablet a4ccf9a4ecbc4ca6978a189cf3a15893 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.955469 2974 leader_election.cc:290] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:50.955649 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 3acfef6f082641a199b80eefcfc772ac with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.956350 749 tablet_service.cc:1552] Processing DeleteTablet for tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.956717 3174 raft_consensus.cc:493] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:50.956838 3174 raft_consensus.cc:515] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:50.956918 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 10746162da4640e6a2a891467690b770 with delete_type TABLET_DATA_DELETED (replaced by tablet 8dc4e730951144d78b77280f6cce6d0e) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.957197 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 13ceabb190494e718a52892faaa53986 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.957432 3174 leader_election.cc:290] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:50.957453 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 6663f13691a24fb991b48f543d9de76b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.957703 749 tablet_service.cc:1552] Processing DeleteTablet for tablet aeff1fbabf7845dc9863f59f286ddcec with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.957979 749 tablet_service.cc:1552] Processing DeleteTablet for tablet c8214c97ed1640f5965aa5c4b999e2f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.959029 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 4df68d1b6a91458e9f212ae81aa25438 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.959095 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "51d12723f5194afb97d2c05fcbaf3776" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:50.959643 983 leader_election.cc:343] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:50.959723 749 tablet_service.cc:1552] Processing DeleteTablet for tablet a471a9accbfc4875bd1d7802f5620808 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.960022 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 4a39870c7eea4a379b284ea2e5ddf760 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.960352 749 tablet_service.cc:1552] Processing DeleteTablet for tablet b30d2054316b49fa8096345527a45d87 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.960377 1036 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "41bf5b5111f64dfa91ed76ed4ab4f6f7" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:50.960635 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 50f9f29c064c4f938a0f20f302ab5e8a with delete_type TABLET_DATA_DELETED (replaced by tablet da81bf8854a24954ba4548185812295d) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.960917 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ec261837cf04bf88a48ff8add1b0d66 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.961233 749 tablet_service.cc:1552] Processing DeleteTablet for tablet f3855ffb51a74101b26fd28429e65bc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.961544 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 5262c2241e524395a73c0eb368917169 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.961817 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 28ddb8a59a214576a25d613d42c2fef5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.962107 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 877aa237fe43452cbdbb4dac46f613ae with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:50.966642 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d43210ec61af423ebde624397dea9f2e" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:50.968963 1797 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 13269a20421240d1a98215129443248b (table default.Table_d887c953026046dc883ede774a58fd8e [id=1d8a79f716a0447cb0393f4759178598], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 13269a20421240d1a98215129443248b consensus state has no leader
W20251211 20:52:50.973542 1797 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 171d40a8a9124f9bb1b38797fef8e5d5 (table default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 171d40a8a9124f9bb1b38797fef8e5d5 consensus state has no leader
I20251211 20:52:50.974534 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "41bf5b5111f64dfa91ed76ed4ab4f6f7" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:50.975821 847 leader_election.cc:304] T d43210ec61af423ebde624397dea9f2e P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:50.975843 914 raft_consensus.cc:3060] T 1578887b2f5748cd83a6fbdec2f3fc99 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:50.976270 3174 raft_consensus.cc:2749] T d43210ec61af423ebde624397dea9f2e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:50.976727 847 leader_election.cc:304] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:50.977244 3174 raft_consensus.cc:2749] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:50.980652 3360 consensus_queue.cc:1048] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:50.985742 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 06b8f8e46e524545aeab6c931efaf116 with delete_type TABLET_DATA_DELETED (replaced by tablet 7bc03ed8c35740f58fe48ddbaa120ea0) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.986215 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 14f367adb60744c8a593d3b9d4711823 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.986655 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet e95b0d8c507a4779a087c8933d464228 with delete_type TABLET_DATA_DELETED (replaced by tablet 9c5c937cc7db4ed18fe03f4a8a99a66a) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.987042 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 07e4336171714da8adc21977ce33ce50 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.987928 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 80c566822fa94ada983aa7266df4bf9d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.988302 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet e1aeae8c18b6499e9dc1accabdc3f633 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.988698 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b1ce6d2a12a4c7ea044b349412b4bf5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.989121 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 5710fc2f24764f818f7e5c86c6fd985e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.989485 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 4810f8d8b53a45eb939bf3c8bf1b4e77 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.989850 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 3562b5effa124378beac7ccf0d27ee7e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:50.992051 1160 ts_tablet_manager.cc:1434] T 7be44df39a8f4d6da793479d425fa684 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.051s user 0.002s sys 0.000s
I20251211 20:52:50.996445 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 5ddf7c1972564ef88e3c0a0bd1b430e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.996948 1160 tablet_bootstrap.cc:492] T 58e35de0792a464a818170886ca8d8c9 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:50.997393 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 18b977f463d5420186c007b38110e07e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.998243 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 41627cb27be8443fa37e514f9b3825a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.998777 1160 tablet_bootstrap.cc:654] T 58e35de0792a464a818170886ca8d8c9 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:50.999153 880 tablet_service.cc:1552] Processing DeleteTablet for tablet a4ccf9a4ecbc4ca6978a189cf3a15893 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:50.999850 880 tablet_service.cc:1552] Processing DeleteTablet for tablet a56f1a7bd5464d289bfd018c826ee574 with delete_type TABLET_DATA_DELETED (replaced by tablet 284e34970f634abcb2288c864f73e963) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.000584 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 06ba98a726424dd89450053c5c5793f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.001251 880 tablet_service.cc:1552] Processing DeleteTablet for tablet e8db5a11a69e4554bc689c38ab57b28a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.001926 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 83067b139f504bc191a515feed0ec771 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.002584 880 tablet_service.cc:1552] Processing DeleteTablet for tablet fbc2d4aea084492c8270ad1d87096e58 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.003340 880 tablet_service.cc:1552] Processing DeleteTablet for tablet fa79b67b9eee45e2b41c4a90967606cc with delete_type TABLET_DATA_DELETED (replaced by tablet 83067b139f504bc191a515feed0ec771) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.003949 3375 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.60:42087 every 8 connection(s)
I20251211 20:52:51.004009 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 13ceabb190494e718a52892faaa53986 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.004595 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 6663f13691a24fb991b48f543d9de76b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.005221 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 10746162da4640e6a2a891467690b770 with delete_type TABLET_DATA_DELETED (replaced by tablet 8dc4e730951144d78b77280f6cce6d0e) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.005842 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 92a71dff4ea8445d8484fc2747471148 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.006434 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 87863daf77304a399318cdf56da9f9bb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.007100 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 3562b5effa124378beac7ccf0d27ee7e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.007723 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 4f807596fa6a4e66b89566b5d5dd1080 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.008719 3110 tablet.cc:1722] T ce1799e6604f4ed7875ddedecab6f327 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251211 20:52:51.004356 2550 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.60:42087
I20251211 20:52:51.010598 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 7d4bba0aa7ea403e8dfcb74cec2e1f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.011341 880 tablet_service.cc:1552] Processing DeleteTablet for tablet e9ead34065e240eeb1f946319537d32a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.012050 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 3621735fcb294e3a8c0180ba9ef86442 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.012826 2962 ts_tablet_manager.cc:1929] T 59e955aa07c14fbbb347b88eca59a74e P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:51.015625 2550 server_base.cc:1179] Dumped server information to /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/master-2/data/info.pb
I20251211 20:52:51.018146 416 external_mini_cluster.cc:1428] Started /tmp/dist-test-task5TWsQc/build/asan/bin/kudu as pid 2550
I20251211 20:52:51.018708 1168 tablet_bootstrap.cc:654] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.019045 2962 log.cc:1199] T 59e955aa07c14fbbb347b88eca59a74e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/59e955aa07c14fbbb347b88eca59a74e
I20251211 20:52:51.019498 2962 ts_tablet_manager.cc:1950] T 59e955aa07c14fbbb347b88eca59a74e P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:51.020367 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 140) took 1954 ms. Trace:
I20251211 20:52:51.027714 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 25454e67437541d9ac2bfab29bad55f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.028105 2962 rpcz_store.cc:276] 1211 20:52:49.065822 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.065857 (+ 35us) service_pool.cc:225] Handling call
1211 20:52:50.933665 (+1867808us) ts_tablet_manager.cc:1151] Deleting tablet 59e955aa07c14fbbb347b88eca59a74e
1211 20:52:50.933677 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:51.004516 (+ 70839us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:51.012715 (+ 8199us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:51.020343 (+ 7628us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1867607}
I20251211 20:52:51.028716 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 52ef1707f6f2425aafd454f303af0333 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.029115 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 141) took 1949 ms. Trace:
I20251211 20:52:51.029186 749 tablet_service.cc:1552] Processing DeleteTablet for tablet e95c63b4d99a4dd0b2c0fa384ca9d552 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.029608 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 88970b575ba943129729eceef5636519 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.029968 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 7b06cd366de5494780b5ac81abbf0702 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.030005 1160 tablet_bootstrap.cc:492] T 58e35de0792a464a818170886ca8d8c9 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:51.030225 1160 ts_tablet_manager.cc:1403] T 58e35de0792a464a818170886ca8d8c9 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.033s user 0.004s sys 0.000s
I20251211 20:52:51.030299 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 373d3f03d7e0408cbe06955f8f60b16c with delete_type TABLET_DATA_DELETED (replaced by tablet 9ae2a68a1d0c45ae93d42472221bdc65) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.030603 749 tablet_service.cc:1552] Processing DeleteTablet for tablet a7979798e92e45eb969197c8c7ac2580 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.030920 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 5d225553d0984b6e9685b5841616c9d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:48 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.031225 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 07e4336171714da8adc21977ce33ce50 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.031505 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 1e13965b6361488eaeb7dd13b467ca65 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.031826 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 58e35de0792a464a818170886ca8d8c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.032117 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 14f367adb60744c8a593d3b9d4711823 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.032397 749 tablet_service.cc:1552] Processing DeleteTablet for tablet c8bb92fc25124ba4be887c453abc7d9a with delete_type TABLET_DATA_DELETED (replaced by tablet 80486aaeb28849c58c855475871853c0) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.032316 1160 raft_consensus.cc:359] T 58e35de0792a464a818170886ca8d8c9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.032608 1160 raft_consensus.cc:385] T 58e35de0792a464a818170886ca8d8c9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.032707 1160 raft_consensus.cc:740] T 58e35de0792a464a818170886ca8d8c9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.032706 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 7bc03ed8c35740f58fe48ddbaa120ea0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.032994 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 32f8d104f6a948d1a97a83769ea82449 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.032995 1160 consensus_queue.cc:260] T 58e35de0792a464a818170886ca8d8c9 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.033715 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a206abd6ea2420089cb07714e0d19c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.033735 1160 ts_tablet_manager.cc:1434] T 58e35de0792a464a818170886ca8d8c9 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251211 20:52:51.034089 1160 tablet_bootstrap.cc:492] T 3621735fcb294e3a8c0180ba9ef86442 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:51.034718 749 tablet_service.cc:1552] Processing DeleteTablet for tablet a471a9accbfc4875bd1d7802f5620808 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.018461 1166 tablet_bootstrap.cc:492] T f26ba8f4bd3a4a1f8fa2ba6e80d543f8 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:51.035039 749 tablet_service.cc:1552] Processing DeleteTablet for tablet a295a24e81f14e5fa126a4a58ffd08cf with delete_type TABLET_DATA_DELETED (replaced by tablet 77bd176108f24e79a632c798566e9e31) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.035280 1166 ts_tablet_manager.cc:1403] T f26ba8f4bd3a4a1f8fa2ba6e80d543f8 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.123s user 0.004s sys 0.000s
I20251211 20:52:51.035349 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 6d4e6528352f43b6a1b47d7a4b6b3689 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.035687 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 215f8dc74a71482daa2ca80e15010f51 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.035989 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 1944267def4e46949fd680ab0a369be9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.036296 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 41c30bded6504bbc9f0af8f34dc81e8d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.036347 1166 raft_consensus.cc:359] T f26ba8f4bd3a4a1f8fa2ba6e80d543f8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.036572 749 tablet_service.cc:1552] Processing DeleteTablet for tablet c7147571483e41ad970e573a08c368ca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.036614 1166 raft_consensus.cc:385] T f26ba8f4bd3a4a1f8fa2ba6e80d543f8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.036741 1166 raft_consensus.cc:740] T f26ba8f4bd3a4a1f8fa2ba6e80d543f8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.036989 1166 consensus_queue.cc:260] T f26ba8f4bd3a4a1f8fa2ba6e80d543f8 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.037077 749 tablet_service.cc:1552] Processing DeleteTablet for tablet f9a3fdf17f274c15ac0bdfa9c174f649 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.037549 1166 ts_tablet_manager.cc:1434] T f26ba8f4bd3a4a1f8fa2ba6e80d543f8 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:51.037882 1166 tablet_bootstrap.cc:492] T 200b6db115964f3c8882b514c794698d P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:51.039629 2962 rpcz_store.cc:276] 1211 20:52:49.079907 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.079970 (+ 63us) service_pool.cc:225] Handling call
1211 20:52:51.028959 (+1948989us) ts_tablet_manager.cc:1151] Deleting tablet c97d410272a947de8b5d7fb40fe4777d
1211 20:52:51.028972 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:51.029095 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1948747}
I20251211 20:52:51.039918 3294 ts_tablet_manager.cc:1916] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:51.041646 1160 tablet_bootstrap.cc:654] T 3621735fcb294e3a8c0180ba9ef86442 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.042119 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 143) took 1952 ms. Trace:
I20251211 20:52:51.042742 2962 rpcz_store.cc:276] 1211 20:52:49.089795 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.089855 (+ 60us) service_pool.cc:225] Handling call
1211 20:52:51.041945 (+1952090us) ts_tablet_manager.cc:1151] Deleting tablet 88970b575ba943129729eceef5636519
1211 20:52:51.041960 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:51.042096 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1951848}
I20251211 20:52:51.043247 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 144) took 1952 ms. Trace:
I20251211 20:52:51.043378 2962 rpcz_store.cc:276] 1211 20:52:49.090377 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.090667 (+ 290us) service_pool.cc:225] Handling call
1211 20:52:51.043114 (+1952447us) ts_tablet_manager.cc:1151] Deleting tablet d15eff49af284a38bb1babdb231a35df
1211 20:52:51.043126 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:51.043231 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1952235}
I20251211 20:52:51.043804 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 145) took 1952 ms. Trace:
I20251211 20:52:51.043951 2962 rpcz_store.cc:276] 1211 20:52:49.091378 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.091419 (+ 41us) service_pool.cc:225] Handling call
1211 20:52:51.043671 (+1952252us) ts_tablet_manager.cc:1151] Deleting tablet a471a9accbfc4875bd1d7802f5620808
1211 20:52:51.043680 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:51.043789 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1952096}
I20251211 20:52:51.046411 3294 ts_tablet_manager.cc:1929] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:51.042678 1166 tablet_bootstrap.cc:654] T 200b6db115964f3c8882b514c794698d P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.046550 3294 log.cc:1199] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/f91487dd7e0f4c8b8483c2b83945da3b
I20251211 20:52:51.047797 2962 tablet_replica.cc:333] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:51.049002 2962 raft_consensus.cc:2243] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:51.049283 2962 raft_consensus.cc:2272] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:51.049517 1096 heartbeater.cc:499] Master 127.0.104.61:40749 was elected leader, sending a full tablet report...
I20251211 20:52:51.049986 2962 ts_tablet_manager.cc:1916] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:51.050697 749 tablet_service.cc:1552] Processing DeleteTablet for tablet e95b0d8c507a4779a087c8933d464228 with delete_type TABLET_DATA_DELETED (replaced by tablet 9c5c937cc7db4ed18fe03f4a8a99a66a) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.051134 749 tablet_service.cc:1552] Processing DeleteTablet for tablet ca89f09cc0954389b679e8bb8b5e4dfb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:56136
W20251211 20:52:51.051348 1797 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.1:55697 (request call id 163) took 2784 ms (client timeout 1000 ms). Trace:
I20251211 20:52:51.051400 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 59e955aa07c14fbbb347b88eca59a74e (table default.Table_f84cd3de863b446894133bb40aaf1a39 [id=f4d608cc41d0408486d3f6b0a09383a4]) successfully deleted
W20251211 20:52:51.052716 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet c97d410272a947de8b5d7fb40fe4777d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c97d410272a947de8b5d7fb40fe4777d already in progress: creating tablet
W20251211 20:52:51.051541 1797 rpcz_store.cc:269] 1211 20:52:48.244032 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:48.498763 (+254731us) service_pool.cc:225] Handling call
1211 20:52:49.508424 (+1009661us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:49.508430 (+ 6us) write_op.cc:454] Released schema lock
1211 20:52:51.028548 (+1520118us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:49.500083 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:49.500213 (+ 130us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:49.500220 (+ 7us) write_op.cc:435] Acquired schema lock
1211 20:52:49.500222 (+ 2us) tablet.cc:660] Decoding operations
1211 20:52:49.500387 (+ 165us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:49.500426 (+ 39us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:49.500431 (+ 5us) tablet.cc:683] Acquiring locks for 4 operations
1211 20:52:49.500544 (+ 113us) tablet.cc:699] Row locks acquired
1211 20:52:49.500546 (+ 2us) write_op.cc:260] PREPARE: finished
1211 20:52:49.500589 (+ 43us) write_op.cc:270] Start()
1211 20:52:49.500627 (+ 38us) write_op.cc:276] Timestamp: P: 1765486369500564 usec, L: 0
1211 20:52:49.500630 (+ 3us) op_driver.cc:348] REPLICATION: starting
1211 20:52:49.500907 (+ 277us) log.cc:844] Serialized 1650 byte log entry
1211 20:52:49.505178 (+ 4271us) op_driver.cc:464] REPLICATION: finished
1211 20:52:49.505327 (+ 149us) write_op.cc:301] APPLY: starting
1211 20:52:49.505344 (+ 17us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:49.505397 (+ 53us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:49.505424 (+ 27us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:49.505523 (+ 99us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:49.505561 (+ 38us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
1211 20:52:49.505568 (+ 7us) write_op.cc:312] APPLY: finished
1211 20:52:49.505664 (+ 96us) log.cc:844] Serialized 41 byte log entry
1211 20:52:49.505727 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:49.505754 (+ 27us) write_op.cc:454] Released schema lock
1211 20:52:49.505761 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":106972,"reported_tablets":447,"child_traces":[["op",{"apply.queue_time_us":101,"apply.run_cpu_time_us":489,"apply.run_wall_time_us":488,"num_ops":4,"prepare.queue_time_us":1067,"prepare.run_cpu_time_us":942,"prepare.run_wall_time_us":942,"raft.queue_time_us":360,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":234,"replication_time_us":4518}]]}
I20251211 20:52:51.054045 1168 tablet_bootstrap.cc:492] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:51.054241 1168 ts_tablet_manager.cc:1403] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.131s user 0.004s sys 0.000s
I20251211 20:52:51.055066 1168 raft_consensus.cc:359] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.055303 1168 raft_consensus.cc:385] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.055414 1168 raft_consensus.cc:740] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.055686 1168 consensus_queue.cc:260] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.056341 1168 ts_tablet_manager.cc:1434] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:51.056666 1168 tablet_bootstrap.cc:492] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:51.058637 1168 tablet_bootstrap.cc:654] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.062047 749 tablet_service.cc:1552] Processing DeleteTablet for tablet d53b074865f4490693152a247b0fffdc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.062672 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 2e29df291d8d40daba83e1d9f97f902b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.063082 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 5710fc2f24764f818f7e5c86c6fd985e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.065423 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 0b267c7feaac43f6aacb7229759c7ec3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.065564 888 tablet_service.cc:1552] Processing DeleteTablet for tablet cce44b80c97948e1af7ac68e77cccfd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.065622 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 7b06cd366de5494780b5ac81abbf0702 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.065730 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 84ed434868aa437a83455b1605deb26c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.065888 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 06b8f8e46e524545aeab6c931efaf116 with delete_type TABLET_DATA_DELETED (replaced by tablet 7bc03ed8c35740f58fe48ddbaa120ea0) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.066254 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 7d1cc9809ab843e693a179a35c53ce25 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.066648 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 098054406a1d459b824a094f50e71e7e with delete_type TABLET_DATA_DELETED (replaced by tablet aa50ddb2789b4be3bfa9aedef9287109) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.067104 749 tablet_service.cc:1552] Processing DeleteTablet for tablet cc43ee61e78249bfa00b11f3a1fa21a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.067458 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 7452dd736b0440fcb4626858e794676f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.067806 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 4810f8d8b53a45eb939bf3c8bf1b4e77 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.068491 3295 ts_tablet_manager.cc:1929] T f22e4d34cde24b37a025c1b5a1046b1b P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:51.068578 749 tablet_service.cc:1552] Processing DeleteTablet for tablet efe4d6bb07374294b8ab50a9fe605f87 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.069290 749 tablet_service.cc:1552] Processing DeleteTablet for tablet d15eff49af284a38bb1babdb231a35df with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.071120 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 01104975fd8b470aa026578b97ff6b8d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.072827 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 3c5819ae1abb48fe8a83af9067726d02 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.075322 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b1ce6d2a12a4c7ea044b349412b4bf5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.075731 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 940dfd8781de444a9caa78dd363c7451 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:56136
W20251211 20:52:51.076112 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 88970b575ba943129729eceef5636519 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 88970b575ba943129729eceef5636519 already in progress: creating tablet
I20251211 20:52:51.076874 1166 tablet_bootstrap.cc:492] T 200b6db115964f3c8882b514c794698d P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:51.077067 1166 ts_tablet_manager.cc:1403] T 200b6db115964f3c8882b514c794698d P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.039s user 0.003s sys 0.004s
I20251211 20:52:51.077795 1166 raft_consensus.cc:359] T 200b6db115964f3c8882b514c794698d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.078032 1166 raft_consensus.cc:385] T 200b6db115964f3c8882b514c794698d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.078125 1166 raft_consensus.cc:740] T 200b6db115964f3c8882b514c794698d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.078400 1166 consensus_queue.cc:260] T 200b6db115964f3c8882b514c794698d P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.078939 1166 ts_tablet_manager.cc:1434] T 200b6db115964f3c8882b514c794698d P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:51.079277 1166 tablet_bootstrap.cc:492] T 10746162da4640e6a2a891467690b770 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:51.081063 1166 tablet_bootstrap.cc:654] T 10746162da4640e6a2a891467690b770 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.086279 3360 raft_consensus.cc:493] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:51.086447 3360 raft_consensus.cc:515] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.087038 3360 leader_election.cc:290] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:51.092638 749 tablet_service.cc:1552] Processing DeleteTablet for tablet d69fb1f6d9594afcbfb844860b41a377 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.087839 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7b06cd366de5494780b5ac81abbf0702" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:51.093111 1048 raft_consensus.cc:2468] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:51.093355 749 tablet_service.cc:1552] Processing DeleteTablet for tablet e1aeae8c18b6499e9dc1accabdc3f633 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.079300 2962 ts_tablet_manager.cc:1929] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:51.093624 2962 log.cc:1199] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/d9cf0d92ccc7419cb6e7694e8e5c7bfb
I20251211 20:52:51.093587 710 leader_election.cc:304] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:51.093851 3360 raft_consensus.cc:2804] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:51.093922 3360 raft_consensus.cc:493] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:51.093981 3360 raft_consensus.cc:3060] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.094024 2962 ts_tablet_manager.cc:1950] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:51.094101 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 5bcc94ec9630484cb7a3e8bf6d4f378a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.094367 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 146) took 2002 ms. Trace:
I20251211 20:52:51.094453 2962 rpcz_store.cc:276] 1211 20:52:49.091820 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.091864 (+ 44us) service_pool.cc:225] Handling call
1211 20:52:51.047733 (+1955869us) ts_tablet_manager.cc:1151] Deleting tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb
1211 20:52:51.047747 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:51.076223 (+ 28476us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:51.079193 (+ 2970us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:51.094329 (+ 15136us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1955679}
I20251211 20:52:51.094643 749 tablet_service.cc:1552] Processing DeleteTablet for tablet c26c3d0170f64a41ae8944c6668cc5f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.094949 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 147) took 2002 ms. Trace:
I20251211 20:52:51.095032 2962 rpcz_store.cc:276] 1211 20:52:49.092237 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.092275 (+ 38us) service_pool.cc:225] Handling call
1211 20:52:51.094800 (+2002525us) ts_tablet_manager.cc:1151] Deleting tablet aeff1fbabf7845dc9863f59f286ddcec
1211 20:52:51.094809 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:51.094915 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2002376}
I20251211 20:52:51.095288 2962 tablet_replica.cc:333] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:51.095474 2962 raft_consensus.cc:2243] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:51.095662 1168 tablet_bootstrap.cc:492] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:51.095692 2962 raft_consensus.cc:2272] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:51.095788 1168 ts_tablet_manager.cc:1403] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.039s user 0.004s sys 0.000s
I20251211 20:52:51.096488 1168 raft_consensus.cc:359] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.096637 1168 raft_consensus.cc:385] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.096694 1168 raft_consensus.cc:740] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.096706 3360 raft_consensus.cc:515] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.097065 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 95866de91f9d437d80f40b7962253610 with delete_type TABLET_DATA_DELETED (replaced by tablet 51d12723f5194afb97d2c05fcbaf3776) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.097258 3360 leader_election.cc:290] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:51.098114 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7b06cd366de5494780b5ac81abbf0702" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:51.098322 1048 raft_consensus.cc:3060] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.100888 1048 raft_consensus.cc:2468] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:51.101368 710 leader_election.cc:304] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:51.101691 3360 raft_consensus.cc:2804] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:51.101800 3360 raft_consensus.cc:697] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:51.102030 3360 consensus_queue.cc:237] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:51.104135 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet d15eff49af284a38bb1babdb231a35df because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d15eff49af284a38bb1babdb231a35df already in progress: creating tablet
I20251211 20:52:51.104288 3385 sys_catalog.cc:263] Verifying existing consensus state
I20251211 20:52:51.105718 1095 heartbeater.cc:344] Connected to a master server at 127.0.104.60:42087
W20251211 20:52:51.106488 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet a471a9accbfc4875bd1d7802f5620808 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a471a9accbfc4875bd1d7802f5620808 already in progress: creating tablet
I20251211 20:52:51.107511 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb (table default.Table_678702734654439b9b10658a0b6b3421 [id=b5b24783442f46f7a87c9758c1ed1397]) successfully deleted
W20251211 20:52:51.107870 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet aeff1fbabf7845dc9863f59f286ddcec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aeff1fbabf7845dc9863f59f286ddcec already in progress: creating tablet
I20251211 20:52:51.113497 3385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Bootstrap starting.
I20251211 20:52:51.121167 1017 tablet_service.cc:1552] Processing DeleteTablet for tablet 41627cb27be8443fa37e514f9b3825a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:36220
I20251211 20:52:51.125808 1166 tablet_bootstrap.cc:492] T 10746162da4640e6a2a891467690b770 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:51.126001 1166 ts_tablet_manager.cc:1403] T 10746162da4640e6a2a891467690b770 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.047s user 0.005s sys 0.001s
I20251211 20:52:51.126664 1166 raft_consensus.cc:359] T 10746162da4640e6a2a891467690b770 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.126837 1166 raft_consensus.cc:385] T 10746162da4640e6a2a891467690b770 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.126906 1166 raft_consensus.cc:740] T 10746162da4640e6a2a891467690b770 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.127094 1166 consensus_queue.cc:260] T 10746162da4640e6a2a891467690b770 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.127560 1166 ts_tablet_manager.cc:1434] T 10746162da4640e6a2a891467690b770 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251211 20:52:51.129211 1166 tablet_bootstrap.cc:492] T a471a9accbfc4875bd1d7802f5620808 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:51.130946 1166 tablet_bootstrap.cc:654] T a471a9accbfc4875bd1d7802f5620808 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.132352 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 22273287aa9245a2a66131fed2c8861c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.133028 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 3fc173c396ff4af5a8589cff18165e85 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:48 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.133396 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 903c7f4b312d4677bb35be1aaa46ad6b with delete_type TABLET_DATA_DELETED (replaced by tablet 200fbba33f8947cca99ddc81e37dfc28) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.137089 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 80df719919a44aba9b4e3c62d6830cf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.128593 2962 ts_tablet_manager.cc:1916] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:51.137857 3294 ts_tablet_manager.cc:1950] T f91487dd7e0f4c8b8483c2b83945da3b P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:51.138836 3294 tablet_replica.cc:333] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:51.139048 3294 raft_consensus.cc:2243] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:51.139194 3294 raft_consensus.cc:2272] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:51.139794 3294 ts_tablet_manager.cc:1916] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:51.143175 2962 ts_tablet_manager.cc:1929] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:51.143304 2962 log.cc:1199] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/3ecf0ea8ce19486c81d1d1a91d57a7ce
I20251211 20:52:51.143749 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 71dc8d20d93243e289330d7b8c635511 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.144443 2962 ts_tablet_manager.cc:1950] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:51.144865 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 148) took 2052 ms. Trace:
I20251211 20:52:51.145021 2962 rpcz_store.cc:276] 1211 20:52:49.092596 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.092636 (+ 40us) service_pool.cc:225] Handling call
1211 20:52:51.095228 (+2002592us) ts_tablet_manager.cc:1151] Deleting tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce
1211 20:52:51.095237 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:51.128292 (+ 33055us) spinlock_profiling.cc:243] Waited 31.9 ms on lock 0x60d000013028. stack: 00007f39ea6b1ee6 00007f39ea6b1ca0 00007f39e968d3ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f39ea42db9a 00007f39ea429146 00007f39ea520d45 00007f39ea52120d 00007f39ea525de1 00007f39ea525d20 00007f39ea520392 000000000097dcc6 000000000097dba1 0000000000a8db81
1211 20:52:51.140252 (+ 11960us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:51.143057 (+ 2805us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:51.144825 (+ 1768us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":73469312,"tablet-delete.queue_time_us":2002414}
I20251211 20:52:51.145849 2962 tablet_replica.cc:333] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:51.146215 2962 raft_consensus.cc:2243] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:51.146399 2962 raft_consensus.cc:2272] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:51.146956 2962 ts_tablet_manager.cc:1916] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:51.149842 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 4d2b7e2a0abe4592bc806d5ef3f894d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.150285 1168 consensus_queue.cc:260] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.150588 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 845b5ac4a5984c0cb4ab188925103dc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.151050 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 29463a8f71f943a0927942235af3c2da with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.152474 1168 ts_tablet_manager.cc:1434] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.057s user 0.002s sys 0.000s
I20251211 20:52:51.152776 1168 tablet_bootstrap.cc:492] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:51.152853 1166 tablet_bootstrap.cc:492] T a471a9accbfc4875bd1d7802f5620808 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:51.153020 1166 ts_tablet_manager.cc:1403] T a471a9accbfc4875bd1d7802f5620808 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.004s
I20251211 20:52:51.077416 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ce495233a5e4f3792faf44680df54ba with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.080435 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 59e955aa07c14fbbb347b88eca59a74e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.080581 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 260bdeb73439437cbf234d20f7264ab3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.085566 914 raft_consensus.cc:3060] T d69fb1f6d9594afcbfb844860b41a377 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.153746 1166 raft_consensus.cc:359] T a471a9accbfc4875bd1d7802f5620808 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.153939 1166 raft_consensus.cc:385] T a471a9accbfc4875bd1d7802f5620808 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.154017 1166 raft_consensus.cc:740] T a471a9accbfc4875bd1d7802f5620808 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.154224 1166 consensus_queue.cc:260] T a471a9accbfc4875bd1d7802f5620808 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.154424 1168 tablet_bootstrap.cc:654] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.154848 1166 ts_tablet_manager.cc:1434] T a471a9accbfc4875bd1d7802f5620808 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20251211 20:52:51.155153 1166 tablet_bootstrap.cc:492] T 6663f13691a24fb991b48f543d9de76b P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:51.155282 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 09cd11f8b23c4f8a98a2b2eb77bb7b8c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:48 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.155679 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 79cc81e91e1846038621af7a0a9f7a2e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.156106 749 tablet_service.cc:1552] Processing DeleteTablet for tablet c112714331bf4bd7a1ddcca17f338d4f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.156435 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 84b2a67c836844ceb3628cc3b7d8ba10 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.085687 908 raft_consensus.cc:3060] T 92efda2bd0c84d07a0af6b537c0cd882 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.088382 1160 tablet_bootstrap.cc:492] T 3621735fcb294e3a8c0180ba9ef86442 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:51.156940 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce (table default.Table_021d4fb1667a475d87424f23aa0d5e28 [id=6bbfcc2561f349519a69ae1f56016cc7]) successfully deleted
I20251211 20:52:51.157116 1160 ts_tablet_manager.cc:1403] T 3621735fcb294e3a8c0180ba9ef86442 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.123s user 0.003s sys 0.004s
I20251211 20:52:51.157932 1160 raft_consensus.cc:359] T 3621735fcb294e3a8c0180ba9ef86442 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.158105 1160 raft_consensus.cc:385] T 3621735fcb294e3a8c0180ba9ef86442 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.158167 1160 raft_consensus.cc:740] T 3621735fcb294e3a8c0180ba9ef86442 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.158370 1160 consensus_queue.cc:260] T 3621735fcb294e3a8c0180ba9ef86442 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.159104 887 tablet_service.cc:1552] Processing DeleteTablet for tablet e4b5a7553110424d901788c7a2ecdce6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.159904 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 28ddb8a59a214576a25d613d42c2fef5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.160611 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 877aa237fe43452cbdbb4dac46f613ae with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.088690 904 raft_consensus.cc:3060] T d391d4e6a101462d8d1220f1e17d0daa P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.100530 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 47779034ad89466e9e1801987128b4dd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.100674 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 5262c2241e524395a73c0eb368917169 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.102294 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ec261837cf04bf88a48ff8add1b0d66 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.106325 960 heartbeater.cc:344] Connected to a master server at 127.0.104.60:42087
I20251211 20:52:51.109797 877 tablet_service.cc:1552] Processing DeleteTablet for tablet c97d410272a947de8b5d7fb40fe4777d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.113425 883 tablet_service.cc:1552] Processing DeleteTablet for tablet bb4004212441473483a209d23c0800ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.113878 881 tablet_service.cc:1552] Processing DeleteTablet for tablet e6ba12f94f5f4d688f4222a4c2fb5436 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.125854 3322 raft_consensus.cc:493] T f25bde14792248b188660036f07ae902 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251211 20:52:51.162735 1809 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.1:55697 (request call id 161) took 5900 ms (client timeout 1000 ms). Trace:
I20251211 20:52:51.162731 3322 raft_consensus.cc:515] T f25bde14792248b188660036f07ae902 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.163357 3322 leader_election.cc:290] T f25bde14792248b188660036f07ae902 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:51.136101 894 tablet_service.cc:1552] Processing DeleteTablet for tablet f3855ffb51a74101b26fd28429e65bc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.147611 876 tablet_service.cc:1552] Processing DeleteTablet for tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.148188 892 tablet_service.cc:1552] Processing DeleteTablet for tablet aeff1fbabf7845dc9863f59f286ddcec with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.148850 882 tablet_service.cc:1552] Processing DeleteTablet for tablet a2058b67ddf1413db338bfa2f64e7a77 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:50110
W20251211 20:52:51.163691 1808 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.3:60211 (request call id 158) took 3953 ms (client timeout 1000 ms). Trace:
W20251211 20:52:51.164366 1810 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.1:55697 (request call id 162) took 4441 ms (client timeout 1000 ms). Trace:
I20251211 20:52:51.166414 1160 ts_tablet_manager.cc:1434] T 3621735fcb294e3a8c0180ba9ef86442 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.009s user 0.000s sys 0.005s
I20251211 20:52:51.166756 1160 tablet_bootstrap.cc:492] T 06b8f8e46e524545aeab6c931efaf116 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:51.167693 882 tablet_service.cc:1552] Processing DeleteTablet for tablet 4df68d1b6a91458e9f212ae81aa25438 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.167927 892 tablet_service.cc:1552] Processing DeleteTablet for tablet a471a9accbfc4875bd1d7802f5620808 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.169162 1160 tablet_bootstrap.cc:654] T 06b8f8e46e524545aeab6c931efaf116 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.169422 892 tablet_service.cc:1552] Processing DeleteTablet for tablet 50f9f29c064c4f938a0f20f302ab5e8a with delete_type TABLET_DATA_DELETED (replaced by tablet da81bf8854a24954ba4548185812295d) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.169632 882 tablet_service.cc:1552] Processing DeleteTablet for tablet c74427cb9d6f4a679e43eae796ff8c3f with delete_type TABLET_DATA_DELETED (replaced by tablet 4e123b96b7534f07ae48a808f47e21a1) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.170560 1166 tablet_bootstrap.cc:654] T 6663f13691a24fb991b48f543d9de76b P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:51.163800 1808 rpcz_store.cc:269] 1211 20:52:47.209298 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:47.351534 (+142236us) service_pool.cc:225] Handling call
1211 20:52:47.840425 (+488891us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.840433 (+ 8us) write_op.cc:454] Released schema lock
1211 20:52:51.162654 (+3322221us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:47.833248 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:47.833387 (+ 139us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:47.833396 (+ 9us) write_op.cc:435] Acquired schema lock
1211 20:52:47.833399 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:47.833508 (+ 109us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:47.833523 (+ 15us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:47.833527 (+ 4us) tablet.cc:683] Acquiring locks for 1 operations
1211 20:52:47.833610 (+ 83us) tablet.cc:699] Row locks acquired
1211 20:52:47.833612 (+ 2us) write_op.cc:260] PREPARE: finished
1211 20:52:47.833654 (+ 42us) write_op.cc:270] Start()
1211 20:52:47.833690 (+ 36us) write_op.cc:276] Timestamp: P: 1765486367833604 usec, L: 0
1211 20:52:47.833693 (+ 3us) op_driver.cc:348] REPLICATION: starting
1211 20:52:47.833898 (+ 205us) log.cc:844] Serialized 527 byte log entry
1211 20:52:47.838244 (+ 4346us) op_driver.cc:464] REPLICATION: finished
1211 20:52:47.838384 (+ 140us) write_op.cc:301] APPLY: starting
1211 20:52:47.838401 (+ 17us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:47.838449 (+ 48us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:47.838489 (+ 40us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:47.838552 (+ 63us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:47.838599 (+ 47us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1211 20:52:47.838606 (+ 7us) write_op.cc:312] APPLY: finished
1211 20:52:47.838697 (+ 91us) log.cc:844] Serialized 23 byte log entry
1211 20:52:47.838764 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.838786 (+ 22us) write_op.cc:454] Released schema lock
1211 20:52:47.838793 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":120828,"reported_tablets":408,"child_traces":[["op",{"apply.queue_time_us":86,"apply.run_cpu_time_us":466,"apply.run_wall_time_us":465,"num_ops":1,"prepare.queue_time_us":769,"prepare.run_cpu_time_us":776,"prepare.run_wall_time_us":776,"raft.queue_time_us":975,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":294,"replication_time_us":4519}]]}
I20251211 20:52:51.172143 882 tablet_service.cc:1552] Processing DeleteTablet for tablet 2197822e0f11484f9679c4858626ac2c with delete_type TABLET_DATA_DELETED (replaced by tablet 16d2ff8925e342729a71c93db378d031) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.175184 882 tablet_service.cc:1552] Processing DeleteTablet for tablet 042f8bf13a7a4712b5bdc04671cd9938 with delete_type TABLET_DATA_DELETED (replaced by tablet 53b4116db0a44dfbb4c1ec5b2acebc67) from {username='slave'} at 127.0.0.1:50110
W20251211 20:52:51.164870 1810 rpcz_store.cc:269] 1211 20:52:46.723215 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:47.342225 (+619010us) service_pool.cc:225] Handling call
1211 20:52:51.164330 (+3822105us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:47.581839 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:47.581951 (+ 112us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:47.581956 (+ 5us) write_op.cc:435] Acquired schema lock
1211 20:52:47.581958 (+ 2us) tablet.cc:660] Decoding operations
1211 20:52:47.582195 (+ 237us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:47.582208 (+ 13us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:47.582213 (+ 5us) tablet.cc:683] Acquiring locks for 11 operations
1211 20:52:47.582463 (+ 250us) tablet.cc:699] Row locks acquired
1211 20:52:47.582466 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:47.582501 (+ 35us) write_op.cc:270] Start()
1211 20:52:47.582531 (+ 30us) write_op.cc:276] Timestamp: P: 1765486367582477 usec, L: 0
1211 20:52:47.582535 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:47.582709 (+ 174us) log.cc:844] Serialized 4268 byte log entry
1211 20:52:47.669995 (+ 87286us) op_driver.cc:464] REPLICATION: finished
1211 20:52:47.670303 (+ 308us) write_op.cc:301] APPLY: starting
1211 20:52:47.670323 (+ 20us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:47.670400 (+ 77us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:47.670431 (+ 31us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:47.670600 (+ 169us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:47.670642 (+ 42us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
1211 20:52:47.670650 (+ 8us) write_op.cc:312] APPLY: finished
1211 20:52:47.670761 (+ 111us) log.cc:844] Serialized 83 byte log entry
1211 20:52:47.670833 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.670865 (+ 32us) write_op.cc:454] Released schema lock
1211 20:52:47.670872 (+ 7us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:47.670955 (+ 83us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.670958 (+ 3us) write_op.cc:454] Released schema lock
Related trace 'op':
1211 20:52:48.763338 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:48.763479 (+ 141us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:48.763486 (+ 7us) write_op.cc:435] Acquired schema lock
1211 20:52:48.763489 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:48.763621 (+ 132us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:48.763637 (+ 16us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:48.763643 (+ 6us) tablet.cc:683] Acquiring locks for 1 operations
1211 20:52:48.763717 (+ 74us) tablet.cc:699] Row locks acquired
1211 20:52:48.763720 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:48.763763 (+ 43us) write_op.cc:270] Start()
1211 20:52:48.763801 (+ 38us) write_op.cc:276] Timestamp: P: 1765486368763715 usec, L: 0
1211 20:52:48.763805 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:48.764042 (+ 237us) log.cc:844] Serialized 356 byte log entry
1211 20:52:48.768131 (+ 4089us) op_driver.cc:464] REPLICATION: finished
1211 20:52:48.768274 (+ 143us) write_op.cc:301] APPLY: starting
1211 20:52:48.768291 (+ 17us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:48.768339 (+ 48us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:48.768373 (+ 34us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:48.768434 (+ 61us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:48.768468 (+ 34us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1211 20:52:48.768474 (+ 6us) write_op.cc:312] APPLY: finished
1211 20:52:48.768555 (+ 81us) log.cc:844] Serialized 23 byte log entry
1211 20:52:48.768621 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.768643 (+ 22us) write_op.cc:454] Released schema lock
1211 20:52:48.768650 (+ 7us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:48.768726 (+ 76us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.768729 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":57730,"reported_tablets":409,"spinlock_wait_cycles":896,"child_traces":[["op",{"apply.queue_time_us":235,"apply.run_cpu_time_us":644,"apply.run_wall_time_us":644,"num_ops":11,"prepare.queue_time_us":43,"prepare.run_cpu_time_us":984,"prepare.run_wall_time_us":984,"raft.queue_time_us":777,"raft.run_cpu_time_us":8,"raft.run_wall_time_us":8,"replication_time_us":87424}],["op",{"apply.queue_time_us":91,"apply.run_cpu_time_us":442,"apply.run_wall_time_us":441,"num_ops":1,"prepare.queue_time_us":55,"prepare.run_cpu_time_us":833,"prepare.run_wall_time_us":832,"raft.queue_time_us":915,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":244,"replication_time_us":4293}]]}
I20251211 20:52:51.181429 907 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7b06cd366de5494780b5ac81abbf0702" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:51.182297 911 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7b06cd366de5494780b5ac81abbf0702" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:51.183272 903 raft_consensus.cc:3060] T 18b977f463d5420186c007b38110e07e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.185174 749 tablet_service.cc:1552] Processing DeleteTablet for tablet de508d79f30f489cb745401f53115165 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.186350 1160 tablet_bootstrap.cc:492] T 06b8f8e46e524545aeab6c931efaf116 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:51.186494 1160 ts_tablet_manager.cc:1403] T 06b8f8e46e524545aeab6c931efaf116 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.003s
I20251211 20:52:51.187276 1160 raft_consensus.cc:359] T 06b8f8e46e524545aeab6c931efaf116 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.187467 1160 raft_consensus.cc:385] T 06b8f8e46e524545aeab6c931efaf116 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.187515 1160 raft_consensus.cc:740] T 06b8f8e46e524545aeab6c931efaf116 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.187773 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 98d575ac680d49d0990877666638f013 with delete_type TABLET_DATA_DELETED (replaced by tablet 834b84a557b04e78b5d2c8c84520a644) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.187793 1160 consensus_queue.cc:260] T 06b8f8e46e524545aeab6c931efaf116 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.188256 749 tablet_service.cc:1552] Processing DeleteTablet for tablet be876471f8bf453aa4467c91891cbbde with delete_type TABLET_DATA_DELETED (replaced by tablet 7d4e63f54d8443ae8322d4659a165e76) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.188493 1160 ts_tablet_manager.cc:1434] T 06b8f8e46e524545aeab6c931efaf116 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:51.188635 749 tablet_service.cc:1552] Processing DeleteTablet for tablet f2ae084982f44fdf8bfecdd2bc4c01b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.188920 1160 tablet_bootstrap.cc:492] T a7979798e92e45eb969197c8c7ac2580 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:51.188961 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 79d92736bb1c46d2bfa5678b2d396ec2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.189255 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 5f2c14c7a972414e8328e0b388bde0a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.189536 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 75dd170c5828409fa69bd68fdf28a72b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.190980 1160 tablet_bootstrap.cc:654] T a7979798e92e45eb969197c8c7ac2580 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.190979 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 3f1840e4b5214ddb878935a79b214eae with delete_type TABLET_DATA_DELETED (replaced by tablet 4fc56df5ccb94198b5b0e516eeae6941) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.191270 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 66cdc6f297b24cfba3a3a0ae9f45d12d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.191639 749 tablet_service.cc:1552] Processing DeleteTablet for tablet bbe3529c301a4a65b58b634a443e1fb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.191962 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b07f110f25645799ed0c89737cd8279 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.192262 1166 tablet_bootstrap.cc:492] T 6663f13691a24fb991b48f543d9de76b P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:51.192260 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 757ec913dff840f992a41a38b05f8d41 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.192413 1166 ts_tablet_manager.cc:1403] T 6663f13691a24fb991b48f543d9de76b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.037s user 0.003s sys 0.000s
I20251211 20:52:51.192539 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 116b21a182474ca191c4a89e9474677d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.192993 745 tablet_service.cc:1552] Processing DeleteTablet for tablet 062f980b65504d3baf99ae400e521775 with delete_type TABLET_DATA_DELETED (replaced by tablet 33fd09ea06ee47908b85911e4ee79708) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.193292 1048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f25bde14792248b188660036f07ae902" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:51.193557 748 tablet_service.cc:1552] Processing DeleteTablet for tablet d3b383ace72748d4bf21678894815c4f with delete_type TABLET_DATA_DELETED (replaced by tablet 745ce63314834c8abdecaa367133a068) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.193630 754 tablet_service.cc:1552] Processing DeleteTablet for tablet 2207af105bc3471ca930d35a91d38e70 with delete_type TABLET_DATA_DELETED (replaced by tablet 8675e2a18f0f4a0e95a87005cf82d8b4) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.188481 3295 log.cc:1199] T f22e4d34cde24b37a025c1b5a1046b1b P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/f22e4d34cde24b37a025c1b5a1046b1b
I20251211 20:52:51.193766 741 tablet_service.cc:1552] Processing DeleteTablet for tablet c549c2bfebd64b87a4fc8ce8210e14f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.193976 739 tablet_service.cc:1552] Processing DeleteTablet for tablet f25bde14792248b188660036f07ae902 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.194005 748 tablet_service.cc:1552] Processing DeleteTablet for tablet 8308ad46551447439a248d20ed1e5922 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.194118 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 993616e226274c229a5924fbe69911af with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.194147 3295 ts_tablet_manager.cc:1950] T f22e4d34cde24b37a025c1b5a1046b1b P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:51.194181 748 tablet_service.cc:1552] Processing DeleteTablet for tablet a56f1a7bd5464d289bfd018c826ee574 with delete_type TABLET_DATA_DELETED (replaced by tablet 284e34970f634abcb2288c864f73e963) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.194435 748 tablet_service.cc:1552] Processing DeleteTablet for tablet 2630ad57845a4c31b20c8a104dd27335 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.194550 746 tablet_service.cc:1552] Processing DeleteTablet for tablet b61c4f0cb15940a988c5acb6879d265a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.194698 740 tablet_service.cc:1552] Processing DeleteTablet for tablet 12aaf618d6474f599458026308d7afdd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.194815 741 tablet_service.cc:1552] Processing DeleteTablet for tablet c8e17645a26947ab8c01f9b093fa422c with delete_type TABLET_DATA_DELETED (replaced by tablet 171d40a8a9124f9bb1b38797fef8e5d5) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.194877 3295 tablet_replica.cc:333] T 0aac6fb3972e4aba8950828c9e49d507 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:51.194833 1783 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet f22e4d34cde24b37a025c1b5a1046b1b (table default.Table_60e84f16bbb3462682f8bdaffd7d1542 [id=981d70bcd97a42bc8a5b7cd3d0e5f0e0]) successfully deleted
I20251211 20:52:51.194914 754 tablet_service.cc:1552] Processing DeleteTablet for tablet c7bd218b3ff340b2a589c9517b49a649 with delete_type TABLET_DATA_DELETED (replaced by tablet dd1ae033dedc436fa3a3f864a7a208a2) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.195031 745 tablet_service.cc:1552] Processing DeleteTablet for tablet c91ff29847d54e55b19321652a857e9f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.195065 3295 raft_consensus.cc:2243] T 0aac6fb3972e4aba8950828c9e49d507 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:51.195230 3295 raft_consensus.cc:2272] T 0aac6fb3972e4aba8950828c9e49d507 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:51.195235 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 9b99860ee6be45a0b358e686332d3137 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9b99860ee6be45a0b358e686332d3137 already in progress: creating tablet
I20251211 20:52:51.195425 749 tablet_service.cc:1552] Processing DeleteTablet for tablet 683d0f2068fb4a6ebc9c169ad7b455ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:56136
I20251211 20:52:51.195812 3295 ts_tablet_manager.cc:1916] T 0aac6fb3972e4aba8950828c9e49d507 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:51.195757 1166 raft_consensus.cc:359] T 6663f13691a24fb991b48f543d9de76b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.196156 1166 raft_consensus.cc:385] T 6663f13691a24fb991b48f543d9de76b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.196259 1166 raft_consensus.cc:740] T 6663f13691a24fb991b48f543d9de76b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.196692 1166 consensus_queue.cc:260] T 6663f13691a24fb991b48f543d9de76b P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.196895 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet f22e4d34cde24b37a025c1b5a1046b1b (table default.Table_60e84f16bbb3462682f8bdaffd7d1542 [id=981d70bcd97a42bc8a5b7cd3d0e5f0e0]) successfully deleted
I20251211 20:52:51.197405 1166 ts_tablet_manager.cc:1434] T 6663f13691a24fb991b48f543d9de76b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251211 20:52:51.197781 1166 tablet_bootstrap.cc:492] T 69085cd41db846e7a6e03bcfee4377f6 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:51.197894 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 2ef524747034491c99f4f9767acd3c9d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2ef524747034491c99f4f9767acd3c9d already in progress: creating tablet
I20251211 20:52:51.199707 1166 tablet_bootstrap.cc:654] T 69085cd41db846e7a6e03bcfee4377f6 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.200340 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 0aac6fb3972e4aba8950828c9e49d507 (table default.Table_bf722f720fc64f2c875ab1edbcaebaf9 [id=fdebf1e48cc24020867e6b24ef2b55b4]) successfully deleted
W20251211 20:52:51.208674 982 leader_election.cc:343] T 29463a8f71f943a0927942235af3c2da P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.208837 982 leader_election.cc:304] T 29463a8f71f943a0927942235af3c2da P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:51.209403 3269 raft_consensus.cc:2749] T 29463a8f71f943a0927942235af3c2da P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:51.214519 1166 tablet_bootstrap.cc:492] T 69085cd41db846e7a6e03bcfee4377f6 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:51.214859 1166 ts_tablet_manager.cc:1403] T 69085cd41db846e7a6e03bcfee4377f6 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.006s
I20251211 20:52:51.215725 1166 raft_consensus.cc:359] T 69085cd41db846e7a6e03bcfee4377f6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.216006 1166 raft_consensus.cc:385] T 69085cd41db846e7a6e03bcfee4377f6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.216126 1166 raft_consensus.cc:740] T 69085cd41db846e7a6e03bcfee4377f6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.216455 1166 consensus_queue.cc:260] T 69085cd41db846e7a6e03bcfee4377f6 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.217005 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f25bde14792248b188660036f07ae902" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:51.217212 1166 ts_tablet_manager.cc:1434] T 69085cd41db846e7a6e03bcfee4377f6 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:51.217582 1166 tablet_bootstrap.cc:492] T e95b0d8c507a4779a087c8933d464228 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:51.217706 711 leader_election.cc:343] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:51.218497 711 leader_election.cc:343] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:51.219146 711 consensus_peers.cc:597] T 6a206abd6ea2420089cb07714e0d19c0 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:51.162844 1809 rpcz_store.cc:269] 1211 20:52:45.260232 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:45.431073 (+170841us) service_pool.cc:225] Handling call
1211 20:52:45.468392 (+ 37319us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x616000040f54. stack: 00007ff715c18ee6 00007ff715c18ca0 00007ff714bf43ad 0000000000984f94 0000000000984f5d 00007ff724f1f6bd 00007ff724f1f7b3 00007ff721ae6521 00007ff7297a596d 00007ff72992ca08 00007ff720439ca6 00007ff7204399d5 00007ff71cbb856e 00007ff71cbb744e 00007ff71cbbd685 00007ff71cbbdfaf
1211 20:52:51.160938 (+5692546us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:52:47.410216 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:47.410373 (+ 157us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:47.410381 (+ 8us) write_op.cc:435] Acquired schema lock
1211 20:52:47.410384 (+ 3us) tablet.cc:660] Decoding operations
1211 20:52:47.410497 (+ 113us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:47.410514 (+ 17us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:47.410520 (+ 6us) tablet.cc:683] Acquiring locks for 1 operations
1211 20:52:47.410598 (+ 78us) tablet.cc:699] Row locks acquired
1211 20:52:47.410601 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:52:47.410646 (+ 45us) write_op.cc:270] Start()
1211 20:52:47.410683 (+ 37us) write_op.cc:276] Timestamp: P: 1765486367410578 usec, L: 0
1211 20:52:47.410686 (+ 3us) op_driver.cc:348] REPLICATION: starting
1211 20:52:47.410904 (+ 218us) log.cc:844] Serialized 527 byte log entry
1211 20:52:47.415798 (+ 4894us) op_driver.cc:464] REPLICATION: finished
1211 20:52:47.415932 (+ 134us) write_op.cc:301] APPLY: starting
1211 20:52:47.415951 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:47.415999 (+ 48us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:47.416037 (+ 38us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:47.416100 (+ 63us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:47.416137 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1211 20:52:47.416145 (+ 8us) write_op.cc:312] APPLY: finished
1211 20:52:47.416242 (+ 97us) log.cc:844] Serialized 23 byte log entry
1211 20:52:47.416311 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.416337 (+ 26us) write_op.cc:454] Released schema lock
1211 20:52:47.416345 (+ 8us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:47.416423 (+ 78us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:47.416427 (+ 4us) write_op.cc:454] Released schema lock
Related trace 'op':
1211 20:52:48.140368 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1211 20:52:48.140516 (+ 148us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:52:48.140523 (+ 7us) write_op.cc:435] Acquired schema lock
1211 20:52:48.140527 (+ 4us) tablet.cc:660] Decoding operations
1211 20:52:48.140658 (+ 131us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:52:48.140670 (+ 12us) write_op.cc:641] Partition lock acquired for write op
1211 20:52:48.140676 (+ 6us) tablet.cc:683] Acquiring locks for 1 operations
1211 20:52:48.140736 (+ 60us) tablet.cc:699] Row locks acquired
1211 20:52:48.140738 (+ 2us) write_op.cc:260] PREPARE: finished
1211 20:52:48.140776 (+ 38us) write_op.cc:270] Start()
1211 20:52:48.140814 (+ 38us) write_op.cc:276] Timestamp: P: 1765486368140742 usec, L: 0
1211 20:52:48.140818 (+ 4us) op_driver.cc:348] REPLICATION: starting
1211 20:52:48.141018 (+ 200us) log.cc:844] Serialized 356 byte log entry
1211 20:52:48.314741 (+173723us) op_driver.cc:464] REPLICATION: finished
1211 20:52:48.324930 (+ 10189us) write_op.cc:301] APPLY: starting
1211 20:52:48.324946 (+ 16us) tablet.cc:1365] starting BulkCheckPresence
1211 20:52:48.324995 (+ 49us) tablet.cc:1368] finished BulkCheckPresence
1211 20:52:48.325045 (+ 50us) tablet.cc:1370] starting ApplyRowOperation cycle
1211 20:52:48.325105 (+ 60us) tablet.cc:1381] finished ApplyRowOperation cycle
1211 20:52:48.325142 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1211 20:52:48.325149 (+ 7us) write_op.cc:312] APPLY: finished
1211 20:52:48.325256 (+ 107us) log.cc:844] Serialized 23 byte log entry
1211 20:52:48.325325 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.325345 (+ 20us) write_op.cc:454] Released schema lock
1211 20:52:48.325351 (+ 6us) write_op.cc:341] FINISH: Updating metrics
1211 20:52:48.325426 (+ 75us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:52:48.325431 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":83955,"reported_tablets":365,"spinlock_wait_cycles":64179584,"child_traces":[["op",{"apply.queue_time_us":82,"apply.run_cpu_time_us":481,"apply.run_wall_time_us":481,"num_ops":1,"prepare.queue_time_us":160,"prepare.run_cpu_time_us":817,"prepare.run_wall_time_us":817,"raft.queue_time_us":959,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":318,"replication_time_us":5075}],["op",{"apply.queue_time_us":10134,"apply.run_cpu_time_us":487,"apply.run_wall_time_us":486,"num_ops":1,"prepare.queue_time_us":85,"prepare.run_cpu_time_us":774,"prepare.run_wall_time_us":1203,"raft.queue_time_us":40,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":7,"replication_time_us":173888}]]}
I20251211 20:52:51.220407 1809 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_948151057bf94b75a251b99e3906f4a4" } new_table_name: "default.Table_968e5e9acbf341e9aa51d2b17bb5b847" modify_external_catalogs: true
I20251211 20:52:51.221583 1166 tablet_bootstrap.cc:654] T e95b0d8c507a4779a087c8933d464228 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.227638 1810 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_68f5e21962ce48b89834785bf8dd332f" } new_table_name: "default.Table_35c976fedbda4fd5b6b0e9fcc80a4223" modify_external_catalogs: true
I20251211 20:52:51.235525 1166 tablet_bootstrap.cc:492] T e95b0d8c507a4779a087c8933d464228 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:51.236481 1166 ts_tablet_manager.cc:1403] T e95b0d8c507a4779a087c8933d464228 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.000s
I20251211 20:52:51.243309 1166 raft_consensus.cc:359] T e95b0d8c507a4779a087c8933d464228 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.244686 1166 raft_consensus.cc:385] T e95b0d8c507a4779a087c8933d464228 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.245092 1166 raft_consensus.cc:740] T e95b0d8c507a4779a087c8933d464228 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.245692 1166 consensus_queue.cc:260] T e95b0d8c507a4779a087c8933d464228 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.246888 1166 ts_tablet_manager.cc:1434] T e95b0d8c507a4779a087c8933d464228 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20251211 20:52:51.253449 847 leader_election.cc:304] T f25bde14792248b188660036f07ae902 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:51.253805 1168 tablet_bootstrap.cc:492] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:51.253947 1168 ts_tablet_manager.cc:1403] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.101s user 0.003s sys 0.000s
I20251211 20:52:51.253979 3322 raft_consensus.cc:2749] T f25bde14792248b188660036f07ae902 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:51.254822 1168 raft_consensus.cc:359] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.255004 1168 raft_consensus.cc:385] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.255059 1168 raft_consensus.cc:740] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.255234 1168 consensus_queue.cc:260] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.258801 3391 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 11f5f94732f049d0be4c97cebfffcb05)
I20251211 20:52:51.259097 3391 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
I20251211 20:52:51.260988 1168 ts_tablet_manager.cc:1434] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20251211 20:52:51.261498 1168 tablet_bootstrap.cc:492] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:51.263200 1168 tablet_bootstrap.cc:654] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.264025 3391 leader_election.cc:290] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11f5f94732f049d0be4c97cebfffcb05 (127.0.104.61:40749), 9230f94a6a1a49cead51aeea9f069087 (127.0.104.60:42087)
W20251211 20:52:51.264837 710 consensus_peers.cc:597] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3621735fcb294e3a8c0180ba9ef86442. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:51.265023 710 consensus_peers.cc:597] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e9ead34065e240eeb1f946319537d32a. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:51.266743 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet f91487dd7e0f4c8b8483c2b83945da3b (table default.Table_70ec351f0d724592bcfb6bf8b469d84d [id=508da7ce4977453d848dc14a4d8f6758]) successfully deleted
I20251211 20:52:51.267310 1829 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1830b0c58cfe495989f3d911dda0409a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1298 } } ignore_live_leader: false dest_uuid: "11f5f94732f049d0be4c97cebfffcb05" is_pre_election: true
I20251211 20:52:51.272373 3269 raft_consensus.cc:493] T 200b6db115964f3c8882b514c794698d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:51.272615 3269 raft_consensus.cc:515] T 200b6db115964f3c8882b514c794698d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.273310 3269 leader_election.cc:290] T 200b6db115964f3c8882b514c794698d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:51.282083 2962 ts_tablet_manager.cc:1929] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:51.282361 2962 log.cc:1199] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/c8214c97ed1640f5965aa5c4b999e2f1
I20251211 20:52:51.283123 2962 ts_tablet_manager.cc:1950] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:51.307257 912 raft_consensus.cc:3060] T 41bf5b5111f64dfa91ed76ed4ab4f6f7 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.308101 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "200b6db115964f3c8882b514c794698d" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:51.311381 882 tablet_service.cc:1552] Processing DeleteTablet for tablet 88970b575ba943129729eceef5636519 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:50110
W20251211 20:52:51.308775 983 leader_election.cc:343] T 200b6db115964f3c8882b514c794698d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.309440 3269 raft_consensus.cc:493] T a471a9accbfc4875bd1d7802f5620808 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:51.313436 3269 raft_consensus.cc:515] T a471a9accbfc4875bd1d7802f5620808 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.314081 3269 leader_election.cc:290] T a471a9accbfc4875bd1d7802f5620808 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:51.314482 882 tablet_service.cc:1552] Processing DeleteTablet for tablet 7bc03ed8c35740f58fe48ddbaa120ea0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.314718 892 tablet_service.cc:1552] Processing DeleteTablet for tablet 4776c456b4d74c90b1fc71d57b62a461 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.315397 901 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a471a9accbfc4875bd1d7802f5620808" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:51.315990 892 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a206abd6ea2420089cb07714e0d19c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:50110
W20251211 20:52:51.316511 983 leader_election.cc:343] T a471a9accbfc4875bd1d7802f5620808 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.322923 892 tablet_service.cc:1552] Processing DeleteTablet for tablet e95b0d8c507a4779a087c8933d464228 with delete_type TABLET_DATA_DELETED (replaced by tablet 9c5c937cc7db4ed18fe03f4a8a99a66a) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.323261 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "062f980b65504d3baf99ae400e521775" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:51.323364 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8214c97ed1640f5965aa5c4b999e2f1" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:51.324318 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "51d12723f5194afb97d2c05fcbaf3776" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:51.324345 882 tablet_service.cc:1552] Processing DeleteTablet for tablet 07e4336171714da8adc21977ce33ce50 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.324345 876 tablet_service.cc:1552] Processing DeleteTablet for tablet 80c566822fa94ada983aa7266df4bf9d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.324486 775 raft_consensus.cc:2468] T 51d12723f5194afb97d2c05fcbaf3776 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 0.
I20251211 20:52:51.324580 892 tablet_service.cc:1552] Processing DeleteTablet for tablet ca89f09cc0954389b679e8bb8b5e4dfb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.325325 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "200b6db115964f3c8882b514c794698d" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:51.326169 3174 raft_consensus.cc:493] T d69fb1f6d9594afcbfb844860b41a377 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6f8d784fd1cd492bbe3809403ec34919)
I20251211 20:52:51.326313 3174 raft_consensus.cc:515] T d69fb1f6d9594afcbfb844860b41a377 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.327263 876 tablet_service.cc:1552] Processing DeleteTablet for tablet c7147571483e41ad970e573a08c368ca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.327479 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 516f7181aaac48cdadf9fbc08d0d3f57 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.328107 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 1944267def4e46949fd680ab0a369be9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.328284 891 tablet_service.cc:1552] Processing DeleteTablet for tablet fb030a62796248869d46b682656eaeb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:26 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.328442 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 2d849c62f7434099a16f62c2a2fc5e85 with delete_type TABLET_DATA_DELETED (replaced by tablet 3dfea28c44d84c68850db65a30d52ee6) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.328555 891 tablet_service.cc:1552] Processing DeleteTablet for tablet e95c63b4d99a4dd0b2c0fa384ca9d552 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.328686 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 32f8d104f6a948d1a97a83769ea82449 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.328809 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 14e2650f8ec1482ea9511b989fdc963c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.329017 889 tablet_service.cc:1552] Processing DeleteTablet for tablet c26c3d0170f64a41ae8944c6668cc5f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.329223 889 tablet_service.cc:1552] Processing DeleteTablet for tablet 793f543d6b614ad7b3ab8e717db2ceaf with delete_type TABLET_DATA_DELETED (replaced by tablet 06b8f8e46e524545aeab6c931efaf116) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.329288 881 tablet_service.cc:1552] Processing DeleteTablet for tablet 7d4e63f54d8443ae8322d4659a165e76 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.329455 883 tablet_service.cc:1552] Processing DeleteTablet for tablet 6d4e6528352f43b6a1b47d7a4b6b3689 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.329630 877 tablet_service.cc:1552] Processing DeleteTablet for tablet 215f8dc74a71482daa2ca80e15010f51 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.329768 887 tablet_service.cc:1552] Processing DeleteTablet for tablet d53b074865f4490693152a247b0fffdc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.329906 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 6029c1fd563d4023a8cf7cfb8ffa0876 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.330031 875 tablet_service.cc:1552] Processing DeleteTablet for tablet 41c30bded6504bbc9f0af8f34dc81e8d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.330274 1166 tablet_bootstrap.cc:492] T b115d737d7fc483d9f5a1f040b511bae P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:51.330302 888 tablet_service.cc:1552] Processing DeleteTablet for tablet b30d2054316b49fa8096345527a45d87 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.330435 875 tablet_service.cc:1552] Processing DeleteTablet for tablet 2e33606930774fe4aafc6e3819618af8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.330715 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c5c937cc7db4ed18fe03f4a8a99a66a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.330861 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 6ad6ab53aad14cc2b6d201acfecf6723 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.330977 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 4201da4241aa48dfb670a713e4453ca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.331130 875 tablet_service.cc:1552] Processing DeleteTablet for tablet 58e35de0792a464a818170886ca8d8c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.331430 882 tablet_service.cc:1552] Processing DeleteTablet for tablet a295a24e81f14e5fa126a4a58ffd08cf with delete_type TABLET_DATA_DELETED (replaced by tablet 77bd176108f24e79a632c798566e9e31) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.331626 875 tablet_service.cc:1552] Processing DeleteTablet for tablet 3acfef6f082641a199b80eefcfc772ac with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.331830 876 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a61cf8f5de14adaa4a1de5bddde9a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.332201 1166 tablet_bootstrap.cc:654] T b115d737d7fc483d9f5a1f040b511bae P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.332229 892 tablet_service.cc:1552] Processing DeleteTablet for tablet 14f367adb60744c8a593d3b9d4711823 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.332480 876 tablet_service.cc:1552] Processing DeleteTablet for tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.332662 875 tablet_service.cc:1552] Processing DeleteTablet for tablet efe4d6bb07374294b8ab50a9fe605f87 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.332962 875 tablet_service.cc:1552] Processing DeleteTablet for tablet bb112b6431af42a6a7b328dfdefbc2a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.333278 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 149) took 2240 ms. Trace:
I20251211 20:52:51.333477 875 tablet_service.cc:1552] Processing DeleteTablet for tablet e28d66ce4711430cb09a9650e99fbe9e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.333482 2962 rpcz_store.cc:276] 1211 20:52:49.092993 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.097297 (+ 4304us) service_pool.cc:225] Handling call
1211 20:52:51.145463 (+2048166us) ts_tablet_manager.cc:1151] Deleting tablet c8214c97ed1640f5965aa5c4b999e2f1
1211 20:52:51.145767 (+ 304us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:51.204529 (+ 58762us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:51.281910 (+ 77381us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:51.333230 (+ 51320us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2047927}
I20251211 20:52:51.333889 2962 tablet_replica.cc:333] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:51.333985 875 tablet_service.cc:1552] Processing DeleteTablet for tablet 2e29df291d8d40daba83e1d9f97f902b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.334185 2962 raft_consensus.cc:2243] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:51.334215 876 tablet_service.cc:1552] Processing DeleteTablet for tablet 0f8808bfd68841e4ae30818018040cf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.334405 2962 raft_consensus.cc:2272] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:51.334540 876 tablet_service.cc:1552] Processing DeleteTablet for tablet bb9fccb6981e446a939a28d7fdd38e25 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.334923 876 tablet_service.cc:1552] Processing DeleteTablet for tablet 449921ea74f04a9c88eb9e094e8226c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.335065 2962 ts_tablet_manager.cc:1916] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:51.335299 876 tablet_service.cc:1552] Processing DeleteTablet for tablet f91487dd7e0f4c8b8483c2b83945da3b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.335781 876 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ef524747034491c99f4f9767acd3c9d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.340097 876 tablet_service.cc:1552] Processing DeleteTablet for tablet f22e4d34cde24b37a025c1b5a1046b1b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.340564 876 tablet_service.cc:1552] Processing DeleteTablet for tablet 9b99860ee6be45a0b358e686332d3137 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.340977 876 tablet_service.cc:1552] Processing DeleteTablet for tablet 12aaf618d6474f599458026308d7afdd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.341456 876 tablet_service.cc:1552] Processing DeleteTablet for tablet b61c4f0cb15940a988c5acb6879d265a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.341744 875 tablet_service.cc:1552] Processing DeleteTablet for tablet d3b383ace72748d4bf21678894815c4f with delete_type TABLET_DATA_DELETED (replaced by tablet 745ce63314834c8abdecaa367133a068) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.342116 875 tablet_service.cc:1552] Processing DeleteTablet for tablet ccc3ddd4b7f74f11a3835997cb7ae8d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.342587 875 tablet_service.cc:1552] Processing DeleteTablet for tablet 5710fc2f24764f818f7e5c86c6fd985e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.343032 875 tablet_service.cc:1552] Processing DeleteTablet for tablet b529b38d754f46ba8cb60b19e9705256 with delete_type TABLET_DATA_DELETED (replaced by tablet 84bb3583857848459e0e865283de623d) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.343675 875 tablet_service.cc:1552] Processing DeleteTablet for tablet 7452dd736b0440fcb4626858e794676f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.344308 875 tablet_service.cc:1552] Processing DeleteTablet for tablet 7d1cc9809ab843e693a179a35c53ce25 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.344827 875 tablet_service.cc:1552] Processing DeleteTablet for tablet 098054406a1d459b824a094f50e71e7e with delete_type TABLET_DATA_DELETED (replaced by tablet aa50ddb2789b4be3bfa9aedef9287109) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.345528 875 tablet_service.cc:1552] Processing DeleteTablet for tablet cc43ee61e78249bfa00b11f3a1fa21a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.350209 823 heartbeater.cc:344] Connected to a master server at 127.0.104.60:42087
I20251211 20:52:51.352419 1166 tablet_bootstrap.cc:492] T b115d737d7fc483d9f5a1f040b511bae P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:51.352595 1166 ts_tablet_manager.cc:1403] T b115d737d7fc483d9f5a1f040b511bae P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.000s
I20251211 20:52:51.353389 1166 raft_consensus.cc:359] T b115d737d7fc483d9f5a1f040b511bae P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.353607 1166 raft_consensus.cc:385] T b115d737d7fc483d9f5a1f040b511bae P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.353713 1166 raft_consensus.cc:740] T b115d737d7fc483d9f5a1f040b511bae P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.354545 1166 consensus_queue.cc:260] T b115d737d7fc483d9f5a1f040b511bae P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.355221 3322 raft_consensus.cc:493] T 95866de91f9d437d80f40b7962253610 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:51.355394 1166 ts_tablet_manager.cc:1434] T b115d737d7fc483d9f5a1f040b511bae P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20251211 20:52:51.355387 3322 raft_consensus.cc:515] T 95866de91f9d437d80f40b7962253610 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.355765 1166 tablet_bootstrap.cc:492] T 1c4b2c22bd4c456eb77af45576de466f P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:51.355769 3174 leader_election.cc:290] T d69fb1f6d9594afcbfb844860b41a377 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:51.357800 1166 tablet_bootstrap.cc:654] T 1c4b2c22bd4c456eb77af45576de466f P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.361231 3322 leader_election.cc:290] T 95866de91f9d437d80f40b7962253610 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:51.369737 876 tablet_service.cc:1552] Processing DeleteTablet for tablet 4810f8d8b53a45eb939bf3c8bf1b4e77 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.374379 891 tablet_service.cc:1552] Processing DeleteTablet for tablet d15eff49af284a38bb1babdb231a35df with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.374859 882 tablet_service.cc:1552] Processing DeleteTablet for tablet eb9da081cb9d43fd9a35c4bc51ea133e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.376467 880 tablet_service.cc:1552] Processing DeleteTablet for tablet f25bde14792248b188660036f07ae902 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.377063 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a471a9accbfc4875bd1d7802f5620808" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:51.378778 1160 tablet_bootstrap.cc:492] T a7979798e92e45eb969197c8c7ac2580 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:51.378841 892 tablet_service.cc:1552] Processing DeleteTablet for tablet 4a39870c7eea4a379b284ea2e5ddf760 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.378790 875 tablet_service.cc:1552] Processing DeleteTablet for tablet 2dde7f28e61d41f48015ba6e4f3757f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.379057 1160 ts_tablet_manager.cc:1403] T a7979798e92e45eb969197c8c7ac2580 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.190s user 0.012s sys 0.003s
I20251211 20:52:51.379280 876 tablet_service.cc:1552] Processing DeleteTablet for tablet c8214c97ed1640f5965aa5c4b999e2f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.379457 882 tablet_service.cc:1552] Processing DeleteTablet for tablet 01104975fd8b470aa026578b97ff6b8d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.379504 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 993616e226274c229a5924fbe69911af with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.379662 894 tablet_service.cc:1552] Processing DeleteTablet for tablet c91ff29847d54e55b19321652a857e9f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:37 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.379813 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 683d0f2068fb4a6ebc9c169ad7b455ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:44 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.381084 1160 raft_consensus.cc:359] T a7979798e92e45eb969197c8c7ac2580 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.381590 1160 raft_consensus.cc:385] T a7979798e92e45eb969197c8c7ac2580 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.381831 1160 raft_consensus.cc:740] T a7979798e92e45eb969197c8c7ac2580 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.382228 3269 raft_consensus.cc:493] T 10746162da4640e6a2a891467690b770 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:51.382414 3269 raft_consensus.cc:515] T 10746162da4640e6a2a891467690b770 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.382333 1160 consensus_queue.cc:260] T a7979798e92e45eb969197c8c7ac2580 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.383078 3269 leader_election.cc:290] T 10746162da4640e6a2a891467690b770 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:51.383157 3322 raft_consensus.cc:493] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:51.383320 3322 raft_consensus.cc:515] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.383911 3322 leader_election.cc:290] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:51.384346 914 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "10746162da4640e6a2a891467690b770" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:51.384891 888 tablet_service.cc:1552] Processing DeleteTablet for tablet c549c2bfebd64b87a4fc8ce8210e14f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:50110
W20251211 20:52:51.385004 983 leader_election.cc:343] T 10746162da4640e6a2a891467690b770 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.386137 2962 ts_tablet_manager.cc:1929] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:51.389264 2962 log.cc:1199] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/a295a24e81f14e5fa126a4a58ffd08cf
I20251211 20:52:51.389818 2962 ts_tablet_manager.cc:1950] T a295a24e81f14e5fa126a4a58ffd08cf P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:51.390789 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 150) took 2292 ms. Trace:
I20251211 20:52:51.393162 3294 ts_tablet_manager.cc:1929] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:51.392611 1166 tablet_bootstrap.cc:492] T 1c4b2c22bd4c456eb77af45576de466f P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:51.393463 3294 log.cc:1199] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/68792ccef54944a09af9b5456e9ff1fe
I20251211 20:52:51.393605 1166 ts_tablet_manager.cc:1403] T 1c4b2c22bd4c456eb77af45576de466f P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.038s user 0.004s sys 0.000s
I20251211 20:52:51.394167 3294 ts_tablet_manager.cc:1950] T 68792ccef54944a09af9b5456e9ff1fe P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:51.394618 1166 raft_consensus.cc:359] T 1c4b2c22bd4c456eb77af45576de466f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.394876 1166 raft_consensus.cc:385] T 1c4b2c22bd4c456eb77af45576de466f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.395030 1166 raft_consensus.cc:740] T 1c4b2c22bd4c456eb77af45576de466f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.395210 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet c8214c97ed1640f5965aa5c4b999e2f1 (table default.Table_60c2303340ad4078a0bf2c796be9faf3 [id=b3fb50681b1b47d3b2c158fee448724b]) successfully deleted
I20251211 20:52:51.395306 1166 consensus_queue.cc:260] T 1c4b2c22bd4c456eb77af45576de466f P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.396072 1166 ts_tablet_manager.cc:1434] T 1c4b2c22bd4c456eb77af45576de466f P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:51.396445 1035 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d69fb1f6d9594afcbfb844860b41a377" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:51.396733 1036 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "95866de91f9d437d80f40b7962253610" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:51.397011 1166 tablet_bootstrap.cc:492] T a56f1a7bd5464d289bfd018c826ee574 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:51.398363 982 leader_election.cc:343] T 062f980b65504d3baf99ae400e521775 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.398312 2962 rpcz_store.cc:276] 1211 20:52:49.097818 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.106582 (+ 8764us) service_pool.cc:225] Handling call
1211 20:52:51.333809 (+2227227us) ts_tablet_manager.cc:1151] Deleting tablet a295a24e81f14e5fa126a4a58ffd08cf
1211 20:52:51.333818 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:51.342269 (+ 8451us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:51.385989 (+ 43720us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:51.390719 (+ 4730us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2218077}
I20251211 20:52:51.401665 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 152) took 2291 ms. Trace:
I20251211 20:52:51.404799 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d69fb1f6d9594afcbfb844860b41a377" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:51.405580 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "95866de91f9d437d80f40b7962253610" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:51.410202 2962 rpcz_store.cc:276] 1211 20:52:49.107816 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.107861 (+ 45us) service_pool.cc:225] Handling call
1211 20:52:51.398917 (+2291056us) ts_tablet_manager.cc:1151] Deleting tablet 6d4e6528352f43b6a1b47d7a4b6b3689
1211 20:52:51.398928 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:51.399024 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2279328}
I20251211 20:52:51.417066 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 154) took 2308 ms. Trace:
I20251211 20:52:51.417955 2962 rpcz_store.cc:276] 1211 20:52:49.108188 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.108237 (+ 49us) service_pool.cc:225] Handling call
1211 20:52:51.416828 (+2308591us) ts_tablet_manager.cc:1151] Deleting tablet 1944267def4e46949fd680ab0a369be9
1211 20:52:51.416841 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:51.417006 (+ 165us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2297139}
I20251211 20:52:51.420784 903 raft_consensus.cc:3060] T 47779034ad89466e9e1801987128b4dd P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.417326 1166 tablet_bootstrap.cc:654] T a56f1a7bd5464d289bfd018c826ee574 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.426036 2962 tablet_replica.cc:333] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:51.426385 2962 raft_consensus.cc:2243] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:51.426627 2962 raft_consensus.cc:2272] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:51.427311 2962 ts_tablet_manager.cc:1916] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:51.433454 3269 raft_consensus.cc:493] T c549c2bfebd64b87a4fc8ce8210e14f6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:51.433604 3269 raft_consensus.cc:515] T c549c2bfebd64b87a4fc8ce8210e14f6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.434226 3269 leader_election.cc:290] T c549c2bfebd64b87a4fc8ce8210e14f6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:51.398638 982 leader_election.cc:304] T 062f980b65504d3baf99ae400e521775 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:51.441175 982 leader_election.cc:343] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.441103 749 tablet_service.cc:1505] Processing CreateTablet for tablet 3c03784820114f5aae409323899209ed (DEFAULT_TABLE table=default.Table_37d416a121e34cc8abb9520f5320dfbd [id=31d686268adc45979dad42439723e680]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:51.441295 982 leader_election.cc:304] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:51.441433 982 raft_consensus.cc:2686] T c8214c97ed1640f5965aa5c4b999e2f1 P e715a811a82446ce8d5bb4aafd5b81c7: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251211 20:52:51.442178 982 leader_election.cc:304] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:51.442229 754 tablet_service.cc:1505] Processing CreateTablet for tablet 0008c88c9c75424b92e115e2fbd0622f (DEFAULT_TABLE table=default.Table_187cac16164c4f23bdf2b49d86a68bde [id=bfff447263594ecbad74cc1e44e634a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:51.442876 741 tablet_service.cc:1505] Processing CreateTablet for tablet 657b1a60f9184449a333c278886fcdb2 (DEFAULT_TABLE table=default.Table_3d02ad4ef2a44392bf94b4aaac7cba48 [id=950353dc429447bd87bca7aa51581666]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:51.443203 740 tablet_service.cc:1505] Processing CreateTablet for tablet 0b47389bf6f14a45834512d4aaee7d55 (DEFAULT_TABLE table=default.Table_6a4f0ea2d527419c9d66de5d00f826d7 [id=8da57e23ab5c4bdf8dac974323349ba8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:51.443427 2974 raft_consensus.cc:2749] T 062f980b65504d3baf99ae400e521775 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:51.443800 2974 raft_consensus.cc:2804] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:51.443871 2974 raft_consensus.cc:493] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:51.443940 2974 raft_consensus.cc:3060] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.446370 2974 raft_consensus.cc:515] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.446501 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 2207af105bc3471ca930d35a91d38e70 with delete_type TABLET_DATA_DELETED (replaced by tablet 8675e2a18f0f4a0e95a87005cf82d8b4) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.447084 880 tablet_service.cc:1552] Processing DeleteTablet for tablet bbe3529c301a4a65b58b634a443e1fb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.447784 746 tablet_service.cc:1505] Processing CreateTablet for tablet 9daa1e63b66d4ea8aa2bedaee30951d4 (DEFAULT_TABLE table=default.Table_7ea32bac879c48dd90e4a2b54f38a61a [id=5c96e452a72c4903adf3b5ebcdf8679f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:51.448037 748 tablet_service.cc:1505] Processing CreateTablet for tablet d2adf75981244f79b5701cdbc7439d5e (DEFAULT_TABLE table=default.Table_56eb53be351a424e80c1f051e43e8b5d [id=a0c9aeca9294408189d2eb4eb8157f12]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:51.449285 910 raft_consensus.cc:3060] T 71dc8d20d93243e289330d7b8c635511 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.449466 745 tablet_service.cc:1505] Processing CreateTablet for tablet ff8d5c355208445aa191145413f066a1 (DEFAULT_TABLE table=default.Table_73e99aea9d194b0ebf4d4d052506595d [id=ac91bc604bbc4630ba4a16d2825db8a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:51.450228 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c03784820114f5aae409323899209ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:51.456712 894 tablet_service.cc:1552] Processing DeleteTablet for tablet 116b21a182474ca191c4a89e9474677d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.457444 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2adf75981244f79b5701cdbc7439d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:51.457800 882 tablet_service.cc:1552] Processing DeleteTablet for tablet be876471f8bf453aa4467c91891cbbde with delete_type TABLET_DATA_DELETED (replaced by tablet 7d4e63f54d8443ae8322d4659a165e76) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.458909 876 tablet_service.cc:1552] Processing DeleteTablet for tablet 5f2c14c7a972414e8328e0b388bde0a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:45 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.459700 875 tablet_service.cc:1552] Processing DeleteTablet for tablet 1e13965b6361488eaeb7dd13b467ca65 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.459858 740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b47389bf6f14a45834512d4aaee7d55. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:51.461162 892 tablet_service.cc:1505] Processing CreateTablet for tablet 3c03784820114f5aae409323899209ed (DEFAULT_TABLE table=default.Table_37d416a121e34cc8abb9520f5320dfbd [id=31d686268adc45979dad42439723e680]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:51.462097 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 3c5819ae1abb48fe8a83af9067726d02 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.463728 1160 ts_tablet_manager.cc:1434] T a7979798e92e45eb969197c8c7ac2580 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.034s user 0.001s sys 0.001s
I20251211 20:52:51.463826 741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 657b1a60f9184449a333c278886fcdb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:51.464075 1160 tablet_bootstrap.cc:492] T fd43a27db69044b68826cda87e16a086 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:51.464557 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 71dc8d20d93243e289330d7b8c635511 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.464902 887 tablet_service.cc:1552] Processing DeleteTablet for tablet e1aeae8c18b6499e9dc1accabdc3f633 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.465178 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 940dfd8781de444a9caa78dd363c7451 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.465602 887 tablet_service.cc:1552] Processing DeleteTablet for tablet d69fb1f6d9594afcbfb844860b41a377 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:29 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.467754 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b1ce6d2a12a4c7ea044b349412b4bf5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.468242 3322 raft_consensus.cc:493] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:51.468395 3322 raft_consensus.cc:515] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.468525 2959 raft_consensus.cc:493] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:51.468627 2959 raft_consensus.cc:515] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.469045 3322 leader_election.cc:290] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:51.469115 2959 leader_election.cc:290] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:51.470463 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0008c88c9c75424b92e115e2fbd0622f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:51.472361 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9daa1e63b66d4ea8aa2bedaee30951d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:51.473537 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 5bcc94ec9630484cb7a3e8bf6d4f378a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.474704 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 95866de91f9d437d80f40b7962253610 with delete_type TABLET_DATA_DELETED (replaced by tablet 51d12723f5194afb97d2c05fcbaf3776) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.475798 887 tablet_service.cc:1505] Processing CreateTablet for tablet 0008c88c9c75424b92e115e2fbd0622f (DEFAULT_TABLE table=default.Table_187cac16164c4f23bdf2b49d86a68bde [id=bfff447263594ecbad74cc1e44e634a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:51.477604 877 tablet_service.cc:1552] Processing DeleteTablet for tablet 84b2a67c836844ceb3628cc3b7d8ba10 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.477715 3295 ts_tablet_manager.cc:1929] T 0aac6fb3972e4aba8950828c9e49d507 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:51.477794 888 tablet_service.cc:1552] Processing DeleteTablet for tablet 062f980b65504d3baf99ae400e521775 with delete_type TABLET_DATA_DELETED (replaced by tablet 33fd09ea06ee47908b85911e4ee79708) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.477837 3295 log.cc:1199] T 0aac6fb3972e4aba8950828c9e49d507 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/0aac6fb3972e4aba8950828c9e49d507
I20251211 20:52:51.477939 891 tablet_service.cc:1552] Processing DeleteTablet for tablet 22273287aa9245a2a66131fed2c8861c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:27 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.478071 913 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "51d12723f5194afb97d2c05fcbaf3776" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:51.478255 3295 ts_tablet_manager.cc:1950] T 0aac6fb3972e4aba8950828c9e49d507 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:51.478451 906 raft_consensus.cc:3060] T 3621735fcb294e3a8c0180ba9ef86442 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:51.478611 983 leader_election.cc:343] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.479015 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff8d5c355208445aa191145413f066a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:51.479142 3295 tablet_replica.cc:333] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:51.479382 3295 raft_consensus.cc:2243] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:51.479609 3295 raft_consensus.cc:2272] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:51.480334 3295 ts_tablet_manager.cc:1916] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:51.481587 901 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c549c2bfebd64b87a4fc8ce8210e14f6" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:51.481912 880 tablet_service.cc:1552] Processing DeleteTablet for tablet 66cdc6f297b24cfba3a3a0ae9f45d12d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:39 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.482084 894 tablet_service.cc:1552] Processing DeleteTablet for tablet bb0fc179a337469eae056c5bb3815e6b with delete_type TABLET_DATA_DELETED (replaced by tablet d53b074865f4490693152a247b0fffdc) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.482213 882 tablet_service.cc:1552] Processing DeleteTablet for tablet 98d575ac680d49d0990877666638f013 with delete_type TABLET_DATA_DELETED (replaced by tablet 834b84a557b04e78b5d2c8c84520a644) from {username='slave'} at 127.0.0.1:50110
W20251211 20:52:51.484040 983 leader_election.cc:343] T c549c2bfebd64b87a4fc8ce8210e14f6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.484664 876 tablet_service.cc:1552] Processing DeleteTablet for tablet 5d225553d0984b6e9685b5841616c9d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:48 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.484830 875 tablet_service.cc:1552] Processing DeleteTablet for tablet a7979798e92e45eb969197c8c7ac2580 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:30 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.484978 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c03784820114f5aae409323899209ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:51.487668 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 80df719919a44aba9b4e3c62d6830cf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:36 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.487979 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0008c88c9c75424b92e115e2fbd0622f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:51.489571 1160 tablet_bootstrap.cc:654] T fd43a27db69044b68826cda87e16a086 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.491842 901 raft_consensus.cc:3060] T 7be44df39a8f4d6da793479d425fa684 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.494797 912 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4b07f110f25645799ed0c89737cd8279" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:51.510744 904 raft_consensus.cc:3060] T 95866de91f9d437d80f40b7962253610 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.511643 2974 leader_election.cc:290] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Requested vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:51.524915 892 tablet_service.cc:1552] Processing DeleteTablet for tablet 4d2b7e2a0abe4592bc806d5ef3f894d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:28 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.526299 1168 tablet_bootstrap.cc:492] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:51.526561 1168 ts_tablet_manager.cc:1403] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.265s user 0.005s sys 0.000s
I20251211 20:52:51.527817 1168 raft_consensus.cc:359] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.528108 1168 raft_consensus.cc:385] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.528321 1168 raft_consensus.cc:740] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.528712 1168 consensus_queue.cc:260] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.529745 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet aab26df748724fe5b2bb7fad9e3bfc0c (table default.Table_22bed4b4e9e24a90818b19c6f5c91e1f [id=df6255d66f334ef2b4406f4b845cf980]). Expected version 1 got 0
I20251211 20:52:51.531154 892 tablet_service.cc:1552] Processing DeleteTablet for tablet 845b5ac4a5984c0cb4ab188925103dc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:32 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.532173 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b52377c8ffd741a598e5328a00d876fe (table default.Table_686a26915b42421380037144900dd97c [id=8a1b0c9d641b4a3c83d1ca48576d110d]). Expected version 1 got 0
I20251211 20:52:51.532320 892 tablet_service.cc:1552] Processing DeleteTablet for tablet 29463a8f71f943a0927942235af3c2da with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.533787 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 0302cc8613934c43a0650f081cf98797 (table default.Table_bf827b216f1b43c196923dc1828ff0d0 [id=ed371e073250439fa44330c7101e9387]). Expected version 1 got 0
I20251211 20:52:51.534351 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 1f4eb1a174c648c897ae2b7612e44585 (table default.Table_e70873b71ba24d7eae548e2acfd243b3 [id=87adcc552a704d56bd2fe812fb3b791f]). Expected version 1 got 0
I20251211 20:52:51.535138 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet a7afcce522494c9ca1d80e6eea913e6e (table default.Table_cb6a4bac7b694f33a37de2ee78c91ca0 [id=8940f18527c249a9baefc170ed82f45a]). Expected version 1 got 0
I20251211 20:52:51.535554 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 84bb3583857848459e0e865283de623d (table default.Table_92281c5fcfee4355a79215ffedeee588 [id=6e5fe0a78e4b48f98dac028802f1ddf4]). Expected version 4 got 0
I20251211 20:52:51.536162 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 7a299ee03d0e48aba7c4f71e3e0f381a (table default.Table_6402a801d48e4a53827b1b62b6cdaba1 [id=6876565d3a9c48ce841b830cf75139ff]). Expected version 3 got 0
I20251211 20:52:51.538236 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 5b3e8ec3faf445ac9b746e65dc4dbfe0 (table default.Table_7fe80ff5483a4b0292091119980f6639 [id=3c3462a0c09748cea0d2ffb544ae2f13]). Expected version 1 got 0
I20251211 20:52:51.540948 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 7ffa06c4a7ee48b0b337a27aff090592 (table default.Table_647731023b044ebb8954f7fdcc3faf97 [id=4b91abd6e9e94a8a941b33e62e23c083]). Expected version 1 got 0
W20251211 20:52:51.541342 1798 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.2:53985 (request call id 171) took 4529 ms (client timeout 1000 ms). Trace:
I20251211 20:52:51.542575 892 tablet_service.cc:1552] Processing DeleteTablet for tablet c112714331bf4bd7a1ddcca17f338d4f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:33 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.542598 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 81fe26e81790448fb436929c03e3c284 (table default.Table_efc22dc69ad0405ebf59fda961e23330 [id=d8a0839df7524fb7b95bfc05fd4ada25]). Expected version 1 got 0
W20251211 20:52:51.544158 711 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 47842us
W20251211 20:52:51.542131 1798 rpcz_store.cc:269] 1211 20:52:47.011469 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:47.350914 (+339445us) service_pool.cc:225] Handling call
1211 20:52:51.540627 (+4189713us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":230911,"reported_tablets":441}
W20251211 20:52:51.545058 711 consensus_peers.cc:597] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:51.545897 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b81a4cef27164824b52695039744db15 (table default.Table_defdeab28749411590cf75b5251f604b [id=11352566d1744193a40a8bb605aa3912]). Expected version 1 got 0
I20251211 20:52:51.547047 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 68792ccef54944a09af9b5456e9ff1fe (table default.Table_00bd643f7bb84217970178aefb485a69 [id=1b90cdd71d064d27948a1a9a7f3f852b]) successfully deleted
W20251211 20:52:51.547367 711 leader_election.cc:343] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:51.547487 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet ba120d28abc54553ae15532de90db719 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba120d28abc54553ae15532de90db719 already in progress: creating tablet
W20251211 20:52:51.547735 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 0add14ccd2af42aeb02fad7a28ed4520 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0add14ccd2af42aeb02fad7a28ed4520 already in progress: creating tablet
W20251211 20:52:51.547937 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 3dde7ed495e64909bd9784dc5eefb04c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3dde7ed495e64909bd9784dc5eefb04c already in progress: creating tablet
W20251211 20:52:51.548247 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 745ce63314834c8abdecaa367133a068 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 745ce63314834c8abdecaa367133a068 already in progress: creating tablet
W20251211 20:52:51.548870 847 leader_election.cc:343] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 87863daf77304a399318cdf56da9f9bb
W20251211 20:52:51.549129 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 2f7b1bc3dd1f4496b3ac6e71652cf012 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f7b1bc3dd1f4496b3ac6e71652cf012 already in progress: creating tablet
I20251211 20:52:51.548960 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 7023afec769c4ca3b2242a225004d482 (table default.Table_d5276e3cc46d4331b36a8878680fc55e [id=3811f8235b2f44cca283401302d613d4]). Expected version 1 got 0
W20251211 20:52:51.549980 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 76cfb26762d54593a150ec98125c6592 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 76cfb26762d54593a150ec98125c6592 already in progress: creating tablet
W20251211 20:52:51.550781 847 leader_election.cc:343] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: b529b38d754f46ba8cb60b19e9705256
I20251211 20:52:51.550526 910 raft_consensus.cc:3060] T f25bde14792248b188660036f07ae902 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.551064 902 raft_consensus.cc:3060] T 06b8f8e46e524545aeab6c931efaf116 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.551972 1160 tablet_bootstrap.cc:492] T fd43a27db69044b68826cda87e16a086 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:51.552013 3294 tablet_replica.cc:333] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:51.552124 1160 ts_tablet_manager.cc:1403] T fd43a27db69044b68826cda87e16a086 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.088s user 0.004s sys 0.004s
I20251211 20:52:51.552204 3294 raft_consensus.cc:2243] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:51.551714 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 72af55c8045b4f8f8e116a5782794d81 (table default.Table_2cff6fab088c4d939b13b504c133cc12 [id=b6c4b72063b3428593d5650a160dc66b]). Expected version 1 got 0
I20251211 20:52:51.552806 1160 raft_consensus.cc:359] T fd43a27db69044b68826cda87e16a086 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.553009 1160 raft_consensus.cc:385] T fd43a27db69044b68826cda87e16a086 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.553074 1160 raft_consensus.cc:740] T fd43a27db69044b68826cda87e16a086 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.553290 1160 consensus_queue.cc:260] T fd43a27db69044b68826cda87e16a086 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.553777 1160 ts_tablet_manager.cc:1434] T fd43a27db69044b68826cda87e16a086 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:51.554097 1160 tablet_bootstrap.cc:492] T e9ead34065e240eeb1f946319537d32a P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:51.554893 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 5ddf7c1972564ef88e3c0a0bd1b430e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5ddf7c1972564ef88e3c0a0bd1b430e0 already in progress: creating tablet
I20251211 20:52:51.559142 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 4cf0cde801bf44449e539d2799b895a7 (table default.Table_907c596bdddc4edca25d14cbacebb23f [id=321235280d1d4fa0bae5cc0333860f85]). Expected version 1 got 0
I20251211 20:52:51.561599 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 7b9f569554fb4836ac2b62542c3ecf7e (table default.Table_1a73d567e8914a83974e3e8f5f58be0b [id=6ef691749df24251a7a7c5919c219754]). Expected version 1 got 0
I20251211 20:52:51.563481 1166 tablet_bootstrap.cc:492] T a56f1a7bd5464d289bfd018c826ee574 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:51.563715 1166 ts_tablet_manager.cc:1403] T a56f1a7bd5464d289bfd018c826ee574 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.167s user 0.001s sys 0.004s
I20251211 20:52:51.564919 3294 raft_consensus.cc:2272] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:51.566076 3294 ts_tablet_manager.cc:1916] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:51.565865 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 938a0d8a090f490b816fdf2b8f1a51b8 (table default.Table_5ca4ab25a21d456c95a0ff68c9d73330 [id=9ffcb594267a4b0eb8d792dbe46d585f]). Expected version 1 got 0
I20251211 20:52:51.567641 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 59acb8474e114bf7ae7f1f043d611459 (table default.Table_c7ed8c0c309c40a3a371ffa130b47cd2 [id=ed1d0670426945aba888b818ad657225]). Expected version 1 got 0
W20251211 20:52:51.563766 982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.1:42177, user_credentials={real_user=slave}} blocked reactor thread for 121602us
W20251211 20:52:51.569392 982 leader_election.cc:343] T 200b6db115964f3c8882b514c794698d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.569516 982 leader_election.cc:304] T 200b6db115964f3c8882b514c794698d P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:51.569803 2974 consensus_queue.cc:1048] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:51.570223 2974 raft_consensus.cc:2749] T 200b6db115964f3c8882b514c794698d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:51.570705 982 leader_election.cc:343] T a471a9accbfc4875bd1d7802f5620808 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.570878 982 leader_election.cc:304] T a471a9accbfc4875bd1d7802f5620808 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:51.571132 2974 raft_consensus.cc:2749] T a471a9accbfc4875bd1d7802f5620808 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:51.572166 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet bbe6e6220eb14e59a821805c438230c9 (table default.Table_16a12ed370b64a38b5e145c4eb024349 [id=b7429ea93123473ab5d795d5f3d5a5f8]). Expected version 1 got 0
I20251211 20:52:51.577370 2974 raft_consensus.cc:493] T 69085cd41db846e7a6e03bcfee4377f6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:51.577518 2974 raft_consensus.cc:515] T 69085cd41db846e7a6e03bcfee4377f6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.577728 1166 raft_consensus.cc:359] T a56f1a7bd5464d289bfd018c826ee574 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.577894 1166 raft_consensus.cc:385] T a56f1a7bd5464d289bfd018c826ee574 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.577955 1166 raft_consensus.cc:740] T a56f1a7bd5464d289bfd018c826ee574 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.578081 2974 leader_election.cc:290] T 69085cd41db846e7a6e03bcfee4377f6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:51.578137 1166 consensus_queue.cc:260] T a56f1a7bd5464d289bfd018c826ee574 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.578970 1166 ts_tablet_manager.cc:1434] T a56f1a7bd5464d289bfd018c826ee574 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.015s user 0.000s sys 0.001s
I20251211 20:52:51.579308 1166 tablet_bootstrap.cc:492] T 47428f41973642dbbd42ba051ea3d3b0 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:51.579567 903 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "69085cd41db846e7a6e03bcfee4377f6" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:51.581472 1160 tablet_bootstrap.cc:654] T e9ead34065e240eeb1f946319537d32a P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.582749 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet bc4e7362e6d045da81bf39f4135f500e (table default.Table_328b7dd76e9e407c83b47a0f51cfcce5 [id=9e7c08f833e14a90b17a9b9f60b702de]). Expected version 1 got 0
I20251211 20:52:51.584122 892 tablet_service.cc:1505] Processing CreateTablet for tablet ff8d5c355208445aa191145413f066a1 (DEFAULT_TABLE table=default.Table_73e99aea9d194b0ebf4d4d052506595d [id=ac91bc604bbc4630ba4a16d2825db8a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:51.584813 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff8d5c355208445aa191145413f066a1. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:51.583986 983 leader_election.cc:343] T 69085cd41db846e7a6e03bcfee4377f6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.585860 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet dfd194cf00f549c8937cb6ec40987240 (table default.Table_25e38d0576684687890bae04bd81d4f4 [id=d919c6132b3c466e8a1404f981b003b8]). Expected version 2 got 0
I20251211 20:52:51.590818 1166 tablet_bootstrap.cc:654] T 47428f41973642dbbd42ba051ea3d3b0 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.591485 1035 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4b07f110f25645799ed0c89737cd8279" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:51.594970 1035 raft_consensus.cc:2468] T 4b07f110f25645799ed0c89737cd8279 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:51.600139 2962 ts_tablet_manager.cc:1929] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20251211 20:52:51.600881 1802 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.2:53985 (request call id 172) took 3396 ms (client timeout 1000 ms). Trace:
W20251211 20:52:51.602150 1802 rpcz_store.cc:269] 1211 20:52:48.203540 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:48.286178 (+ 82638us) service_pool.cc:225] Handling call
1211 20:52:51.600088 (+3313910us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":9371,"reported_tablets":471}
I20251211 20:52:51.608806 1802 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_92efd469dd324f7d886e2cbc90ece61c" } modify_external_catalogs: true
I20251211 20:52:51.610354 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 79cc81e91e1846038621af7a0a9f7a2e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.611785 3294 ts_tablet_manager.cc:1929] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:51.613111 887 tablet_service.cc:1552] Processing DeleteTablet for tablet de508d79f30f489cb745401f53115165 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:35 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.615324 847 leader_election.cc:304] T d69fb1f6d9594afcbfb844860b41a377 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:51.616305 3294 log.cc:1199] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/18b977f463d5420186c007b38110e07e
I20251211 20:52:51.616925 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 75dd170c5828409fa69bd68fdf28a72b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:38 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.615207 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "10746162da4640e6a2a891467690b770" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:51.610255 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 7ca98b43863a47c091baae0a6ce65630 (table default.Table_f0a70914acc94231a4e84155c3e4d13f [id=56c070ad80624c698ebe2e01b4929b6e]). Expected version 1 got 0
I20251211 20:52:51.617964 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet a295a24e81f14e5fa126a4a58ffd08cf (table default.Table_1021d997007b41208355c31d22f4e26c [id=8c644777215b4617a8ce57691c34ce8c]) successfully deleted
I20251211 20:52:51.617974 1802 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46568:
table { table_name: "default.Table_92efd469dd324f7d886e2cbc90ece61c" } modify_external_catalogs: true
W20251211 20:52:51.618371 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 6d4e6528352f43b6a1b47d7a4b6b3689 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d4e6528352f43b6a1b47d7a4b6b3689 already in progress: creating tablet
W20251211 20:52:51.612900 711 consensus_peers.cc:597] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:51.616545 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c549c2bfebd64b87a4fc8ce8210e14f6" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:51.617313 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "51d12723f5194afb97d2c05fcbaf3776" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919"
I20251211 20:52:51.618949 773 raft_consensus.cc:3060] T 51d12723f5194afb97d2c05fcbaf3776 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.619932 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "69085cd41db846e7a6e03bcfee4377f6" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:51.621446 2962 log.cc:1199] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/c26c3d0170f64a41ae8944c6668cc5f3
I20251211 20:52:51.621843 773 raft_consensus.cc:2468] T 51d12723f5194afb97d2c05fcbaf3776 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e715a811a82446ce8d5bb4aafd5b81c7 in term 1.
I20251211 20:52:51.621886 2962 ts_tablet_manager.cc:1950] T c26c3d0170f64a41ae8944c6668cc5f3 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
W20251211 20:52:51.622532 710 consensus_peers.cc:597] T a295a24e81f14e5fa126a4a58ffd08cf P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a295a24e81f14e5fa126a4a58ffd08cf. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:51.622669 3353 consensus_queue.cc:1048] T d53b074865f4490693152a247b0fffdc P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251211 20:52:51.624766 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 1944267def4e46949fd680ab0a369be9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1944267def4e46949fd680ab0a369be9 already in progress: creating tablet
I20251211 20:52:51.625172 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet e3f711f02af843adb4f5c3c750924e33 (table default.Table_34bd5155aa3b4ef7818022ba9346c074 [id=83a61d23e80946439a14096ab3ab1bd9]). Expected version 2 got 0
I20251211 20:52:51.625905 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 969a8587019c4202b6e705d126f83d44 (table default.Table_6460313002684bba93b30ad8631f0f49 [id=22347b15537d451d9927a17e8e5a202a]). Expected version 1 got 0
I20251211 20:52:51.626513 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 78f77d8916d84edabbe95ce17d8df2c1 (table default.Table_e2af0e2fbd3d46d595136c7a7bf24e4c [id=ddc12ae6d8ed4587b5ef4da3ef81a055]). Expected version 1 got 0
I20251211 20:52:51.627264 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet df457783b2534b3ea23bb1fd529ce268 (table default.Table_6e9f3ca164384cc8ac4501deb3f0c6aa [id=342c2469d6e74e3e9217236e3aa863d6]). Expected version 1 got 0
I20251211 20:52:51.627907 3428 consensus_queue.cc:1048] T f2ae084982f44fdf8bfecdd2bc4c01b9 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:51.628233 1166 tablet_bootstrap.cc:492] T 47428f41973642dbbd42ba051ea3d3b0 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:51.628417 1166 ts_tablet_manager.cc:1403] T 47428f41973642dbbd42ba051ea3d3b0 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.049s user 0.003s sys 0.001s
I20251211 20:52:51.630249 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 3f1840e4b5214ddb878935a79b214eae with delete_type TABLET_DATA_DELETED (replaced by tablet 4fc56df5ccb94198b5b0e516eeae6941) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.630694 1166 raft_consensus.cc:359] T 47428f41973642dbbd42ba051ea3d3b0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.630770 886 tablet_service.cc:1552] Processing DeleteTablet for tablet f2ae084982f44fdf8bfecdd2bc4c01b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:34 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.630961 1166 raft_consensus.cc:385] T 47428f41973642dbbd42ba051ea3d3b0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.631068 1166 raft_consensus.cc:740] T 47428f41973642dbbd42ba051ea3d3b0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.631088 886 tablet_service.cc:1552] Processing DeleteTablet for tablet f9a3fdf17f274c15ac0bdfa9c174f649 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.631395 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b07f110f25645799ed0c89737cd8279 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:31 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.631358 1166 consensus_queue.cc:260] T 47428f41973642dbbd42ba051ea3d3b0 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.631992 1166 ts_tablet_manager.cc:1434] T 47428f41973642dbbd42ba051ea3d3b0 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251211 20:52:51.632798 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 155) took 2524 ms. Trace:
I20251211 20:52:51.632894 2962 rpcz_store.cc:276] 1211 20:52:49.108590 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.108652 (+ 62us) service_pool.cc:225] Handling call
1211 20:52:51.421571 (+2312919us) ts_tablet_manager.cc:1151] Deleting tablet c26c3d0170f64a41ae8944c6668cc5f3
1211 20:52:51.425974 (+ 4403us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:51.432623 (+ 6649us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:51.600017 (+167394us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:51.632779 (+ 32762us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2301844}
I20251211 20:52:51.633227 2962 tablet_replica.cc:333] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:51.633424 2962 raft_consensus.cc:2243] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:51.633558 2962 raft_consensus.cc:2272] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:51.634079 2962 ts_tablet_manager.cc:1916] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:51.633229 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet d6923cfd826d495291d9ac8e0b3b44c6 (table default.Table_68ccb2052d824ccf92f7162d2961fcb1 [id=9ac9591d4f04482d926710fdf6e797c6]). Expected version 1 got 0
I20251211 20:52:51.634615 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b2974f16034c44cf85ff85b5095cf6a0 (table default.Table_031de0f4cc7744bc9e320261f8b46696 [id=be36dabaa6d44325b4855b9f13bda847]). Expected version 1 got 0
I20251211 20:52:51.635787 3423 raft_consensus.cc:493] T f26ba8f4bd3a4a1f8fa2ba6e80d543f8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:51.635617 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b6cc819332e1445c96ce3ffac4861454 (table default.Table_db4c0f706e0643b39d39ffa84352826d [id=4d459c42bf584068ac3851227d80331a]). Expected version 1 got 0
I20251211 20:52:51.635983 3423 raft_consensus.cc:515] T f26ba8f4bd3a4a1f8fa2ba6e80d543f8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.636224 1166 tablet_bootstrap.cc:492] T 1b8acfdf1a26427aa9863280dc39b818 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:51.636519 983 consensus_peers.cc:597] T 903c7f4b312d4677bb35be1aaa46ad6b P e715a811a82446ce8d5bb4aafd5b81c7 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:51.636840 3423 leader_election.cc:290] T f26ba8f4bd3a4a1f8fa2ba6e80d543f8 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:51.637105 903 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f26ba8f4bd3a4a1f8fa2ba6e80d543f8" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:51.637581 983 leader_election.cc:343] T f26ba8f4bd3a4a1f8fa2ba6e80d543f8 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.639147 1166 tablet_bootstrap.cc:654] T 1b8acfdf1a26427aa9863280dc39b818 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.638916 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 16d2ff8925e342729a71c93db378d031 (table default.Table_34e6ed4f13da437ea50d349b92e83bd6 [id=4a74b9a60cbb4198b55cee630b2f4bef]). Expected version 2 got 0
I20251211 20:52:51.639801 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 3c03784820114f5aae409323899209ed (DEFAULT_TABLE table=default.Table_37d416a121e34cc8abb9520f5320dfbd [id=31d686268adc45979dad42439723e680]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:51.639878 1027 tablet_service.cc:1505] Processing CreateTablet for tablet 0008c88c9c75424b92e115e2fbd0622f (DEFAULT_TABLE table=default.Table_187cac16164c4f23bdf2b49d86a68bde [id=bfff447263594ecbad74cc1e44e634a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:51.640108 3294 ts_tablet_manager.cc:1950] T 18b977f463d5420186c007b38110e07e P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:51.640177 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c03784820114f5aae409323899209ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:51.640383 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0008c88c9c75424b92e115e2fbd0622f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:51.640916 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 165) took 1013 ms. Trace:
I20251211 20:52:51.640997 3294 rpcz_store.cc:276] 1211 20:52:50.627760 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.628036 (+ 276us) service_pool.cc:225] Handling call
1211 20:52:51.551967 (+923931us) ts_tablet_manager.cc:1151] Deleting tablet 18b977f463d5420186c007b38110e07e
1211 20:52:51.551975 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:51.586216 (+ 34241us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:51.611693 (+ 25477us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:51.640895 (+ 29202us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":11749,"tablet-delete.queue_time_us":923834}
I20251211 20:52:51.641407 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 167) took 1009 ms. Trace:
I20251211 20:52:51.641470 3294 rpcz_store.cc:276] 1211 20:52:50.632093 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.632599 (+ 506us) service_pool.cc:225] Handling call
1211 20:52:51.641293 (+1008694us) ts_tablet_manager.cc:1151] Deleting tablet 042f8bf13a7a4712b5bdc04671cd9938
1211 20:52:51.641303 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:51.641392 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1008486}
I20251211 20:52:51.641685 3294 tablet_replica.cc:333] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:51.641855 3294 raft_consensus.cc:2243] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:51.641999 3294 raft_consensus.cc:2272] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:51.641602 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 5e47af616e1b4d2db3780c1d67cdcb47 (table default.Table_296bdf30a11541c389afe3df0b0f667a [id=052edcbde60c41449d468c8a449fc3f1]). Expected version 1 got 0
I20251211 20:52:51.642486 3294 ts_tablet_manager.cc:1916] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:51.643112 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet ab4e7a9062134e489d6f9abe6ff26fb4 (table default.Table_4cbc8841b276406cb4434629653a5ce3 [id=b958532650374d609199cab9c07bb64b]). Expected version 2 got 0
W20251211 20:52:51.648367 1780 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet a7afcce522494c9ca1d80e6eea913e6e (table default.Table_cb6a4bac7b694f33a37de2ee78c91ca0 [id=8940f18527c249a9baefc170ed82f45a], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a7afcce522494c9ca1d80e6eea913e6e consensus state has no leader
I20251211 20:52:51.648689 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 18b977f463d5420186c007b38110e07e (table default.Table_37520ef8d4194d8aa3eadef57196a291 [id=a7e8389b5a9a4b01b7a6c327fd2022b9]) successfully deleted
W20251211 20:52:51.649088 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 042f8bf13a7a4712b5bdc04671cd9938 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 042f8bf13a7a4712b5bdc04671cd9938 already in progress: creating tablet
I20251211 20:52:51.649914 1027 tablet_service.cc:1505] Processing CreateTablet for tablet ff8d5c355208445aa191145413f066a1 (DEFAULT_TABLE table=default.Table_73e99aea9d194b0ebf4d4d052506595d [id=ac91bc604bbc4630ba4a16d2825db8a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:51.650357 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff8d5c355208445aa191145413f066a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:51.650431 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 4c0455a5b4f44412803475d1c4852b48 (table default.Table_e00092673b0e4f09b8d7eec63e1ee034 [id=18497e06d7254441a4655ce1cdc5a9bf]). Expected version 1 got 0
I20251211 20:52:51.653191 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 4c7b74f370b042ee8575f821c07dc085 (table default.Table_731826f26f9940b8afe425d694b642d9 [id=17dd9cea9ac844128ff9f82adb3f919f]). Expected version 1 got 0
I20251211 20:52:51.654160 1017 tablet_service.cc:1505] Processing CreateTablet for tablet 657b1a60f9184449a333c278886fcdb2 (DEFAULT_TABLE table=default.Table_3d02ad4ef2a44392bf94b4aaac7cba48 [id=950353dc429447bd87bca7aa51581666]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:51.654237 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 0b47389bf6f14a45834512d4aaee7d55 (DEFAULT_TABLE table=default.Table_6a4f0ea2d527419c9d66de5d00f826d7 [id=8da57e23ab5c4bdf8dac974323349ba8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:51.654501 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 657b1a60f9184449a333c278886fcdb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:51.654709 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b47389bf6f14a45834512d4aaee7d55. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:51.655460 1795 catalog_manager.cc:5654] T 497c10742f01476db885bd70c4d2c491 P e715a811a82446ce8d5bb4aafd5b81c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } }
I20251211 20:52:51.658027 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 497c10742f01476db885bd70c4d2c491 (table default.Table_18b21232dfa74378b27defe672bb2574 [id=602d7eee703948d99a0edcb7cac0651b]). Expected version 1 got 0
I20251211 20:52:51.659327 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3003ef71a3554c71a3b24cdfb9c293bc (table default.Table_92efd469dd324f7d886e2cbc90ece61c [id=6e7fb3271e0c48c0bb9d56fe4fccee3a]). Expected version 1 got 0
I20251211 20:52:51.660562 3423 raft_consensus.cc:493] T a56f1a7bd5464d289bfd018c826ee574 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:51.660746 3423 raft_consensus.cc:515] T a56f1a7bd5464d289bfd018c826ee574 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.661443 3423 leader_election.cc:290] T a56f1a7bd5464d289bfd018c826ee574 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:51.662750 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet f94b656e3c1844509fdae36e33b263e8 (table default.Table_ed03ad475bed4804a65abe20efef0625 [id=e23f4305669a474aafa18928f130de3a]). Expected version 2 got 0
I20251211 20:52:51.663414 1029 tablet_service.cc:1505] Processing CreateTablet for tablet 9daa1e63b66d4ea8aa2bedaee30951d4 (DEFAULT_TABLE table=default.Table_7ea32bac879c48dd90e4a2b54f38a61a [id=5c96e452a72c4903adf3b5ebcdf8679f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:51.663722 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a56f1a7bd5464d289bfd018c826ee574" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:51.663936 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9daa1e63b66d4ea8aa2bedaee30951d4. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:51.664238 983 leader_election.cc:343] T a56f1a7bd5464d289bfd018c826ee574 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.665131 886 tablet_service.cc:1552] Processing DeleteTablet for tablet 79d92736bb1c46d2bfa5678b2d396ec2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.665716 886 tablet_service.cc:1505] Processing CreateTablet for tablet 657b1a60f9184449a333c278886fcdb2 (DEFAULT_TABLE table=default.Table_3d02ad4ef2a44392bf94b4aaac7cba48 [id=950353dc429447bd87bca7aa51581666]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:51.666116 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 657b1a60f9184449a333c278886fcdb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:51.667222 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 03184dcc56dd4e92adc7d0ad22aa8d8b (table default.Table_68261841754847ad8d22e86c29886144 [id=697f23a617f64762bf406b386cde7473]). Expected version 1 got 0
I20251211 20:52:51.668083 3428 raft_consensus.cc:493] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:51.668222 3428 raft_consensus.cc:515] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.669416 3428 leader_election.cc:290] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:51.672089 1025 tablet_service.cc:1505] Processing CreateTablet for tablet d2adf75981244f79b5701cdbc7439d5e (DEFAULT_TABLE table=default.Table_56eb53be351a424e80c1f051e43e8b5d [id=a0c9aeca9294408189d2eb4eb8157f12]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:51.672456 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2adf75981244f79b5701cdbc7439d5e. 1 dirs total, 0 dirs full, 0 dirs failed
W20251211 20:52:51.673429 711 consensus_peers.cc:597] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:51.676486 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet c26c3d0170f64a41ae8944c6668cc5f3 (table default.Table_7325cceb834541c895db5f104ae375aa [id=a1149606f150473ab2d908bf42316081]) successfully deleted
I20251211 20:52:51.678480 886 tablet_service.cc:1505] Processing CreateTablet for tablet 0b47389bf6f14a45834512d4aaee7d55 (DEFAULT_TABLE table=default.Table_6a4f0ea2d527419c9d66de5d00f826d7 [id=8da57e23ab5c4bdf8dac974323349ba8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:51.678939 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b47389bf6f14a45834512d4aaee7d55. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:51.679068 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f26ba8f4bd3a4a1f8fa2ba6e80d543f8" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:51.681820 3428 raft_consensus.cc:493] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:51.681988 3428 raft_consensus.cc:515] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.682617 3428 leader_election.cc:290] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:51.682204 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 8dc4e730951144d78b77280f6cce6d0e (table default.Table_890c9027415a44ccaf95c0adede51992 [id=7bc8a2b91fe54b91b6594b371fea7bd3]). Expected version 2 got 0
I20251211 20:52:51.683626 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a56f1a7bd5464d289bfd018c826ee574" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:51.686153 847 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 70882.8us
I20251211 20:52:51.686478 847 leader_election.cc:304] T 95866de91f9d437d80f40b7962253610 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:51.686038 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 13269a20421240d1a98215129443248b (table default.Table_d887c953026046dc883ede774a58fd8e [id=1d8a79f716a0447cb0393f4759178598]). Expected version 1 got 0
W20251211 20:52:51.686925 847 leader_election.cc:343] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 87863daf77304a399318cdf56da9f9bb
I20251211 20:52:51.687011 847 leader_election.cc:304] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
W20251211 20:52:51.687294 847 leader_election.cc:343] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: b529b38d754f46ba8cb60b19e9705256
I20251211 20:52:51.687372 847 leader_election.cc:304] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:51.687598 3397 raft_consensus.cc:2749] T d69fb1f6d9594afcbfb844860b41a377 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:52:51.687981 3174 raft_consensus.cc:2749] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:51.688016 3397 raft_consensus.cc:2749] T b529b38d754f46ba8cb60b19e9705256 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:51.688640 903 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e59f8afe1834485d9fc64615e7bfe174" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:51.688999 982 leader_election.cc:343] T 10746162da4640e6a2a891467690b770 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.689105 982 leader_election.cc:304] T 10746162da4640e6a2a891467690b770 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:51.689340 711 leader_election.cc:343] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.689610 3423 raft_consensus.cc:2749] T 10746162da4640e6a2a891467690b770 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:51.690097 982 leader_election.cc:343] T c549c2bfebd64b87a4fc8ce8210e14f6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.690954 982 leader_election.cc:304] T c549c2bfebd64b87a4fc8ce8210e14f6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:51.691165 3423 raft_consensus.cc:2749] T c549c2bfebd64b87a4fc8ce8210e14f6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:51.691516 982 leader_election.cc:343] T 69085cd41db846e7a6e03bcfee4377f6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.691640 982 leader_election.cc:304] T 69085cd41db846e7a6e03bcfee4377f6 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:51.691835 3423 raft_consensus.cc:2749] T 69085cd41db846e7a6e03bcfee4377f6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:51.691146 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 51d12723f5194afb97d2c05fcbaf3776 (table default.Table_823505200e3142dbbe8cd56ac78daf55 [id=ec0bb10933fa4529bef3580bee6409e7]). Expected version 2 got 0
I20251211 20:52:51.692198 982 leader_election.cc:304] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:51.692392 3423 raft_consensus.cc:2804] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:51.692505 3423 raft_consensus.cc:697] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 LEADER]: Becoming Leader. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Running, Role: LEADER
I20251211 20:52:51.692724 3423 consensus_queue.cc:237] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:51.695971 982 leader_election.cc:343] T f26ba8f4bd3a4a1f8fa2ba6e80d543f8 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.696079 982 leader_election.cc:304] T f26ba8f4bd3a4a1f8fa2ba6e80d543f8 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:51.695611 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 46a2a070cf6a42fe8bc54a42eb802fbc (table default.Table_4e1f03b9d87b4fe59dbcb3a947ee299d [id=8e66ea58499b40e59294a135324aa527]). Expected version 1 got 0
I20251211 20:52:51.697224 3269 raft_consensus.cc:2749] T f26ba8f4bd3a4a1f8fa2ba6e80d543f8 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:51.697237 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 5773c13034714a008c8d7dbe777b4828 (table default.Table_66eb1b021eed464f82f9cd223c88297b [id=94ff09564cd649c7a2d0e94b0f3c7b3a]). Expected version 1 got 0
W20251211 20:52:51.697852 982 leader_election.cc:343] T a56f1a7bd5464d289bfd018c826ee574 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.698493 982 leader_election.cc:304] T a56f1a7bd5464d289bfd018c826ee574 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:51.698987 3269 raft_consensus.cc:2749] T a56f1a7bd5464d289bfd018c826ee574 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:51.699993 3398 raft_consensus.cc:2749] T 95866de91f9d437d80f40b7962253610 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:51.699906 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 18f125dd9efc401181f0644dad7aeffe (table default.Table_fa6592c2e2624d81ad21abb13ed36045 [id=9b48fc2b39db4fd5a4e5da19e34427dc]). Expected version 1 got 0
I20251211 20:52:51.702028 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet aa50ddb2789b4be3bfa9aedef9287109 (table default.Table_44f971096d1e41f480e594985ffd5c17 [id=d4e1349374a346a5b8be180f9bdc329a]). Expected version 1 got 0
I20251211 20:52:51.703214 3434 tablet.cc:1722] T 7be44df39a8f4d6da793479d425fa684 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20251211 20:52:51.704027 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 81150.3us
I20251211 20:52:51.705363 3353 consensus_queue.cc:1048] T 3f1840e4b5214ddb878935a79b214eae P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:51.706485 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet ff4d5ec1f87247e896f9c099c532265a (table default.Table_3ac4746e17264f5a93890f3600505781 [id=475e0a1ec0344d10addbcfa42fe05b68]). Expected version 1 got 0
I20251211 20:52:51.707978 710 leader_election.cc:304] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:51.708400 3353 raft_consensus.cc:2804] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:51.708468 3353 raft_consensus.cc:493] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:51.708521 3353 raft_consensus.cc:3060] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.708688 3269 consensus_queue.cc:1048] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:51.708416 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet bc6a1c84501d4e179cd2828b50fddc23 (table default.Table_8bb5b228d60e4d29a41d9ba40c33d45d [id=6ef058fada9941629e0ea4d1906abdac]). Expected version 1 got 0
I20251211 20:52:51.710966 3353 raft_consensus.cc:515] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.710553 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3dfea28c44d84c68850db65a30d52ee6 (table default.Table_c504f8f3796f43b697488bd604268543 [id=2af76b9d32094a4ca656060a4478e08a]). Expected version 1 got 0
I20251211 20:52:51.711509 3353 leader_election.cc:290] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:51.712359 1168 ts_tablet_manager.cc:1434] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.186s user 0.002s sys 0.000s
I20251211 20:52:51.712715 1168 tablet_bootstrap.cc:492] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:51.714294 1168 tablet_bootstrap.cc:654] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.713938 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet f00128ba84c245d9a0e35f26bc842b3e (table default.Table_6f8c133c7a364b93af2c83d8911a0ac3 [id=6a21c38ee7224a02b937df088494f5d8]). Expected version 1 got 0
W20251211 20:52:51.721681 710 consensus_peers.cc:597] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ecf0ea8ce19486c81d1d1a91d57a7ce. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:51.722184 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 3c865c99e9314f5f91658b9126da4d39 (table default.Table_0d7df3fba38849a49b65d8290812b95c [id=7f966d5cdad3423bb770414802a3cf7b]). Expected version 1 got 0
I20251211 20:52:51.724766 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "062f980b65504d3baf99ae400e521775" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:51.725358 711 leader_election.cc:343] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.724287 1795 catalog_manager.cc:5654] T 44e2d2bdecf247baaac26e06bccc94de P e715a811a82446ce8d5bb4aafd5b81c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } }
I20251211 20:52:51.727689 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet e89d4d76bf64453892a4d0b55fcc3454 (table default.Table_fba99384e9f14160b3e3504f8d1d0b07 [id=01bba75f854d4c338d6c09a446dd7499]). Expected version 1 got 0
I20251211 20:52:51.730460 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 834b84a557b04e78b5d2c8c84520a644 (table default.Table_909bbe6da1a648d68199fb27424e4342 [id=cc99459a5039424dbc192ff748b6708b]). Expected version 2 got 0
I20251211 20:52:51.732961 901 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4b07f110f25645799ed0c89737cd8279" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:51.733942 711 leader_election.cc:343] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.734311 887 tablet_service.cc:1505] Processing CreateTablet for tablet d2adf75981244f79b5701cdbc7439d5e (DEFAULT_TABLE table=default.Table_56eb53be351a424e80c1f051e43e8b5d [id=a0c9aeca9294408189d2eb4eb8157f12]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:52:51.734803 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2adf75981244f79b5701cdbc7439d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:51.736338 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet b115d737d7fc483d9f5a1f040b511bae (table default.Table_6d94053661f142b79292e19752df02df [id=c5be59a2c3934b3789641ceeb2d4d6e2]). Expected version 2 got 0
I20251211 20:52:51.738163 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet da81bf8854a24954ba4548185812295d (table default.Table_18262e22c26d4d66ba4c6134fbde0368 [id=da8cf2b99b42409091484a9513623d6f]). Expected version 1 got 0
I20251211 20:52:51.742975 1166 tablet_bootstrap.cc:492] T 1b8acfdf1a26427aa9863280dc39b818 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:51.743302 1166 ts_tablet_manager.cc:1403] T 1b8acfdf1a26427aa9863280dc39b818 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.107s user 0.003s sys 0.000s
I20251211 20:52:51.745991 875 tablet_service.cc:1552] Processing DeleteTablet for tablet c8bb92fc25124ba4be887c453abc7d9a with delete_type TABLET_DATA_DELETED (replaced by tablet 80486aaeb28849c58c855475871853c0) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.746466 875 tablet_service.cc:1552] Processing DeleteTablet for tablet 3fc173c396ff4af5a8589cff18165e85 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:48 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.746876 875 tablet_service.cc:1552] Processing DeleteTablet for tablet 903c7f4b312d4677bb35be1aaa46ad6b with delete_type TABLET_DATA_DELETED (replaced by tablet 200fbba33f8947cca99ddc81e37dfc28) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.747236 875 tablet_service.cc:1552] Processing DeleteTablet for tablet 09cd11f8b23c4f8a98a2b2eb77bb7b8c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:48 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.747690 1783 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 0aac6fb3972e4aba8950828c9e49d507 (table default.Table_bf722f720fc64f2c875ab1edbcaebaf9 [id=fdebf1e48cc24020867e6b24ef2b55b4]) successfully deleted
W20251211 20:52:51.748143 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 0add14ccd2af42aeb02fad7a28ed4520 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0add14ccd2af42aeb02fad7a28ed4520 already in progress: creating tablet
W20251211 20:52:51.748585 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 76cfb26762d54593a150ec98125c6592 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 76cfb26762d54593a150ec98125c6592 already in progress: creating tablet
I20251211 20:52:51.749342 1036 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "062f980b65504d3baf99ae400e521775" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:51.749521 1036 raft_consensus.cc:2468] T 062f980b65504d3baf99ae400e521775 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:51.749962 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 33fd09ea06ee47908b85911e4ee79708 (table default.Table_3c3cd48c8fe5411e96cdf811d492925d [id=30620a01c27f456bac05fa7c02a2cb4d]). Expected version 2 got 0
I20251211 20:52:51.752331 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 6212b3b4425543168837946a54d60834 (table default.Table_8c30c4151d484db0bb9e2b9ca262248d [id=2d489cd7d73e403d84bf10960e6a2dfb]). Expected version 1 got 0
I20251211 20:52:51.757164 1166 raft_consensus.cc:359] T 1b8acfdf1a26427aa9863280dc39b818 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.756910 1795 catalog_manager.cc:5654] T fd43a27db69044b68826cda87e16a086 P e715a811a82446ce8d5bb4aafd5b81c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } }
I20251211 20:52:51.758833 1166 raft_consensus.cc:385] T 1b8acfdf1a26427aa9863280dc39b818 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.759779 1166 raft_consensus.cc:740] T 1b8acfdf1a26427aa9863280dc39b818 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.760478 3294 ts_tablet_manager.cc:1929] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:51.760627 3294 log.cc:1199] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/7d4bba0aa7ea403e8dfcb74cec2e1f9b
I20251211 20:52:51.760192 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet fd43a27db69044b68826cda87e16a086 (table default.Table_7833c67862334235b0e752bfcf44ff81 [id=f1d32002c612403a82bdff5baf209b9e]). Expected version 2 got 0
I20251211 20:52:51.761001 3294 ts_tablet_manager.cc:1950] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:51.761282 1160 tablet_bootstrap.cc:492] T e9ead34065e240eeb1f946319537d32a P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:51.761616 1166 consensus_queue.cc:260] T 1b8acfdf1a26427aa9863280dc39b818 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.763250 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e59f8afe1834485d9fc64615e7bfe174" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:51.763484 1033 raft_consensus.cc:2468] T e59f8afe1834485d9fc64615e7bfe174 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
W20251211 20:52:51.764016 1097 tablet.cc:2378] T 47428f41973642dbbd42ba051ea3d3b0 P e715a811a82446ce8d5bb4aafd5b81c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251211 20:52:51.764072 1160 ts_tablet_manager.cc:1403] T e9ead34065e240eeb1f946319537d32a P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.210s user 0.002s sys 0.013s
I20251211 20:52:51.764734 901 raft_consensus.cc:3060] T d43210ec61af423ebde624397dea9f2e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.765139 1166 ts_tablet_manager.cc:1434] T 1b8acfdf1a26427aa9863280dc39b818 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.022s user 0.003s sys 0.000s
I20251211 20:52:51.764026 1795 catalog_manager.cc:5654] T 94658fcf512a48d0a992a7e2e0f7c176 P e715a811a82446ce8d5bb4aafd5b81c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1). New cstate: current_term: 1 leader_uuid: "6f8d784fd1cd492bbe3809403ec34919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } }
I20251211 20:52:51.766433 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 94658fcf512a48d0a992a7e2e0f7c176 (table default.Table_c9d86e82aebb4ef0bc83789720352e38 [id=97377a8ef56c4cdebe95d15955207404]). Expected version 1 got 0
I20251211 20:52:51.767297 1166 tablet_bootstrap.cc:492] T 07e4336171714da8adc21977ce33ce50 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:51.767999 983 consensus_peers.cc:597] T 7a0a20f95416425489af94408ac1e97c P e715a811a82446ce8d5bb4aafd5b81c7 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:51.768282 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet fb29f983c4524a49bdc5c0c1144ad708 (table default.Table_7809ac85fdfe4c1d9c58afb54edecb27 [id=c4b0bfff99f64888beb70d9154d2f156]). Expected version 1 got 0
I20251211 20:52:51.769467 892 tablet_service.cc:1552] Processing DeleteTablet for tablet 757ec913dff840f992a41a38b05f8d41 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-11 20:52:40 UTC) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.770044 886 tablet_service.cc:1505] Processing CreateTablet for tablet 9daa1e63b66d4ea8aa2bedaee30951d4 (DEFAULT_TABLE table=default.Table_7ea32bac879c48dd90e4a2b54f38a61a [id=5c96e452a72c4903adf3b5ebcdf8679f]), partition=RANGE (key) PARTITION UNBOUNDED
W20251211 20:52:51.771531 847 consensus_peers.cc:597] T 7d4bba0aa7ea403e8dfcb74cec2e1f9b P eb0f1933d9a74fccaa4618047fc05c41 -> Peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Couldn't send request to peer 6f8d784fd1cd492bbe3809403ec34919. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:51.774415 1040 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4b07f110f25645799ed0c89737cd8279" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:51.774592 1040 raft_consensus.cc:3060] T 4b07f110f25645799ed0c89737cd8279 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.777184 1040 raft_consensus.cc:2468] T 4b07f110f25645799ed0c89737cd8279 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:51.776535 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 171d40a8a9124f9bb1b38797fef8e5d5 (table default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5]). Expected version 2 got 0
I20251211 20:52:51.781203 2962 ts_tablet_manager.cc:1929] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:51.781332 2962 log.cc:1199] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/7b06cd366de5494780b5ac81abbf0702
I20251211 20:52:51.780694 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet c081396b492049b996ac4661e1318388 (table default.Table_00ae5c111de94ba0a2680ad22c726e16 [id=95c9ed1ec1834e2dbfd3198887b75369]). Expected version 1 got 0
I20251211 20:52:51.781576 3438 raft_consensus.cc:493] T 58e35de0792a464a818170886ca8d8c9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:51.781649 2962 ts_tablet_manager.cc:1950] T 7b06cd366de5494780b5ac81abbf0702 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:51.781700 3438 raft_consensus.cc:515] T 58e35de0792a464a818170886ca8d8c9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.781917 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 157) took 2673 ms. Trace:
I20251211 20:52:51.781986 2962 rpcz_store.cc:276] 1211 20:52:49.108799 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.108890 (+ 91us) service_pool.cc:225] Handling call
1211 20:52:51.633178 (+2524288us) ts_tablet_manager.cc:1151] Deleting tablet 7b06cd366de5494780b5ac81abbf0702
1211 20:52:51.633187 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:51.651841 (+ 18654us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:51.781077 (+129236us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:51.781903 (+ 826us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2513388}
I20251211 20:52:51.782274 2962 tablet_replica.cc:333] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:51.782282 3438 leader_election.cc:290] T 58e35de0792a464a818170886ca8d8c9 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:51.782447 2962 raft_consensus.cc:2243] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:51.782501 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 7b06cd366de5494780b5ac81abbf0702 (table default.Table_b23d9e7bfa424f92a22e13faaeba571f [id=dcd95d2b036045be9b2eb93cdff84982]) successfully deleted
I20251211 20:52:51.782601 2962 raft_consensus.cc:2272] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:51.782888 1160 raft_consensus.cc:359] T e9ead34065e240eeb1f946319537d32a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.783034 1160 raft_consensus.cc:385] T e9ead34065e240eeb1f946319537d32a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.783080 1160 raft_consensus.cc:740] T e9ead34065e240eeb1f946319537d32a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.783114 2962 ts_tablet_manager.cc:1916] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:51.783275 1160 consensus_queue.cc:260] T e9ead34065e240eeb1f946319537d32a P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.783702 1040 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "58e35de0792a464a818170886ca8d8c9" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:51.783776 1160 ts_tablet_manager.cc:1434] T e9ead34065e240eeb1f946319537d32a P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.020s user 0.001s sys 0.001s
I20251211 20:52:51.784201 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9daa1e63b66d4ea8aa2bedaee30951d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:52:51.784010 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 4fe5c534b572491c911aae01eaa79bae (table default.Table_c1f9dca5c2144d50b67681dcc6800e77 [id=14fa8debfe5443ce81587f02dfa15847]). Expected version 2 got 0
I20251211 20:52:51.784636 1166 tablet_bootstrap.cc:654] T 07e4336171714da8adc21977ce33ce50 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.786880 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 2e800d8cc5ea4535a065d7d9243326b4 (table default.Table_083c478bedec45a5bf5203e4ed333860 [id=8baf0ea87d734d58896adbd0d6ef99c2]). Expected version 1 got 0
I20251211 20:52:51.788463 887 tablet_service.cc:1552] Processing DeleteTablet for tablet 373d3f03d7e0408cbe06955f8f60b16c with delete_type TABLET_DATA_DELETED (replaced by tablet 9ae2a68a1d0c45ae93d42472221bdc65) from {username='slave'} at 127.0.0.1:50110
I20251211 20:52:51.788143 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet d43210ec61af423ebde624397dea9f2e (table default.Table_af0df4d1ee144455a6041e5a6a0b9b80 [id=7c5b2e33fa1c4b12b3941577360908a8]). Expected version 3 got 0
I20251211 20:52:51.789992 1160 tablet_bootstrap.cc:492] T 6029c1fd563d4023a8cf7cfb8ffa0876 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:51.789490 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 8675e2a18f0f4a0e95a87005cf82d8b4 (table default.Table_bd9633c55bd242c18aac3a7f2c18a329 [id=767f50bee5464316a18f73e64bab817a]). Expected version 2 got 0
I20251211 20:52:51.791561 3295 ts_tablet_manager.cc:1929] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:51.791750 3295 log.cc:1199] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/b631e6b3048e428f901af1b5b0b1ab3c
I20251211 20:52:51.792153 3295 ts_tablet_manager.cc:1950] T b631e6b3048e428f901af1b5b0b1ab3c P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:51.793334 1160 tablet_bootstrap.cc:654] T 6029c1fd563d4023a8cf7cfb8ffa0876 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.793678 3295 tablet_replica.cc:333] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:51.793780 1783 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet b631e6b3048e428f901af1b5b0b1ab3c (table default.Table_dd84a1d3fe964c76b146f798f6781baa [id=e80ce2c7f3aa48539b6d2fad357b3b49]) successfully deleted
I20251211 20:52:51.793923 3295 raft_consensus.cc:2243] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 [term 1 LEADER]: Raft consensus shutting down.
W20251211 20:52:51.794282 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 745ce63314834c8abdecaa367133a068 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 745ce63314834c8abdecaa367133a068 already in progress: creating tablet
I20251211 20:52:51.794040 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet f26ba8f4bd3a4a1f8fa2ba6e80d543f8 (table default.Table_71e9c158dc22435a8fbb02aecca8919d [id=b966d0c0b49d4023894950c5dc06f678]). Expected version 2 got 0
W20251211 20:52:51.794759 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 2f7b1bc3dd1f4496b3ac6e71652cf012 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f7b1bc3dd1f4496b3ac6e71652cf012 already in progress: creating tablet
I20251211 20:52:51.796463 1795 catalog_manager.cc:5746] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845) does not have the latest schema for tablet 1c4b2c22bd4c456eb77af45576de466f (table default.Table_e3a99b043e4f420a93ebfee6aa87f2ec [id=3e7d6cefcb7d412191c6540e9ad77c2d]). Expected version 2 got 0
I20251211 20:52:51.800860 1166 tablet_bootstrap.cc:492] T 07e4336171714da8adc21977ce33ce50 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:51.801285 1166 ts_tablet_manager.cc:1403] T 07e4336171714da8adc21977ce33ce50 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.035s user 0.005s sys 0.000s
I20251211 20:52:51.802070 1166 raft_consensus.cc:359] T 07e4336171714da8adc21977ce33ce50 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.802239 1166 raft_consensus.cc:385] T 07e4336171714da8adc21977ce33ce50 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.802294 1166 raft_consensus.cc:740] T 07e4336171714da8adc21977ce33ce50 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.802516 1166 consensus_queue.cc:260] T 07e4336171714da8adc21977ce33ce50 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.803020 1166 ts_tablet_manager.cc:1434] T 07e4336171714da8adc21977ce33ce50 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:51.771560 823 heartbeater.cc:344] Connected to a master server at 127.0.104.60:42087
I20251211 20:52:51.776117 3436 raft_consensus.cc:493] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:51.782977 775 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "58e35de0792a464a818170886ca8d8c9" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:51.803377 3436 raft_consensus.cc:515] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.803303 1166 tablet_bootstrap.cc:492] T 8e09b172980c4e40b1d3f695d8e54c37 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:51.788296 1168 tablet_bootstrap.cc:492] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:51.803872 1168 ts_tablet_manager.cc:1403] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.091s user 0.003s sys 0.000s
I20251211 20:52:51.803992 3436 leader_election.cc:290] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:51.805147 1166 tablet_bootstrap.cc:654] T 8e09b172980c4e40b1d3f695d8e54c37 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.814397 847 leader_election.cc:304] T 58e35de0792a464a818170886ca8d8c9 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:51.814898 3438 raft_consensus.cc:2749] T 58e35de0792a464a818170886ca8d8c9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:51.815469 901 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c112714331bf4bd7a1ddcca17f338d4f" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:51.819396 1168 raft_consensus.cc:359] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.819619 1168 raft_consensus.cc:385] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.819679 1168 raft_consensus.cc:740] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.819854 1168 consensus_queue.cc:260] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.821962 1168 ts_tablet_manager.cc:1434] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.018s user 0.002s sys 0.000s
I20251211 20:52:51.822273 1168 tablet_bootstrap.cc:492] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:51.823760 1168 tablet_bootstrap.cc:654] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:51.826233 711 leader_election.cc:343] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.828972 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 171) took 1187 ms. Trace:
I20251211 20:52:51.829282 3448 raft_consensus.cc:493] T 12aaf618d6474f599458026308d7afdd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:51.830202 3448 raft_consensus.cc:515] T 12aaf618d6474f599458026308d7afdd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.831637 901 raft_consensus.cc:3060] T 5bcc94ec9630484cb7a3e8bf6d4f378a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.831955 3294 rpcz_store.cc:276] 1211 20:52:50.641297 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.641346 (+ 49us) service_pool.cc:225] Handling call
1211 20:52:51.641648 (+1000302us) ts_tablet_manager.cc:1151] Deleting tablet 7d4bba0aa7ea403e8dfcb74cec2e1f9b
1211 20:52:51.641655 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:51.681624 (+ 39969us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:51.760341 (+ 78717us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:51.828947 (+ 68606us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":25300,"tablet-delete.queue_time_us":1000068}
W20251211 20:52:51.829988 710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.3:36845, user_credentials={real_user=slave}} blocked reactor thread for 48726.5us
I20251211 20:52:51.832144 902 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "12aaf618d6474f599458026308d7afdd" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:51.832438 710 consensus_peers.cc:597] T c26c3d0170f64a41ae8944c6668cc5f3 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c26c3d0170f64a41ae8944c6668cc5f3. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:51.832398 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 7d4bba0aa7ea403e8dfcb74cec2e1f9b (table default.Table_6bfc3963d4e346858acdfd9b9016245d [id=4ba40e6e45c7463f9730a2380b5ff856]) successfully deleted
W20251211 20:52:51.831456 711 consensus_peers.cc:597] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:51.833175 983 leader_election.cc:343] T 12aaf618d6474f599458026308d7afdd P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.833482 710 leader_election.cc:304] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:51.834003 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 170) took 1192 ms. Trace:
I20251211 20:52:51.834084 3294 rpcz_store.cc:276] 1211 20:52:50.641200 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.641846 (+ 646us) service_pool.cc:225] Handling call
1211 20:52:51.833888 (+1192042us) ts_tablet_manager.cc:1151] Deleting tablet 6ad6ab53aad14cc2b6d201acfecf6723
1211 20:52:51.833900 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:51.833988 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1191863}
I20251211 20:52:51.834261 3269 raft_consensus.cc:493] T e95b0d8c507a4779a087c8933d464228 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:51.834477 3269 raft_consensus.cc:515] T e95b0d8c507a4779a087c8933d464228 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:51.834554 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 6ad6ab53aad14cc2b6d201acfecf6723 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6ad6ab53aad14cc2b6d201acfecf6723 already in progress: creating tablet
I20251211 20:52:51.835547 902 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e95b0d8c507a4779a087c8933d464228" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:51.835652 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e95b0d8c507a4779a087c8933d464228" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:51.836000 983 leader_election.cc:343] T e95b0d8c507a4779a087c8933d464228 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:51.836740 982 leader_election.cc:343] T e95b0d8c507a4779a087c8933d464228 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.836818 982 leader_election.cc:304] T e95b0d8c507a4779a087c8933d464228 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:51.837056 2974 raft_consensus.cc:2749] T e95b0d8c507a4779a087c8933d464228 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:51.837384 3428 raft_consensus.cc:2804] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:51.837463 3428 raft_consensus.cc:493] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:51.837523 3428 raft_consensus.cc:3060] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.840443 3428 raft_consensus.cc:515] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.841143 3428 leader_election.cc:290] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:51.842106 902 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "062f980b65504d3baf99ae400e521775" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:51.842648 711 leader_election.cc:343] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.835209 3269 leader_election.cc:290] T e95b0d8c507a4779a087c8933d464228 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:51.844982 3448 leader_election.cc:290] T 12aaf618d6474f599458026308d7afdd P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:51.845232 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "12aaf618d6474f599458026308d7afdd" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:51.845700 982 leader_election.cc:343] T 12aaf618d6474f599458026308d7afdd P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.845813 982 leader_election.cc:304] T 12aaf618d6474f599458026308d7afdd P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:51.846460 710 leader_election.cc:304] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:51.846711 3448 raft_consensus.cc:2749] T 12aaf618d6474f599458026308d7afdd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:51.847055 3294 tablet_replica.cc:333] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:51.847256 3294 raft_consensus.cc:2243] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:51.848050 3428 raft_consensus.cc:2804] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:51.848133 3428 raft_consensus.cc:493] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:51.848240 3428 raft_consensus.cc:3060] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.850795 3428 raft_consensus.cc:515] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.852252 3428 leader_election.cc:290] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:51.852949 962 tablet.cc:2378] T e9ead34065e240eeb1f946319537d32a P eb0f1933d9a74fccaa4618047fc05c41: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251211 20:52:51.853685 961 heartbeater.cc:499] Master 127.0.104.61:40749 was elected leader, sending a full tablet report...
I20251211 20:52:51.855225 3436 consensus_queue.cc:1048] T 4fe5c534b572491c911aae01eaa79bae P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:51.855926 3294 raft_consensus.cc:2272] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:51.856667 710 leader_election.cc:304] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:51.857363 3436 raft_consensus.cc:2804] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:51.857473 3436 raft_consensus.cc:697] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:51.857708 3436 consensus_queue.cc:237] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.858145 902 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e59f8afe1834485d9fc64615e7bfe174" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:51.867856 711 leader_election.cc:343] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.868438 3294 ts_tablet_manager.cc:1916] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:51.874816 3295 raft_consensus.cc:2272] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:51.881728 3295 ts_tablet_manager.cc:1916] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:51.881150 1168 tablet_bootstrap.cc:492] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:51.894064 1168 ts_tablet_manager.cc:1403] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.072s user 0.004s sys 0.000s
I20251211 20:52:51.895038 1168 raft_consensus.cc:359] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:51.881753 711 consensus_peers.cc:597] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:51.895911 3448 raft_consensus.cc:493] T 47428f41973642dbbd42ba051ea3d3b0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:51.896102 3448 raft_consensus.cc:515] T 47428f41973642dbbd42ba051ea3d3b0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.897152 902 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "47428f41973642dbbd42ba051ea3d3b0" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:51.897311 3448 leader_election.cc:290] T 47428f41973642dbbd42ba051ea3d3b0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
W20251211 20:52:51.897696 983 leader_election.cc:343] T 47428f41973642dbbd42ba051ea3d3b0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.897526 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "47428f41973642dbbd42ba051ea3d3b0" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:51.898224 982 leader_election.cc:343] T 47428f41973642dbbd42ba051ea3d3b0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:51.898327 982 leader_election.cc:304] T 47428f41973642dbbd42ba051ea3d3b0 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:51.898779 1160 tablet_bootstrap.cc:492] T 6029c1fd563d4023a8cf7cfb8ffa0876 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:51.898777 3448 raft_consensus.cc:2749] T 47428f41973642dbbd42ba051ea3d3b0 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:51.899003 825 tablet.cc:2378] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251211 20:52:51.899013 1160 ts_tablet_manager.cc:1403] T 6029c1fd563d4023a8cf7cfb8ffa0876 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.109s user 0.001s sys 0.006s
I20251211 20:52:51.899737 1160 raft_consensus.cc:359] T 6029c1fd563d4023a8cf7cfb8ffa0876 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.899893 1160 raft_consensus.cc:385] T 6029c1fd563d4023a8cf7cfb8ffa0876 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.899942 1160 raft_consensus.cc:740] T 6029c1fd563d4023a8cf7cfb8ffa0876 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.900154 1160 consensus_queue.cc:260] T 6029c1fd563d4023a8cf7cfb8ffa0876 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.907416 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c112714331bf4bd7a1ddcca17f338d4f" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:51.907816 1033 raft_consensus.cc:2468] T c112714331bf4bd7a1ddcca17f338d4f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
W20251211 20:52:51.910902 711 consensus_peers.cc:597] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:51.911723 1160 ts_tablet_manager.cc:1434] T 6029c1fd563d4023a8cf7cfb8ffa0876 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20251211 20:52:51.912082 1160 tablet_bootstrap.cc:492] T 01104975fd8b470aa026578b97ff6b8d P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:51.914834 1160 tablet_bootstrap.cc:654] T 01104975fd8b470aa026578b97ff6b8d P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.916824 1168 raft_consensus.cc:385] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.916944 1168 raft_consensus.cc:740] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.917143 1168 consensus_queue.cc:260] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.917243 3438 raft_consensus.cc:493] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:51.917353 3438 raft_consensus.cc:515] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.917757 1168 ts_tablet_manager.cc:1434] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.024s user 0.002s sys 0.000s
I20251211 20:52:51.917910 3438 leader_election.cc:290] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:51.919149 847 leader_election.cc:343] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 793f543d6b614ad7b3ab8e717db2ceaf
W20251211 20:52:51.920177 847 leader_election.cc:343] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 793f543d6b614ad7b3ab8e717db2ceaf
I20251211 20:52:51.920279 847 leader_election.cc:304] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:51.920598 3398 raft_consensus.cc:493] T fd43a27db69044b68826cda87e16a086 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:51.920709 3398 raft_consensus.cc:515] T fd43a27db69044b68826cda87e16a086 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.921263 3398 leader_election.cc:290] T fd43a27db69044b68826cda87e16a086 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:51.921378 3398 raft_consensus.cc:2749] T 793f543d6b614ad7b3ab8e717db2ceaf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:51.922019 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "062f980b65504d3baf99ae400e521775" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:51.922257 1033 raft_consensus.cc:3060] T 062f980b65504d3baf99ae400e521775 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.922658 1040 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd43a27db69044b68826cda87e16a086" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:51.923455 2962 ts_tablet_manager.cc:1929] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:51.923558 2962 log.cc:1199] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/5bcc94ec9630484cb7a3e8bf6d4f378a
I20251211 20:52:51.926342 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd43a27db69044b68826cda87e16a086" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:51.927054 847 leader_election.cc:304] T fd43a27db69044b68826cda87e16a086 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:51.927513 3438 raft_consensus.cc:2749] T fd43a27db69044b68826cda87e16a086 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:51.929857 1035 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e59f8afe1834485d9fc64615e7bfe174" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:51.930039 1035 raft_consensus.cc:3060] T e59f8afe1834485d9fc64615e7bfe174 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.929765 2962 ts_tablet_manager.cc:1950] T 5bcc94ec9630484cb7a3e8bf6d4f378a P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:51.933109 1035 raft_consensus.cc:2468] T e59f8afe1834485d9fc64615e7bfe174 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:51.926113 1033 raft_consensus.cc:2468] T 062f980b65504d3baf99ae400e521775 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:51.936650 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 158) took 2827 ms. Trace:
I20251211 20:52:51.937498 2962 rpcz_store.cc:276] 1211 20:52:49.108930 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.109004 (+ 74us) service_pool.cc:225] Handling call
1211 20:52:51.782231 (+2673227us) ts_tablet_manager.cc:1151] Deleting tablet 5bcc94ec9630484cb7a3e8bf6d4f378a
1211 20:52:51.782240 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:51.847775 (+ 65535us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:51.923345 (+ 75570us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:51.936624 (+ 13279us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2662397}
I20251211 20:52:51.939345 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 159) took 2830 ms. Trace:
I20251211 20:52:51.939818 2962 rpcz_store.cc:276] 1211 20:52:49.109001 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.109084 (+ 83us) service_pool.cc:225] Handling call
1211 20:52:51.938697 (+2829613us) ts_tablet_manager.cc:1151] Deleting tablet 3dde7ed495e64909bd9784dc5eefb04c
1211 20:52:51.938805 (+ 108us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:51.939225 (+ 420us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2818804}
I20251211 20:52:51.941082 2962 tablet_replica.cc:333] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:51.943096 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 5bcc94ec9630484cb7a3e8bf6d4f378a (table default.Table_da0851457f78469c9a4b783e05c6a58e [id=c1d4515be510468fb7d972514e6c87d9]) successfully deleted
I20251211 20:52:51.942867 2962 raft_consensus.cc:2243] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
W20251211 20:52:51.943673 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet 3dde7ed495e64909bd9784dc5eefb04c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3dde7ed495e64909bd9784dc5eefb04c already in progress: creating tablet
I20251211 20:52:51.943687 2962 raft_consensus.cc:2272] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:51.947919 711 consensus_peers.cc:597] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:51.949529 2962 ts_tablet_manager.cc:1916] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:51.950987 1166 tablet_bootstrap.cc:492] T 8e09b172980c4e40b1d3f695d8e54c37 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:51.951193 1166 ts_tablet_manager.cc:1403] T 8e09b172980c4e40b1d3f695d8e54c37 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.148s user 0.031s sys 0.021s
I20251211 20:52:51.951193 901 raft_consensus.cc:3060] T 58e35de0792a464a818170886ca8d8c9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:51.952163 1166 raft_consensus.cc:359] T 8e09b172980c4e40b1d3f695d8e54c37 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.952404 1166 raft_consensus.cc:385] T 8e09b172980c4e40b1d3f695d8e54c37 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.952507 1166 raft_consensus.cc:740] T 8e09b172980c4e40b1d3f695d8e54c37 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.952786 1166 consensus_queue.cc:260] T 8e09b172980c4e40b1d3f695d8e54c37 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.953456 1166 ts_tablet_manager.cc:1434] T 8e09b172980c4e40b1d3f695d8e54c37 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:51.954181 1166 tablet_bootstrap.cc:492] T 13ceabb190494e718a52892faaa53986 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:51.956086 1166 tablet_bootstrap.cc:654] T 13ceabb190494e718a52892faaa53986 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.964779 3438 raft_consensus.cc:493] T a7979798e92e45eb969197c8c7ac2580 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:51.964931 3438 raft_consensus.cc:515] T a7979798e92e45eb969197c8c7ac2580 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.965502 3438 leader_election.cc:290] T a7979798e92e45eb969197c8c7ac2580 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:51.966343 1035 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a7979798e92e45eb969197c8c7ac2580" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:51.967145 1160 tablet_bootstrap.cc:492] T 01104975fd8b470aa026578b97ff6b8d P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:51.967398 1160 ts_tablet_manager.cc:1403] T 01104975fd8b470aa026578b97ff6b8d P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.055s user 0.001s sys 0.008s
I20251211 20:52:51.976610 1160 raft_consensus.cc:359] T 01104975fd8b470aa026578b97ff6b8d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.977660 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a7979798e92e45eb969197c8c7ac2580" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:51.989974 1166 tablet_bootstrap.cc:492] T 13ceabb190494e718a52892faaa53986 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:51.990149 1166 ts_tablet_manager.cc:1403] T 13ceabb190494e718a52892faaa53986 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.036s user 0.008s sys 0.006s
I20251211 20:52:51.991315 1166 raft_consensus.cc:359] T 13ceabb190494e718a52892faaa53986 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.991503 1166 raft_consensus.cc:385] T 13ceabb190494e718a52892faaa53986 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.991569 1166 raft_consensus.cc:740] T 13ceabb190494e718a52892faaa53986 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.993080 1166 consensus_queue.cc:260] T 13ceabb190494e718a52892faaa53986 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.993594 1166 ts_tablet_manager.cc:1434] T 13ceabb190494e718a52892faaa53986 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251211 20:52:51.993872 1166 tablet_bootstrap.cc:492] T fbc2d4aea084492c8270ad1d87096e58 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:51.988214 847 leader_election.cc:304] T a7979798e92e45eb969197c8c7ac2580 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:51.995453 3398 raft_consensus.cc:2749] T a7979798e92e45eb969197c8c7ac2580 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:51.995602 1166 tablet_bootstrap.cc:654] T fbc2d4aea084492c8270ad1d87096e58 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:51.977378 1160 raft_consensus.cc:385] T 01104975fd8b470aa026578b97ff6b8d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:51.996167 1160 raft_consensus.cc:740] T 01104975fd8b470aa026578b97ff6b8d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:51.997188 1160 consensus_queue.cc:260] T 01104975fd8b470aa026578b97ff6b8d P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:51.999202 1166 tablet_bootstrap.cc:492] T fbc2d4aea084492c8270ad1d87096e58 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:51.999383 1166 ts_tablet_manager.cc:1403] T fbc2d4aea084492c8270ad1d87096e58 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20251211 20:52:52.001163 1166 raft_consensus.cc:359] T fbc2d4aea084492c8270ad1d87096e58 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.001381 1166 raft_consensus.cc:385] T fbc2d4aea084492c8270ad1d87096e58 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.001480 1166 raft_consensus.cc:740] T fbc2d4aea084492c8270ad1d87096e58 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.001700 1166 consensus_queue.cc:260] T fbc2d4aea084492c8270ad1d87096e58 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.002259 1166 ts_tablet_manager.cc:1434] T fbc2d4aea084492c8270ad1d87096e58 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20251211 20:52:52.002573 1166 tablet_bootstrap.cc:492] T aeff1fbabf7845dc9863f59f286ddcec P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:52.003125 1160 ts_tablet_manager.cc:1434] T 01104975fd8b470aa026578b97ff6b8d P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.036s user 0.003s sys 0.000s
I20251211 20:52:52.004220 1166 tablet_bootstrap.cc:654] T aeff1fbabf7845dc9863f59f286ddcec P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.007092 1160 tablet_bootstrap.cc:492] T 98d575ac680d49d0990877666638f013 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:52.011063 1168 tablet_bootstrap.cc:492] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:52.013557 1168 tablet_bootstrap.cc:654] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.026114 1160 tablet_bootstrap.cc:654] T 98d575ac680d49d0990877666638f013 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.028187 3436 consensus_queue.cc:1048] T 969a8587019c4202b6e705d126f83d44 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:52.029066 3438 raft_consensus.cc:493] T 01104975fd8b470aa026578b97ff6b8d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.029211 3438 raft_consensus.cc:515] T 01104975fd8b470aa026578b97ff6b8d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.029783 3438 leader_election.cc:290] T 01104975fd8b470aa026578b97ff6b8d P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:52.030268 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01104975fd8b470aa026578b97ff6b8d" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:52.030721 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01104975fd8b470aa026578b97ff6b8d" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:52.031726 847 leader_election.cc:304] T 01104975fd8b470aa026578b97ff6b8d P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:52.032135 3438 raft_consensus.cc:2749] T 01104975fd8b470aa026578b97ff6b8d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:52.033517 1166 tablet_bootstrap.cc:492] T aeff1fbabf7845dc9863f59f286ddcec P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:52.033712 1166 ts_tablet_manager.cc:1403] T aeff1fbabf7845dc9863f59f286ddcec P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.031s user 0.004s sys 0.006s
I20251211 20:52:52.034386 3438 raft_consensus.cc:493] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.034454 1166 raft_consensus.cc:359] T aeff1fbabf7845dc9863f59f286ddcec P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.034502 3438 raft_consensus.cc:515] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.034613 1166 raft_consensus.cc:385] T aeff1fbabf7845dc9863f59f286ddcec P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.034660 1166 raft_consensus.cc:740] T aeff1fbabf7845dc9863f59f286ddcec P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.034842 1166 consensus_queue.cc:260] T aeff1fbabf7845dc9863f59f286ddcec P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.034999 3438 leader_election.cc:290] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:52.035312 1166 ts_tablet_manager.cc:1434] T aeff1fbabf7845dc9863f59f286ddcec P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251211 20:52:52.035604 1166 tablet_bootstrap.cc:492] T fcf5afcfd7f142d29ff91272913e5fcb P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
W20251211 20:52:52.036026 847 leader_election.cc:343] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: a2058b67ddf1413db338bfa2f64e7a77
I20251211 20:52:52.037285 1166 tablet_bootstrap.cc:654] T fcf5afcfd7f142d29ff91272913e5fcb P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.037237 710 leader_election.cc:304] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.038945 2962 ts_tablet_manager.cc:1929] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:52.039151 2962 log.cc:1199] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/58e35de0792a464a818170886ca8d8c9
I20251211 20:52:52.039815 2962 ts_tablet_manager.cc:1950] T 58e35de0792a464a818170886ca8d8c9 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:52.050328 3448 raft_consensus.cc:493] T 6663f13691a24fb991b48f543d9de76b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.050495 3448 raft_consensus.cc:515] T 6663f13691a24fb991b48f543d9de76b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.051131 3448 leader_election.cc:290] T 6663f13691a24fb991b48f543d9de76b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:52.054047 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6663f13691a24fb991b48f543d9de76b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:52.054639 982 leader_election.cc:343] T 6663f13691a24fb991b48f543d9de76b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.054674 901 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6663f13691a24fb991b48f543d9de76b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:52.055322 983 leader_election.cc:343] T 6663f13691a24fb991b48f543d9de76b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.055433 983 leader_election.cc:304] T 6663f13691a24fb991b48f543d9de76b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.055936 3448 raft_consensus.cc:2749] T 6663f13691a24fb991b48f543d9de76b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:52.058089 847 leader_election.cc:343] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: a2058b67ddf1413db338bfa2f64e7a77
I20251211 20:52:52.058205 847 leader_election.cc:304] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:52.058616 3438 raft_consensus.cc:2749] T a2058b67ddf1413db338bfa2f64e7a77 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:52.061048 3448 raft_consensus.cc:493] T fbc2d4aea084492c8270ad1d87096e58 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.061182 3448 raft_consensus.cc:515] T fbc2d4aea084492c8270ad1d87096e58 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.061748 3448 leader_election.cc:290] T fbc2d4aea084492c8270ad1d87096e58 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:52.062733 902 raft_consensus.cc:3060] T fd43a27db69044b68826cda87e16a086 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:52.066759 3457 raft_consensus.cc:2804] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:52.066922 3457 raft_consensus.cc:493] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:52.067063 3457 raft_consensus.cc:3060] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:52.068943 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fbc2d4aea084492c8270ad1d87096e58" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:52.069623 982 leader_election.cc:343] T fbc2d4aea084492c8270ad1d87096e58 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.070941 3457 raft_consensus.cc:515] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.071967 3457 leader_election.cc:290] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:52.071923 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c112714331bf4bd7a1ddcca17f338d4f" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:52.072418 711 leader_election.cc:343] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.072834 901 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fbc2d4aea084492c8270ad1d87096e58" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:52.073369 983 leader_election.cc:343] T fbc2d4aea084492c8270ad1d87096e58 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.073482 983 leader_election.cc:304] T fbc2d4aea084492c8270ad1d87096e58 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.073922 3448 raft_consensus.cc:2749] T fbc2d4aea084492c8270ad1d87096e58 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:52.074431 710 consensus_peers.cc:597] T 5bcc94ec9630484cb7a3e8bf6d4f378a P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:52.075171 3448 raft_consensus.cc:493] T aeff1fbabf7845dc9863f59f286ddcec P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.075289 3448 raft_consensus.cc:515] T aeff1fbabf7845dc9863f59f286ddcec P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.075752 1160 tablet_bootstrap.cc:492] T 98d575ac680d49d0990877666638f013 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:52.075850 3448 leader_election.cc:290] T aeff1fbabf7845dc9863f59f286ddcec P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:52.075875 1160 ts_tablet_manager.cc:1403] T 98d575ac680d49d0990877666638f013 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.070s user 0.002s sys 0.001s
I20251211 20:52:52.076546 3428 consensus_queue.cc:1048] T 834b84a557b04e78b5d2c8c84520a644 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:52.076589 902 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aeff1fbabf7845dc9863f59f286ddcec" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:52.077175 1160 raft_consensus.cc:359] T 98d575ac680d49d0990877666638f013 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.077378 1160 raft_consensus.cc:385] T 98d575ac680d49d0990877666638f013 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.077476 1160 raft_consensus.cc:740] T 98d575ac680d49d0990877666638f013 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.077739 1160 consensus_queue.cc:260] T 98d575ac680d49d0990877666638f013 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.078186 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 160) took 2969 ms. Trace:
I20251211 20:52:52.078279 1160 ts_tablet_manager.cc:1434] T 98d575ac680d49d0990877666638f013 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20251211 20:52:52.078418 983 leader_election.cc:343] T aeff1fbabf7845dc9863f59f286ddcec P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.078536 1160 tablet_bootstrap.cc:492] T a295a24e81f14e5fa126a4a58ffd08cf P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:52.079382 902 raft_consensus.cc:3060] T a7979798e92e45eb969197c8c7ac2580 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:52.080323 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aeff1fbabf7845dc9863f59f286ddcec" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:52.083009 982 leader_election.cc:343] T aeff1fbabf7845dc9863f59f286ddcec P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.083154 982 leader_election.cc:304] T aeff1fbabf7845dc9863f59f286ddcec P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.083387 3448 raft_consensus.cc:2749] T aeff1fbabf7845dc9863f59f286ddcec P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:52.078423 2962 rpcz_store.cc:276] 1211 20:52:49.109067 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.109488 (+ 421us) service_pool.cc:225] Handling call
1211 20:52:51.940916 (+2831428us) ts_tablet_manager.cc:1151] Deleting tablet 58e35de0792a464a818170886ca8d8c9
1211 20:52:51.940929 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.004844 (+ 63915us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.038793 (+ 33949us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.078165 (+ 39372us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2820887}
I20251211 20:52:52.086786 2962 tablet_replica.cc:333] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:52.087046 2962 raft_consensus.cc:2243] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:52.087281 2962 raft_consensus.cc:2272] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:52.087852 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 58e35de0792a464a818170886ca8d8c9 (table default.Table_43474d2631e0443d9eaecae68a1c110d [id=e50846ecd18341919707dfa446765ce1]) successfully deleted
I20251211 20:52:52.089636 2962 ts_tablet_manager.cc:1916] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:52.091248 824 heartbeater.cc:499] Master 127.0.104.61:40749 was elected leader, sending a full tablet report...
I20251211 20:52:52.096206 3294 ts_tablet_manager.cc:1929] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:52.096467 3294 log.cc:1199] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/2e33606930774fe4aafc6e3819618af8
I20251211 20:52:52.098068 3294 ts_tablet_manager.cc:1950] T 2e33606930774fe4aafc6e3819618af8 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:52.098433 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 169) took 1457 ms. Trace:
I20251211 20:52:52.098568 3294 rpcz_store.cc:276] 1211 20:52:50.641090 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.642052 (+ 962us) service_pool.cc:225] Handling call
1211 20:52:51.847003 (+1204951us) ts_tablet_manager.cc:1151] Deleting tablet 2e33606930774fe4aafc6e3819618af8
1211 20:52:51.847016 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:51.946771 (+ 99755us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.043492 (+ 96721us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.098416 (+ 54924us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8307,"tablet-delete.queue_time_us":1204834}
I20251211 20:52:52.098981 3438 raft_consensus.cc:493] T e9ead34065e240eeb1f946319537d32a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.099124 3438 raft_consensus.cc:515] T e9ead34065e240eeb1f946319537d32a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.099748 3438 leader_election.cc:290] T e9ead34065e240eeb1f946319537d32a P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:52.100473 1166 tablet_bootstrap.cc:492] T fcf5afcfd7f142d29ff91272913e5fcb P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:52.100677 1166 ts_tablet_manager.cc:1403] T fcf5afcfd7f142d29ff91272913e5fcb P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.065s user 0.008s sys 0.016s
I20251211 20:52:52.100708 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 2e33606930774fe4aafc6e3819618af8 (table default.Table_290ed4f01870438ca11f93a98c4ce8e8 [id=9e23a3e5095f4a358fe7859424c27d59]) successfully deleted
I20251211 20:52:52.101151 3294 tablet_replica.cc:333] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:52.101377 3294 raft_consensus.cc:2243] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:52.101575 1166 raft_consensus.cc:359] T fcf5afcfd7f142d29ff91272913e5fcb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.101683 3294 raft_consensus.cc:2272] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:52.101822 1166 raft_consensus.cc:385] T fcf5afcfd7f142d29ff91272913e5fcb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.101938 1166 raft_consensus.cc:740] T fcf5afcfd7f142d29ff91272913e5fcb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.102201 3448 raft_consensus.cc:493] T b115d737d7fc483d9f5a1f040b511bae P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.102231 3294 ts_tablet_manager.cc:1916] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:52.102311 3448 raft_consensus.cc:515] T b115d737d7fc483d9f5a1f040b511bae P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.102867 3448 leader_election.cc:290] T b115d737d7fc483d9f5a1f040b511bae P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:52.103351 1166 consensus_queue.cc:260] T fcf5afcfd7f142d29ff91272913e5fcb P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.103487 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b115d737d7fc483d9f5a1f040b511bae" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:52.099511 710 leader_election.cc:304] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.100297 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e9ead34065e240eeb1f946319537d32a" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:52.104051 1166 ts_tablet_manager.cc:1434] T fcf5afcfd7f142d29ff91272913e5fcb P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251211 20:52:52.104151 3428 raft_consensus.cc:2804] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:52.104269 3428 raft_consensus.cc:697] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
W20251211 20:52:52.104331 982 leader_election.cc:343] T b115d737d7fc483d9f5a1f040b511bae P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.104540 1166 tablet_bootstrap.cc:492] T 43044ba5660345eaa2b4e6562cc8d448 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:52.104485 3428 consensus_queue.cc:237] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.104668 902 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b115d737d7fc483d9f5a1f040b511bae" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:52.105093 983 leader_election.cc:343] T b115d737d7fc483d9f5a1f040b511bae P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.105190 983 leader_election.cc:304] T b115d737d7fc483d9f5a1f040b511bae P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.105505 3448 raft_consensus.cc:2749] T b115d737d7fc483d9f5a1f040b511bae P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:52.105512 847 leader_election.cc:343] T e9ead34065e240eeb1f946319537d32a P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: e9ead34065e240eeb1f946319537d32a
W20251211 20:52:52.105708 847 leader_election.cc:343] T e9ead34065e240eeb1f946319537d32a P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.105794 847 leader_election.cc:304] T e9ead34065e240eeb1f946319537d32a P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:52.106159 3438 raft_consensus.cc:2749] T e9ead34065e240eeb1f946319537d32a P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:52.106492 3295 ts_tablet_manager.cc:1929] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:52.106649 3295 log.cc:1199] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/68792ccef54944a09af9b5456e9ff1fe
I20251211 20:52:52.107074 3295 ts_tablet_manager.cc:1950] T 68792ccef54944a09af9b5456e9ff1fe P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:52.107110 1166 tablet_bootstrap.cc:654] T 43044ba5660345eaa2b4e6562cc8d448 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.109941 1168 tablet_bootstrap.cc:492] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:52.110144 1168 ts_tablet_manager.cc:1403] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.099s user 0.003s sys 0.001s
I20251211 20:52:52.110972 1168 raft_consensus.cc:359] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.111826 1168 raft_consensus.cc:385] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.111933 1168 raft_consensus.cc:740] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.112183 1168 consensus_queue.cc:260] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.112803 1168 ts_tablet_manager.cc:1434] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251211 20:52:52.113183 1168 tablet_bootstrap.cc:492] T c8214c97ed1640f5965aa5c4b999e2f1 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:52.115046 1168 tablet_bootstrap.cc:654] T c8214c97ed1640f5965aa5c4b999e2f1 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.124922 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 136) took 1238 ms. Trace:
I20251211 20:52:52.125139 3295 rpcz_store.cc:276] 1211 20:52:50.886507 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.886552 (+ 45us) service_pool.cc:225] Handling call
1211 20:52:51.793642 (+907090us) ts_tablet_manager.cc:1151] Deleting tablet 68792ccef54944a09af9b5456e9ff1fe
1211 20:52:51.793648 (+ 6us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:51.885497 (+ 91849us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.106386 (+220889us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.124890 (+ 18504us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":906948}
I20251211 20:52:52.125957 1783 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 68792ccef54944a09af9b5456e9ff1fe (table default.Table_00bd643f7bb84217970178aefb485a69 [id=1b90cdd71d064d27948a1a9a7f3f852b]) successfully deleted
I20251211 20:52:52.126521 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 137) took 1234 ms. Trace:
I20251211 20:52:52.126680 3295 rpcz_store.cc:276] 1211 20:52:50.892459 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.892519 (+ 60us) service_pool.cc:225] Handling call
1211 20:52:52.126378 (+1233859us) ts_tablet_manager.cc:1151] Deleting tablet d0673c37c2fd47eea8ab10d9e91b8d7a
1211 20:52:52.126390 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.126501 (+ 111us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1233654}
W20251211 20:52:52.127229 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet d0673c37c2fd47eea8ab10d9e91b8d7a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d0673c37c2fd47eea8ab10d9e91b8d7a already in progress: creating tablet
I20251211 20:52:52.127866 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 138) took 1234 ms. Trace:
I20251211 20:52:52.128087 3295 rpcz_store.cc:276] 1211 20:52:50.893283 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.893332 (+ 49us) service_pool.cc:225] Handling call
1211 20:52:52.127733 (+1234401us) ts_tablet_manager.cc:1151] Deleting tablet 87c4ec07e82d4bf79b4eeefe732674ff
1211 20:52:52.127744 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.127849 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1234214}
W20251211 20:52:52.128582 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 87c4ec07e82d4bf79b4eeefe732674ff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 87c4ec07e82d4bf79b4eeefe732674ff already in progress: creating tablet
I20251211 20:52:52.128931 3295 tablet_replica.cc:333] T 1578887b2f5748cd83a6fbdec2f3fc99 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:52.129159 3295 raft_consensus.cc:2243] T 1578887b2f5748cd83a6fbdec2f3fc99 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:52.129364 3295 raft_consensus.cc:2272] T 1578887b2f5748cd83a6fbdec2f3fc99 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:52.129980 3295 ts_tablet_manager.cc:1916] T 1578887b2f5748cd83a6fbdec2f3fc99 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:52.148487 1166 tablet_bootstrap.cc:492] T 43044ba5660345eaa2b4e6562cc8d448 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:52.148733 1166 ts_tablet_manager.cc:1403] T 43044ba5660345eaa2b4e6562cc8d448 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.044s user 0.009s sys 0.007s
I20251211 20:52:52.149577 1166 raft_consensus.cc:359] T 43044ba5660345eaa2b4e6562cc8d448 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.149827 1166 raft_consensus.cc:385] T 43044ba5660345eaa2b4e6562cc8d448 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.149935 1166 raft_consensus.cc:740] T 43044ba5660345eaa2b4e6562cc8d448 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.150229 1166 consensus_queue.cc:260] T 43044ba5660345eaa2b4e6562cc8d448 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.116822 710 leader_election.cc:304] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.150889 1166 ts_tablet_manager.cc:1434] T 43044ba5660345eaa2b4e6562cc8d448 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:52.151219 1166 tablet_bootstrap.cc:492] T cc43ee61e78249bfa00b11f3a1fa21a3 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:52.151539 2429 raft_consensus.cc:2804] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:52.151687 2429 raft_consensus.cc:697] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:52.151916 2429 consensus_queue.cc:237] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.153228 1166 tablet_bootstrap.cc:654] T cc43ee61e78249bfa00b11f3a1fa21a3 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.153769 3436 consensus_queue.cc:1048] T 79d92736bb1c46d2bfa5678b2d396ec2 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:52.154546 1160 tablet_bootstrap.cc:654] T a295a24e81f14e5fa126a4a58ffd08cf P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:52.155148 710 consensus_peers.cc:597] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3acfef6f082641a199b80eefcfc772ac. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:52.156495 710 consensus_peers.cc:597] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d9cf0d92ccc7419cb6e7694e8e5c7bfb. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:52.158179 1166 tablet_bootstrap.cc:492] T cc43ee61e78249bfa00b11f3a1fa21a3 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:52.158356 1166 ts_tablet_manager.cc:1403] T cc43ee61e78249bfa00b11f3a1fa21a3 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20251211 20:52:52.159096 1166 raft_consensus.cc:359] T cc43ee61e78249bfa00b11f3a1fa21a3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.159322 1166 raft_consensus.cc:385] T cc43ee61e78249bfa00b11f3a1fa21a3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.159407 1166 raft_consensus.cc:740] T cc43ee61e78249bfa00b11f3a1fa21a3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.159693 1166 consensus_queue.cc:260] T cc43ee61e78249bfa00b11f3a1fa21a3 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.160378 1166 ts_tablet_manager.cc:1434] T cc43ee61e78249bfa00b11f3a1fa21a3 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:52.160687 1166 tablet_bootstrap.cc:492] T 664c6a3afe5b4a4caf87dede87f199ba P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:52.162331 1166 tablet_bootstrap.cc:654] T 664c6a3afe5b4a4caf87dede87f199ba P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:52.166918 711 consensus_peers.cc:597] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:52.172766 1040 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c112714331bf4bd7a1ddcca17f338d4f" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:52.172953 1040 raft_consensus.cc:3060] T c112714331bf4bd7a1ddcca17f338d4f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:52.183234 3436 raft_consensus.cc:493] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.183506 3436 raft_consensus.cc:515] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.175505 1040 raft_consensus.cc:2468] T c112714331bf4bd7a1ddcca17f338d4f P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:52.184208 3436 leader_election.cc:290] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:52.185225 902 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "116b21a182474ca191c4a89e9474677d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:52.185786 711 leader_election.cc:343] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.194223 1041 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "116b21a182474ca191c4a89e9474677d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:52.194435 1041 raft_consensus.cc:2468] T 116b21a182474ca191c4a89e9474677d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:52.194973 3457 consensus_queue.cc:1048] T f9a3fdf17f274c15ac0bdfa9c174f649 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251211 20:52:52.200016 711 consensus_peers.cc:597] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:52.208328 710 leader_election.cc:304] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.208596 3000 raft_consensus.cc:2804] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:52.208715 3000 raft_consensus.cc:697] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:52.208946 3000 consensus_queue.cc:237] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.219394 1168 tablet_bootstrap.cc:492] T c8214c97ed1640f5965aa5c4b999e2f1 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:52.220660 1168 ts_tablet_manager.cc:1403] T c8214c97ed1640f5965aa5c4b999e2f1 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.108s user 0.002s sys 0.002s
I20251211 20:52:52.224768 2962 ts_tablet_manager.cc:1929] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:52.228510 710 leader_election.cc:304] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.231002 1160 tablet_bootstrap.cc:492] T a295a24e81f14e5fa126a4a58ffd08cf P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:52.231160 1160 ts_tablet_manager.cc:1403] T a295a24e81f14e5fa126a4a58ffd08cf P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.153s user 0.008s sys 0.002s
I20251211 20:52:52.231840 1160 raft_consensus.cc:359] T a295a24e81f14e5fa126a4a58ffd08cf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.232002 1160 raft_consensus.cc:385] T a295a24e81f14e5fa126a4a58ffd08cf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.232049 1160 raft_consensus.cc:740] T a295a24e81f14e5fa126a4a58ffd08cf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.232223 1160 consensus_queue.cc:260] T a295a24e81f14e5fa126a4a58ffd08cf P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.233129 1160 ts_tablet_manager.cc:1434] T a295a24e81f14e5fa126a4a58ffd08cf P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:52.229519 1168 raft_consensus.cc:359] T c8214c97ed1640f5965aa5c4b999e2f1 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.234165 1168 raft_consensus.cc:385] T c8214c97ed1640f5965aa5c4b999e2f1 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.234973 1168 raft_consensus.cc:740] T c8214c97ed1640f5965aa5c4b999e2f1 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.236825 3436 raft_consensus.cc:2804] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:52.236959 3436 raft_consensus.cc:493] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:52.237076 3436 raft_consensus.cc:3060] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:52.237020 1168 consensus_queue.cc:260] T c8214c97ed1640f5965aa5c4b999e2f1 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.241190 1160 tablet_bootstrap.cc:492] T bd2bf1113dfe4b67b263b56f3d3db07d P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:52.242735 1160 tablet_bootstrap.cc:654] T bd2bf1113dfe4b67b263b56f3d3db07d P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.244024 3436 raft_consensus.cc:515] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.244412 1168 ts_tablet_manager.cc:1434] T c8214c97ed1640f5965aa5c4b999e2f1 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.023s user 0.001s sys 0.001s
I20251211 20:52:52.244941 901 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "116b21a182474ca191c4a89e9474677d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:52.245378 711 leader_election.cc:343] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.245631 3436 leader_election.cc:290] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:52.246026 1168 tablet_bootstrap.cc:492] T c549c2bfebd64b87a4fc8ce8210e14f6 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:52.235855 1166 tablet_bootstrap.cc:492] T 664c6a3afe5b4a4caf87dede87f199ba P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:52.248697 1166 ts_tablet_manager.cc:1403] T 664c6a3afe5b4a4caf87dede87f199ba P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.088s user 0.004s sys 0.002s
I20251211 20:52:52.249570 1166 raft_consensus.cc:359] T 664c6a3afe5b4a4caf87dede87f199ba P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.249745 1166 raft_consensus.cc:385] T 664c6a3afe5b4a4caf87dede87f199ba P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.249790 1166 raft_consensus.cc:740] T 664c6a3afe5b4a4caf87dede87f199ba P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.250098 1166 consensus_queue.cc:260] T 664c6a3afe5b4a4caf87dede87f199ba P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.250619 1166 ts_tablet_manager.cc:1434] T 664c6a3afe5b4a4caf87dede87f199ba P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:52.250907 1166 tablet_bootstrap.cc:492] T ab4e7a9062134e489d6f9abe6ff26fb4 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:52.252342 1160 tablet_bootstrap.cc:492] T bd2bf1113dfe4b67b263b56f3d3db07d P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:52.252765 1160 ts_tablet_manager.cc:1403] T bd2bf1113dfe4b67b263b56f3d3db07d P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20251211 20:52:52.253142 1166 tablet_bootstrap.cc:654] T ab4e7a9062134e489d6f9abe6ff26fb4 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.254393 1160 raft_consensus.cc:359] T bd2bf1113dfe4b67b263b56f3d3db07d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.254850 1160 raft_consensus.cc:385] T bd2bf1113dfe4b67b263b56f3d3db07d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.255062 1160 raft_consensus.cc:740] T bd2bf1113dfe4b67b263b56f3d3db07d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.255904 2962 log.cc:1199] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/215f8dc74a71482daa2ca80e15010f51
I20251211 20:52:52.255548 1160 consensus_queue.cc:260] T bd2bf1113dfe4b67b263b56f3d3db07d P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.256573 2962 ts_tablet_manager.cc:1950] T 215f8dc74a71482daa2ca80e15010f51 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:52.258059 1160 ts_tablet_manager.cc:1434] T bd2bf1113dfe4b67b263b56f3d3db07d P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20251211 20:52:52.259366 1160 tablet_bootstrap.cc:492] T 84b2a67c836844ceb3628cc3b7d8ba10 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:52.265498 1168 tablet_bootstrap.cc:654] T c549c2bfebd64b87a4fc8ce8210e14f6 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.271402 3423 raft_consensus.cc:493] T 8e09b172980c4e40b1d3f695d8e54c37 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.274858 3423 raft_consensus.cc:515] T 8e09b172980c4e40b1d3f695d8e54c37 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.275660 3423 leader_election.cc:290] T 8e09b172980c4e40b1d3f695d8e54c37 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:52.277232 901 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8e09b172980c4e40b1d3f695d8e54c37" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:52.277801 983 leader_election.cc:343] T 8e09b172980c4e40b1d3f695d8e54c37 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.278079 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 153) took 3169 ms. Trace:
I20251211 20:52:52.280416 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8e09b172980c4e40b1d3f695d8e54c37" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:52.283120 3294 ts_tablet_manager.cc:1929] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:52.296828 1160 tablet_bootstrap.cc:654] T 84b2a67c836844ceb3628cc3b7d8ba10 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.311323 3385 log.cc:826] T 00000000000000000000000000000000 P 9230f94a6a1a49cead51aeea9f069087: Log is configured to *not* fsync() on all Append() calls
I20251211 20:52:52.318233 2962 rpcz_store.cc:276] 1211 20:52:49.108112 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.111642 (+ 3530us) service_pool.cc:225] Handling call
1211 20:52:52.086724 (+2975082us) ts_tablet_manager.cc:1151] Deleting tablet 215f8dc74a71482daa2ca80e15010f51
1211 20:52:52.086739 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.092855 (+ 6116us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.224636 (+131781us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.278062 (+ 53426us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2966713}
I20251211 20:52:52.319017 2962 tablet_replica.cc:333] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:52.319315 2962 raft_consensus.cc:2243] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:52.319630 2962 raft_consensus.cc:2272] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:52.320338 2962 ts_tablet_manager.cc:1916] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:52.281204 982 leader_election.cc:343] T 8e09b172980c4e40b1d3f695d8e54c37 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.321056 982 leader_election.cc:304] T 8e09b172980c4e40b1d3f695d8e54c37 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.321424 3423 raft_consensus.cc:2749] T 8e09b172980c4e40b1d3f695d8e54c37 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:52.322086 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 215f8dc74a71482daa2ca80e15010f51 (table default.Table_b7f9abd9a3b04fd4a6ef3b477330ea9e [id=9b089b452ac940fe929dc03bddca83e0]) successfully deleted
I20251211 20:52:52.323681 1037 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "116b21a182474ca191c4a89e9474677d" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:52.323848 1037 raft_consensus.cc:3060] T 116b21a182474ca191c4a89e9474677d P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:52.326365 1037 raft_consensus.cc:2468] T 116b21a182474ca191c4a89e9474677d P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:52.330075 3397 raft_consensus.cc:493] T 6029c1fd563d4023a8cf7cfb8ffa0876 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.330214 3397 raft_consensus.cc:515] T 6029c1fd563d4023a8cf7cfb8ffa0876 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.331009 3397 leader_election.cc:290] T 6029c1fd563d4023a8cf7cfb8ffa0876 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:52.337262 1160 tablet_bootstrap.cc:492] T 84b2a67c836844ceb3628cc3b7d8ba10 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:52.337595 1160 ts_tablet_manager.cc:1403] T 84b2a67c836844ceb3628cc3b7d8ba10 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.079s user 0.005s sys 0.002s
W20251211 20:52:52.338899 847 leader_election.cc:343] T 6029c1fd563d4023a8cf7cfb8ffa0876 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 6029c1fd563d4023a8cf7cfb8ffa0876
I20251211 20:52:52.340106 1160 raft_consensus.cc:359] T 84b2a67c836844ceb3628cc3b7d8ba10 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.341610 1160 raft_consensus.cc:385] T 84b2a67c836844ceb3628cc3b7d8ba10 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.343254 1160 raft_consensus.cc:740] T 84b2a67c836844ceb3628cc3b7d8ba10 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.345526 1160 consensus_queue.cc:260] T 84b2a67c836844ceb3628cc3b7d8ba10 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.352869 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b52377c8ffd741a598e5328a00d876fe (table default.Table_686a26915b42421380037144900dd97c [id=8a1b0c9d641b4a3c83d1ca48576d110d]). Expected version 1 got 0
I20251211 20:52:52.355329 3294 log.cc:1199] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/e9ead34065e240eeb1f946319537d32a
I20251211 20:52:52.355907 3294 ts_tablet_manager.cc:1950] T e9ead34065e240eeb1f946319537d32a P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:52.356976 1166 tablet_bootstrap.cc:492] T ab4e7a9062134e489d6f9abe6ff26fb4 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:52.357187 1166 ts_tablet_manager.cc:1403] T ab4e7a9062134e489d6f9abe6ff26fb4 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.106s user 0.004s sys 0.006s
I20251211 20:52:52.363217 1096 heartbeater.cc:499] Master 127.0.104.61:40749 was elected leader, sending a full tablet report...
I20251211 20:52:52.366156 1160 ts_tablet_manager.cc:1434] T 84b2a67c836844ceb3628cc3b7d8ba10 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.028s user 0.003s sys 0.001s
I20251211 20:52:52.366504 1160 tablet_bootstrap.cc:492] T 845b5ac4a5984c0cb4ab188925103dc9 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:52.368243 1160 tablet_bootstrap.cc:654] T 845b5ac4a5984c0cb4ab188925103dc9 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.362612 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 0302cc8613934c43a0650f081cf98797 (table default.Table_bf827b216f1b43c196923dc1828ff0d0 [id=ed371e073250439fa44330c7101e9387]). Expected version 1 got 0
I20251211 20:52:52.368643 3423 raft_consensus.cc:493] T cc43ee61e78249bfa00b11f3a1fa21a3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.368780 3423 raft_consensus.cc:515] T cc43ee61e78249bfa00b11f3a1fa21a3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.369408 3423 leader_election.cc:290] T cc43ee61e78249bfa00b11f3a1fa21a3 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:52.370042 901 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cc43ee61e78249bfa00b11f3a1fa21a3" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:52.370582 983 leader_election.cc:343] T cc43ee61e78249bfa00b11f3a1fa21a3 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.372895 3423 raft_consensus.cc:493] T 664c6a3afe5b4a4caf87dede87f199ba P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.373077 3423 raft_consensus.cc:515] T 664c6a3afe5b4a4caf87dede87f199ba P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.373620 3423 leader_election.cc:290] T 664c6a3afe5b4a4caf87dede87f199ba P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:52.372951 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 1f4eb1a174c648c897ae2b7612e44585 (table default.Table_e70873b71ba24d7eae548e2acfd243b3 [id=87adcc552a704d56bd2fe812fb3b791f]). Expected version 1 got 0
I20251211 20:52:52.374372 901 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "664c6a3afe5b4a4caf87dede87f199ba" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:52.374899 983 leader_election.cc:343] T 664c6a3afe5b4a4caf87dede87f199ba P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.375327 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet a7afcce522494c9ca1d80e6eea913e6e (table default.Table_cb6a4bac7b694f33a37de2ee78c91ca0 [id=8940f18527c249a9baefc170ed82f45a]). Expected version 1 got 0
I20251211 20:52:52.375900 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cc43ee61e78249bfa00b11f3a1fa21a3" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:52.376650 847 leader_election.cc:343] T 6029c1fd563d4023a8cf7cfb8ffa0876 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 6029c1fd563d4023a8cf7cfb8ffa0876
I20251211 20:52:52.376770 847 leader_election.cc:304] T 6029c1fd563d4023a8cf7cfb8ffa0876 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:52.377151 3397 raft_consensus.cc:2749] T 6029c1fd563d4023a8cf7cfb8ffa0876 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:52.377995 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "664c6a3afe5b4a4caf87dede87f199ba" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:52.378700 3423 raft_consensus.cc:493] T 1c4b2c22bd4c456eb77af45576de466f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.378990 3423 raft_consensus.cc:515] T 1c4b2c22bd4c456eb77af45576de466f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.378389 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 84bb3583857848459e0e865283de623d (table default.Table_92281c5fcfee4355a79215ffedeee588 [id=6e5fe0a78e4b48f98dac028802f1ddf4]). Expected version 4 got 0
I20251211 20:52:52.379647 3423 leader_election.cc:290] T 1c4b2c22bd4c456eb77af45576de466f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:52.380311 901 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1c4b2c22bd4c456eb77af45576de466f" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:52.382575 1160 tablet_bootstrap.cc:492] T 845b5ac4a5984c0cb4ab188925103dc9 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:52.382768 1160 ts_tablet_manager.cc:1403] T 845b5ac4a5984c0cb4ab188925103dc9 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.003s
I20251211 20:52:52.383338 901 raft_consensus.cc:3060] T 98d575ac680d49d0990877666638f013 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:52.380887 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 7a299ee03d0e48aba7c4f71e3e0f381a (table default.Table_6402a801d48e4a53827b1b62b6cdaba1 [id=6876565d3a9c48ce841b830cf75139ff]). Expected version 3 got 0
I20251211 20:52:52.385865 710 leader_election.cc:304] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.386122 3505 raft_consensus.cc:2804] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:52.386229 3505 raft_consensus.cc:697] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:52.386441 3505 consensus_queue.cc:237] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.387279 1160 raft_consensus.cc:359] T 845b5ac4a5984c0cb4ab188925103dc9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.387780 1160 raft_consensus.cc:385] T 845b5ac4a5984c0cb4ab188925103dc9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.387984 1160 raft_consensus.cc:740] T 845b5ac4a5984c0cb4ab188925103dc9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.388326 1160 consensus_queue.cc:260] T 845b5ac4a5984c0cb4ab188925103dc9 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.387933 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 5b3e8ec3faf445ac9b746e65dc4dbfe0 (table default.Table_7fe80ff5483a4b0292091119980f6639 [id=3c3462a0c09748cea0d2ffb544ae2f13]). Expected version 1 got 0
I20251211 20:52:52.389328 1160 ts_tablet_manager.cc:1434] T 845b5ac4a5984c0cb4ab188925103dc9 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20251211 20:52:52.390251 1160 tablet_bootstrap.cc:492] T 94658fcf512a48d0a992a7e2e0f7c176 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:52.391283 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 7ffa06c4a7ee48b0b337a27aff090592 (table default.Table_647731023b044ebb8954f7fdcc3faf97 [id=4b91abd6e9e94a8a941b33e62e23c083]). Expected version 1 got 0
I20251211 20:52:52.393488 1160 tablet_bootstrap.cc:654] T 94658fcf512a48d0a992a7e2e0f7c176 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.393030 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 81fe26e81790448fb436929c03e3c284 (table default.Table_efc22dc69ad0405ebf59fda961e23330 [id=d8a0839df7524fb7b95bfc05fd4ada25]). Expected version 1 got 0
I20251211 20:52:52.394403 3295 ts_tablet_manager.cc:1929] T 1578887b2f5748cd83a6fbdec2f3fc99 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:52.394558 3295 log.cc:1199] T 1578887b2f5748cd83a6fbdec2f3fc99 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/1578887b2f5748cd83a6fbdec2f3fc99
I20251211 20:52:52.395100 3295 ts_tablet_manager.cc:1950] T 1578887b2f5748cd83a6fbdec2f3fc99 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:52.396112 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 139) took 1502 ms. Trace:
I20251211 20:52:52.395815 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b81a4cef27164824b52695039744db15 (table default.Table_defdeab28749411590cf75b5251f604b [id=11352566d1744193a40a8bb605aa3912]). Expected version 1 got 0
I20251211 20:52:52.396199 3295 rpcz_store.cc:276] 1211 20:52:50.894135 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.894182 (+ 47us) service_pool.cc:225] Handling call
1211 20:52:52.128878 (+1234696us) ts_tablet_manager.cc:1151] Deleting tablet 1578887b2f5748cd83a6fbdec2f3fc99
1211 20:52:52.128897 (+ 19us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.217783 (+ 88886us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.394292 (+176509us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.396092 (+ 1800us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1234541}
I20251211 20:52:52.396708 1783 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 1578887b2f5748cd83a6fbdec2f3fc99 (table default.Table_328b7dd76e9e407c83b47a0f51cfcce5 [id=9e7c08f833e14a90b17a9b9f60b702de]) successfully deleted
I20251211 20:52:52.398036 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 7023afec769c4ca3b2242a225004d482 (table default.Table_d5276e3cc46d4331b36a8878680fc55e [id=3811f8235b2f44cca283401302d613d4]). Expected version 1 got 0
I20251211 20:52:52.400192 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 72af55c8045b4f8f8e116a5782794d81 (table default.Table_2cff6fab088c4d939b13b504c133cc12 [id=b6c4b72063b3428593d5650a160dc66b]). Expected version 1 got 0
I20251211 20:52:52.406848 1160 tablet_bootstrap.cc:492] T 94658fcf512a48d0a992a7e2e0f7c176 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:52.407008 1160 ts_tablet_manager.cc:1403] T 94658fcf512a48d0a992a7e2e0f7c176 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.000s
I20251211 20:52:52.410141 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 4cf0cde801bf44449e539d2799b895a7 (table default.Table_907c596bdddc4edca25d14cbacebb23f [id=321235280d1d4fa0bae5cc0333860f85]). Expected version 1 got 0
I20251211 20:52:52.412104 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 7b9f569554fb4836ac2b62542c3ecf7e (table default.Table_1a73d567e8914a83974e3e8f5f58be0b [id=6ef691749df24251a7a7c5919c219754]). Expected version 1 got 0
I20251211 20:52:52.414631 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 3b2c6064c5eb4c0f8979c485f5d85a75 (table default.Table_cd52450ec0cc4892a722eef698ddca35 [id=8867000ef77f4beaadb17dc2df97477f]). Expected version 1 got 0
I20251211 20:52:52.417025 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 172) took 1775 ms. Trace:
I20251211 20:52:52.417132 3294 rpcz_store.cc:276] 1211 20:52:50.641716 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.642185 (+ 469us) service_pool.cc:225] Handling call
1211 20:52:52.101106 (+1458921us) ts_tablet_manager.cc:1151] Deleting tablet e9ead34065e240eeb1f946319537d32a
1211 20:52:52.101116 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.146010 (+ 44894us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.282997 (+136987us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.416996 (+133999us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1458821}
I20251211 20:52:52.417717 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet e9ead34065e240eeb1f946319537d32a (table default.Table_3826c483ba654865be6be42ae333a65b [id=6addaf2424e9450e9a11ee71c17f055d]) successfully deleted
I20251211 20:52:52.417866 3294 tablet_replica.cc:333] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:52.417994 2429 raft_consensus.cc:493] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.417577 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 938a0d8a090f490b816fdf2b8f1a51b8 (table default.Table_5ca4ab25a21d456c95a0ff68c9d73330 [id=9ffcb594267a4b0eb8d792dbe46d585f]). Expected version 1 got 0
W20251211 20:52:52.418556 710 consensus_peers.cc:597] T 7b06cd366de5494780b5ac81abbf0702 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7b06cd366de5494780b5ac81abbf0702. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:52:52.418716 710 consensus_peers.cc:597] T f3855ffb51a74101b26fd28429e65bc2 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:52.419971 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 59acb8474e114bf7ae7f1f043d611459 (table default.Table_c7ed8c0c309c40a3a371ffa130b47cd2 [id=ed1d0670426945aba888b818ad657225]). Expected version 1 got 0
I20251211 20:52:52.421921 1168 tablet_bootstrap.cc:492] T c549c2bfebd64b87a4fc8ce8210e14f6 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:52.422140 1168 ts_tablet_manager.cc:1403] T c549c2bfebd64b87a4fc8ce8210e14f6 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.177s user 0.002s sys 0.026s
I20251211 20:52:52.422978 1168 raft_consensus.cc:359] T c549c2bfebd64b87a4fc8ce8210e14f6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.423220 1168 raft_consensus.cc:385] T c549c2bfebd64b87a4fc8ce8210e14f6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.423343 1168 raft_consensus.cc:740] T c549c2bfebd64b87a4fc8ce8210e14f6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.423617 1168 consensus_queue.cc:260] T c549c2bfebd64b87a4fc8ce8210e14f6 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.424091 1168 ts_tablet_manager.cc:1434] T c549c2bfebd64b87a4fc8ce8210e14f6 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251211 20:52:52.424443 1168 tablet_bootstrap.cc:492] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:52.407851 1160 raft_consensus.cc:359] T 94658fcf512a48d0a992a7e2e0f7c176 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.424968 1160 raft_consensus.cc:385] T 94658fcf512a48d0a992a7e2e0f7c176 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.425055 1160 raft_consensus.cc:740] T 94658fcf512a48d0a992a7e2e0f7c176 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.425267 1160 consensus_queue.cc:260] T 94658fcf512a48d0a992a7e2e0f7c176 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.426506 1160 ts_tablet_manager.cc:1434] T 94658fcf512a48d0a992a7e2e0f7c176 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.019s user 0.002s sys 0.000s
I20251211 20:52:52.426728 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet bbe6e6220eb14e59a821805c438230c9 (table default.Table_16a12ed370b64a38b5e145c4eb024349 [id=b7429ea93123473ab5d795d5f3d5a5f8]). Expected version 1 got 0
W20251211 20:52:52.426956 710 consensus_peers.cc:597] T 215f8dc74a71482daa2ca80e15010f51 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 215f8dc74a71482daa2ca80e15010f51. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:52.427168 2429 raft_consensus.cc:515] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:52.427798 710 consensus_peers.cc:597] T 58e35de0792a464a818170886ca8d8c9 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 58e35de0792a464a818170886ca8d8c9. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:52.428068 2429 leader_election.cc:290] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:52.428469 901 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "683d0f2068fb4a6ebc9c169ad7b455ce" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:52.428833 1160 tablet_bootstrap.cc:492] T 497c10742f01476db885bd70c4d2c491 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:52.428985 711 leader_election.cc:343] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.430402 901 raft_consensus.cc:3060] T 01104975fd8b470aa026578b97ff6b8d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:52.380885 983 leader_election.cc:343] T 1c4b2c22bd4c456eb77af45576de466f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:52.432183 983 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.2:34691, user_credentials={real_user=slave}} blocked reactor thread for 51285.4us
I20251211 20:52:52.394979 3423 raft_consensus.cc:493] T 13ceabb190494e718a52892faaa53986 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.432410 1168 tablet_bootstrap.cc:654] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.432508 3423 raft_consensus.cc:515] T 13ceabb190494e718a52892faaa53986 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.433115 3423 leader_election.cc:290] T 13ceabb190494e718a52892faaa53986 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
W20251211 20:52:52.380367 982 leader_election.cc:343] T cc43ee61e78249bfa00b11f3a1fa21a3 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.433697 3294 raft_consensus.cc:2243] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:52.433785 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13ceabb190494e718a52892faaa53986" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:52.433766 3505 raft_consensus.cc:493] T c8214c97ed1640f5965aa5c4b999e2f1 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.434197 3294 raft_consensus.cc:2272] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:52.434803 3294 ts_tablet_manager.cc:1916] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:52.434262 3505 raft_consensus.cc:515] T c8214c97ed1640f5965aa5c4b999e2f1 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.435997 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8214c97ed1640f5965aa5c4b999e2f1" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:52.436300 1160 tablet_bootstrap.cc:654] T 497c10742f01476db885bd70c4d2c491 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:52.437214 711 leader_election.cc:343] T c8214c97ed1640f5965aa5c4b999e2f1 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.437742 3505 leader_election.cc:290] T c8214c97ed1640f5965aa5c4b999e2f1 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:52.433913 982 leader_election.cc:304] T cc43ee61e78249bfa00b11f3a1fa21a3 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.438562 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet bc4e7362e6d045da81bf39f4135f500e (table default.Table_328b7dd76e9e407c83b47a0f51cfcce5 [id=9e7c08f833e14a90b17a9b9f60b702de]). Expected version 1 got 0
I20251211 20:52:52.439009 3423 raft_consensus.cc:2749] T cc43ee61e78249bfa00b11f3a1fa21a3 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:52.439337 982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.1:42177, user_credentials={real_user=slave}} blocked reactor thread for 58955.7us
W20251211 20:52:52.439713 982 leader_election.cc:343] T 664c6a3afe5b4a4caf87dede87f199ba P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.439841 982 leader_election.cc:304] T 664c6a3afe5b4a4caf87dede87f199ba P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.439613 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet dfd194cf00f549c8937cb6ec40987240 (table default.Table_25e38d0576684687890bae04bd81d4f4 [id=d919c6132b3c466e8a1404f981b003b8]). Expected version 2 got 0
I20251211 20:52:52.440093 3423 raft_consensus.cc:2749] T 664c6a3afe5b4a4caf87dede87f199ba P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:52.440928 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1c4b2c22bd4c456eb77af45576de466f" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:52.441807 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13ceabb190494e718a52892faaa53986" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:52.442801 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet f26ba8f4bd3a4a1f8fa2ba6e80d543f8 (table default.Table_71e9c158dc22435a8fbb02aecca8919d [id=b966d0c0b49d4023894950c5dc06f678]). Expected version 2 got 0
I20251211 20:52:52.427616 3448 raft_consensus.cc:493] T 1b8acfdf1a26427aa9863280dc39b818 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.443398 3448 raft_consensus.cc:515] T 1b8acfdf1a26427aa9863280dc39b818 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.443992 3448 leader_election.cc:290] T 1b8acfdf1a26427aa9863280dc39b818 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
W20251211 20:52:52.434253 983 leader_election.cc:343] T 13ceabb190494e718a52892faaa53986 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.425642 1095 heartbeater.cc:344] Connected to a master server at 127.0.104.60:42087
I20251211 20:52:52.444734 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1b8acfdf1a26427aa9863280dc39b818" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:52.445248 983 leader_election.cc:343] T 1b8acfdf1a26427aa9863280dc39b818 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.447194 906 raft_consensus.cc:3060] T 845b5ac4a5984c0cb4ab188925103dc9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:52.425618 1166 raft_consensus.cc:359] T ab4e7a9062134e489d6f9abe6ff26fb4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.449640 1166 raft_consensus.cc:385] T ab4e7a9062134e489d6f9abe6ff26fb4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.450006 1166 raft_consensus.cc:740] T ab4e7a9062134e489d6f9abe6ff26fb4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.450337 1166 consensus_queue.cc:260] T ab4e7a9062134e489d6f9abe6ff26fb4 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.451439 1166 ts_tablet_manager.cc:1434] T ab4e7a9062134e489d6f9abe6ff26fb4 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.094s user 0.001s sys 0.001s
I20251211 20:52:52.452752 1166 tablet_bootstrap.cc:492] T 83067b139f504bc191a515feed0ec771 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:52.454698 1166 tablet_bootstrap.cc:654] T 83067b139f504bc191a515feed0ec771 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.455912 1168 tablet_bootstrap.cc:492] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:52.455926 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 140) took 1561 ms. Trace:
I20251211 20:52:52.456001 3295 rpcz_store.cc:276] 1211 20:52:50.894734 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.894775 (+ 41us) service_pool.cc:225] Handling call
1211 20:52:52.455762 (+1560987us) ts_tablet_manager.cc:1151] Deleting tablet ba120d28abc54553ae15532de90db719
1211 20:52:52.455772 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.455904 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1560833}
W20251211 20:52:52.456583 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet ba120d28abc54553ae15532de90db719 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba120d28abc54553ae15532de90db719 already in progress: creating tablet
I20251211 20:52:52.456995 1168 ts_tablet_manager.cc:1403] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.033s user 0.004s sys 0.000s
I20251211 20:52:52.457106 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 141) took 1561 ms. Trace:
I20251211 20:52:52.457183 3295 rpcz_store.cc:276] 1211 20:52:50.895313 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.895356 (+ 43us) service_pool.cc:225] Handling call
1211 20:52:52.456978 (+1561622us) ts_tablet_manager.cc:1151] Deleting tablet 4585e05d06ef42cfb06c4ceb7524fd71
1211 20:52:52.456989 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.457088 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1561471}
W20251211 20:52:52.457608 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 4585e05d06ef42cfb06c4ceb7524fd71 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4585e05d06ef42cfb06c4ceb7524fd71 already in progress: creating tablet
I20251211 20:52:52.458019 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 142) took 1562 ms. Trace:
I20251211 20:52:52.458127 3295 rpcz_store.cc:276] 1211 20:52:50.896035 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.896083 (+ 48us) service_pool.cc:225] Handling call
1211 20:52:52.457900 (+1561817us) ts_tablet_manager.cc:1151] Deleting tablet 3dde7ed495e64909bd9784dc5eefb04c
1211 20:52:52.457910 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.458002 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1561665}
W20251211 20:52:52.458565 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 3dde7ed495e64909bd9784dc5eefb04c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3dde7ed495e64909bd9784dc5eefb04c already in progress: creating tablet
I20251211 20:52:52.458928 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 143) took 1562 ms. Trace:
I20251211 20:52:52.459031 3295 rpcz_store.cc:276] 1211 20:52:50.896716 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.896765 (+ 49us) service_pool.cc:225] Handling call
1211 20:52:52.458812 (+1562047us) ts_tablet_manager.cc:1151] Deleting tablet e709051411354522bdb327c34dbf533a
1211 20:52:52.458821 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.458911 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1561858}
W20251211 20:52:52.459452 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet e709051411354522bdb327c34dbf533a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e709051411354522bdb327c34dbf533a already in progress: creating tablet
I20251211 20:52:52.459970 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 144) took 1562 ms. Trace:
I20251211 20:52:52.460071 3295 rpcz_store.cc:276] 1211 20:52:50.897502 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.897546 (+ 44us) service_pool.cc:225] Handling call
1211 20:52:52.459853 (+1562307us) ts_tablet_manager.cc:1151] Deleting tablet 6cfa15533fad4aec84f8d9038227faf6
1211 20:52:52.459861 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.459953 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1562168}
W20251211 20:52:52.460474 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 6cfa15533fad4aec84f8d9038227faf6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6cfa15533fad4aec84f8d9038227faf6 already in progress: creating tablet
I20251211 20:52:52.461064 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 1c4b2c22bd4c456eb77af45576de466f (table default.Table_e3a99b043e4f420a93ebfee6aa87f2ec [id=3e7d6cefcb7d412191c6540e9ad77c2d]). Expected version 2 got 0
I20251211 20:52:52.465605 3294 ts_tablet_manager.cc:1929] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:52.466449 3294 log.cc:1199] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/3621735fcb294e3a8c0180ba9ef86442
I20251211 20:52:52.468431 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet e3f711f02af843adb4f5c3c750924e33 (table default.Table_34bd5155aa3b4ef7818022ba9346c074 [id=83a61d23e80946439a14096ab3ab1bd9]). Expected version 2 got 0
I20251211 20:52:52.468868 3294 ts_tablet_manager.cc:1950] T 3621735fcb294e3a8c0180ba9ef86442 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:52.466698 1168 raft_consensus.cc:359] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.473229 1168 raft_consensus.cc:385] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.473591 1168 raft_consensus.cc:740] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.474766 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 173) took 1832 ms. Trace:
I20251211 20:52:52.474704 1168 consensus_queue.cc:260] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.475330 3294 rpcz_store.cc:276] 1211 20:52:50.641802 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.642313 (+ 511us) service_pool.cc:225] Handling call
1211 20:52:52.417813 (+1775500us) ts_tablet_manager.cc:1151] Deleting tablet 3621735fcb294e3a8c0180ba9ef86442
1211 20:52:52.417824 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.445483 (+ 27659us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.464924 (+ 19441us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.474572 (+ 9648us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":15585,"tablet-delete.queue_time_us":1775391}
I20251211 20:52:52.477523 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 3621735fcb294e3a8c0180ba9ef86442 (table default.Table_01868fec1884449da230c4236ab2d506 [id=d303c5c8fd81400ab473845a942ec140]) successfully deleted
I20251211 20:52:52.478582 1168 ts_tablet_manager.cc:1434] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.021s user 0.003s sys 0.000s
I20251211 20:52:52.478857 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 174) took 1836 ms. Trace:
I20251211 20:52:52.479526 3294 rpcz_store.cc:276] 1211 20:52:50.642527 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.642632 (+ 105us) service_pool.cc:225] Handling call
1211 20:52:52.477831 (+1835199us) ts_tablet_manager.cc:1151] Deleting tablet 84ed434868aa437a83455b1605deb26c
1211 20:52:52.478025 (+ 194us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.478666 (+ 641us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1834883}
I20251211 20:52:52.480098 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 7ca98b43863a47c091baae0a6ce65630 (table default.Table_f0a70914acc94231a4e84155c3e4d13f [id=56c070ad80624c698ebe2e01b4929b6e]). Expected version 1 got 0
W20251211 20:52:52.481441 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 84ed434868aa437a83455b1605deb26c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84ed434868aa437a83455b1605deb26c already in progress: creating tablet
I20251211 20:52:52.481334 1168 tablet_bootstrap.cc:492] T 59e955aa07c14fbbb347b88eca59a74e P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:52.483098 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 175) took 1839 ms. Trace:
I20251211 20:52:52.483769 3294 rpcz_store.cc:276] 1211 20:52:50.642932 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.642974 (+ 42us) service_pool.cc:225] Handling call
1211 20:52:52.482229 (+1839255us) ts_tablet_manager.cc:1151] Deleting tablet eb9da081cb9d43fd9a35c4bc51ea133e
1211 20:52:52.482412 (+ 183us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.482839 (+ 427us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1838946}
I20251211 20:52:52.485656 901 raft_consensus.cc:3060] T a295a24e81f14e5fa126a4a58ffd08cf P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:52.485807 1780 catalog_manager.cc:4952] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet eb9da081cb9d43fd9a35c4bc51ea133e because the tablet was not found. No further retry: Not found: Tablet not found: eb9da081cb9d43fd9a35c4bc51ea133e
I20251211 20:52:52.487442 3294 tablet_replica.cc:333] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:52.488817 3294 raft_consensus.cc:2243] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:52.491789 3294 raft_consensus.cc:2272] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:52.492837 1160 tablet_bootstrap.cc:492] T 497c10742f01476db885bd70c4d2c491 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:52.492975 1160 ts_tablet_manager.cc:1403] T 497c10742f01476db885bd70c4d2c491 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.066s user 0.008s sys 0.011s
I20251211 20:52:52.493716 1160 raft_consensus.cc:359] T 497c10742f01476db885bd70c4d2c491 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.493881 1160 raft_consensus.cc:385] T 497c10742f01476db885bd70c4d2c491 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.493930 1160 raft_consensus.cc:740] T 497c10742f01476db885bd70c4d2c491 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.494135 1160 consensus_queue.cc:260] T 497c10742f01476db885bd70c4d2c491 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.494612 1160 ts_tablet_manager.cc:1434] T 497c10742f01476db885bd70c4d2c491 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251211 20:52:52.493932 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 969a8587019c4202b6e705d126f83d44 (table default.Table_6460313002684bba93b30ad8631f0f49 [id=22347b15537d451d9927a17e8e5a202a]). Expected version 1 got 0
I20251211 20:52:52.497184 3294 ts_tablet_manager.cc:1916] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:52.498993 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet fb29f983c4524a49bdc5c0c1144ad708 (table default.Table_7809ac85fdfe4c1d9c58afb54edecb27 [id=c4b0bfff99f64888beb70d9154d2f156]). Expected version 1 got 0
I20251211 20:52:52.499943 1168 tablet_bootstrap.cc:654] T 59e955aa07c14fbbb347b88eca59a74e P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.502274 1160 tablet_bootstrap.cc:492] T 1e13965b6361488eaeb7dd13b467ca65 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:52.502655 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet ab4e7a9062134e489d6f9abe6ff26fb4 (table default.Table_4cbc8841b276406cb4434629653a5ce3 [id=b958532650374d609199cab9c07bb64b]). Expected version 2 got 0
I20251211 20:52:52.504040 1160 tablet_bootstrap.cc:654] T 1e13965b6361488eaeb7dd13b467ca65 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.505813 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet d6923cfd826d495291d9ac8e0b3b44c6 (table default.Table_68ccb2052d824ccf92f7162d2961fcb1 [id=9ac9591d4f04482d926710fdf6e797c6]). Expected version 1 got 0
I20251211 20:52:52.506937 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "683d0f2068fb4a6ebc9c169ad7b455ce" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:52.507210 1045 raft_consensus.cc:2468] T 683d0f2068fb4a6ebc9c169ad7b455ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:52.507751 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b2974f16034c44cf85ff85b5095cf6a0 (table default.Table_031de0f4cc7744bc9e320261f8b46696 [id=be36dabaa6d44325b4855b9f13bda847]). Expected version 1 got 0
I20251211 20:52:52.510324 1160 tablet_bootstrap.cc:492] T 1e13965b6361488eaeb7dd13b467ca65 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:52.510490 1160 ts_tablet_manager.cc:1403] T 1e13965b6361488eaeb7dd13b467ca65 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20251211 20:52:52.511312 1160 raft_consensus.cc:359] T 1e13965b6361488eaeb7dd13b467ca65 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.511503 1160 raft_consensus.cc:385] T 1e13965b6361488eaeb7dd13b467ca65 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.511622 1160 raft_consensus.cc:740] T 1e13965b6361488eaeb7dd13b467ca65 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.512101 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b6cc819332e1445c96ce3ffac4861454 (table default.Table_db4c0f706e0643b39d39ffa84352826d [id=4d459c42bf584068ac3851227d80331a]). Expected version 1 got 0
I20251211 20:52:52.513752 1160 consensus_queue.cc:260] T 1e13965b6361488eaeb7dd13b467ca65 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.514312 1160 ts_tablet_manager.cc:1434] T 1e13965b6361488eaeb7dd13b467ca65 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20251211 20:52:52.514379 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1b8acfdf1a26427aa9863280dc39b818" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:52.514621 1160 tablet_bootstrap.cc:492] T 79cc81e91e1846038621af7a0a9f7a2e P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:52.514658 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet f2d6c9781b4d4c37bc08703347963d5d (table default.Table_2aa7ff9186a04249abf8cd971cc0cb17 [id=b50ab5b032ba4c98938324460512468f]). Expected version 1 got 0
I20251211 20:52:52.516402 1160 tablet_bootstrap.cc:654] T 79cc81e91e1846038621af7a0a9f7a2e P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.516304 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 16d2ff8925e342729a71c93db378d031 (table default.Table_34e6ed4f13da437ea50d349b92e83bd6 [id=4a74b9a60cbb4198b55cee630b2f4bef]). Expected version 2 got 0
I20251211 20:52:52.517918 710 leader_election.cc:304] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.518293 3457 raft_consensus.cc:2804] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:52.518375 3457 raft_consensus.cc:493] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:52.518435 3457 raft_consensus.cc:3060] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:52.521229 3457 raft_consensus.cc:515] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.521931 3457 leader_election.cc:290] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:52.522890 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "683d0f2068fb4a6ebc9c169ad7b455ce" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:52.523501 711 leader_election.cc:343] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:52.523198 710 leader_election.cc:343] T c8214c97ed1640f5965aa5c4b999e2f1 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: c8214c97ed1640f5965aa5c4b999e2f1
I20251211 20:52:52.524603 710 leader_election.cc:304] T c8214c97ed1640f5965aa5c4b999e2f1 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919; no voters: e715a811a82446ce8d5bb4aafd5b81c7, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.525076 3155 raft_consensus.cc:2749] T c8214c97ed1640f5965aa5c4b999e2f1 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:52.525844 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 145) took 1627 ms. Trace:
I20251211 20:52:52.526002 3295 rpcz_store.cc:276] 1211 20:52:50.898187 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.898229 (+ 42us) service_pool.cc:225] Handling call
1211 20:52:52.460718 (+1562489us) ts_tablet_manager.cc:1151] Deleting tablet 48820386e41b47129a13bdd5768f021b
1211 20:52:52.460727 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.525777 (+ 65050us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1562285}
W20251211 20:52:52.526579 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 48820386e41b47129a13bdd5768f021b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 48820386e41b47129a13bdd5768f021b already in progress: creating tablet
I20251211 20:52:52.526993 3295 tablet_replica.cc:333] T bd2bf1113dfe4b67b263b56f3d3db07d P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:52.527247 3295 raft_consensus.cc:2243] T bd2bf1113dfe4b67b263b56f3d3db07d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:52.527496 3295 raft_consensus.cc:2272] T bd2bf1113dfe4b67b263b56f3d3db07d P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:52.528373 1168 tablet_bootstrap.cc:492] T 59e955aa07c14fbbb347b88eca59a74e P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:52.528589 1168 ts_tablet_manager.cc:1403] T 59e955aa07c14fbbb347b88eca59a74e P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.048s user 0.001s sys 0.004s
I20251211 20:52:52.529393 1168 raft_consensus.cc:359] T 59e955aa07c14fbbb347b88eca59a74e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.529637 1168 raft_consensus.cc:385] T 59e955aa07c14fbbb347b88eca59a74e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.529770 1168 raft_consensus.cc:740] T 59e955aa07c14fbbb347b88eca59a74e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.530090 1168 consensus_queue.cc:260] T 59e955aa07c14fbbb347b88eca59a74e P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.530704 1168 ts_tablet_manager.cc:1434] T 59e955aa07c14fbbb347b88eca59a74e P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:52.531014 1168 tablet_bootstrap.cc:492] T 66cdc6f297b24cfba3a3a0ae9f45d12d P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:52.531798 906 raft_consensus.cc:3060] T 1e13965b6361488eaeb7dd13b467ca65 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:52.534705 902 raft_consensus.cc:3060] T 84b2a67c836844ceb3628cc3b7d8ba10 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:52.492316 2962 ts_tablet_manager.cc:1929] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:52.540294 2962 log.cc:1199] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/f3855ffb51a74101b26fd28429e65bc2
I20251211 20:52:52.540762 2962 ts_tablet_manager.cc:1950] T f3855ffb51a74101b26fd28429e65bc2 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:52.547542 3000 raft_consensus.cc:493] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.547788 3000 raft_consensus.cc:515] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.548806 3000 leader_election.cc:290] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:52.548789 910 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4388893262b54ad8be1bdf82516031a5" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:52.549340 711 leader_election.cc:343] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.549753 3295 ts_tablet_manager.cc:1916] T bd2bf1113dfe4b67b263b56f3d3db07d P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:52.552623 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 4c0455a5b4f44412803475d1c4852b48 (table default.Table_e00092673b0e4f09b8d7eec63e1ee034 [id=18497e06d7254441a4655ce1cdc5a9bf]). Expected version 1 got 0
I20251211 20:52:52.553092 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 497c10742f01476db885bd70c4d2c491 (table default.Table_18b21232dfa74378b27defe672bb2574 [id=602d7eee703948d99a0edcb7cac0651b]). Expected version 1 got 0
W20251211 20:52:52.555056 1295 leader_election.cc:336] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9230f94a6a1a49cead51aeea9f069087 (127.0.104.60:42087): Timed out: RequestConsensusVote RPC to 127.0.104.60:42087 timed out after 0.635s (SENT)
I20251211 20:52:52.555209 1295 leader_election.cc:304] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1830b0c58cfe495989f3d911dda0409a; no voters: 11f5f94732f049d0be4c97cebfffcb05, 9230f94a6a1a49cead51aeea9f069087
I20251211 20:52:52.557365 3294 ts_tablet_manager.cc:1929] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:52.558512 1168 tablet_bootstrap.cc:654] T 66cdc6f297b24cfba3a3a0ae9f45d12d P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.558569 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet f94b656e3c1844509fdae36e33b263e8 (table default.Table_ed03ad475bed4804a65abe20efef0625 [id=e23f4305669a474aafa18928f130de3a]). Expected version 2 got 0
I20251211 20:52:52.565361 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet aab26df748724fe5b2bb7fad9e3bfc0c (table default.Table_22bed4b4e9e24a90818b19c6f5c91e1f [id=df6255d66f334ef2b4406f4b845cf980]). Expected version 1 got 0
I20251211 20:52:52.565589 1166 tablet_bootstrap.cc:492] T 83067b139f504bc191a515feed0ec771 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:52.565780 1166 ts_tablet_manager.cc:1403] T 83067b139f504bc191a515feed0ec771 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.113s user 0.002s sys 0.002s
I20251211 20:52:52.566624 1166 raft_consensus.cc:359] T 83067b139f504bc191a515feed0ec771 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.567027 1166 raft_consensus.cc:385] T 83067b139f504bc191a515feed0ec771 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.567269 1166 raft_consensus.cc:740] T 83067b139f504bc191a515feed0ec771 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.559151 3294 log.cc:1199] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/3ecf0ea8ce19486c81d1d1a91d57a7ce
I20251211 20:52:52.568094 1166 consensus_queue.cc:260] T 83067b139f504bc191a515feed0ec771 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.568125 3294 ts_tablet_manager.cc:1950] T 3ecf0ea8ce19486c81d1d1a91d57a7ce P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:52.568689 3521 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20251211 20:52:52.569456 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 03184dcc56dd4e92adc7d0ad22aa8d8b (table default.Table_68261841754847ad8d22e86c29886144 [id=697f23a617f64762bf406b386cde7473]). Expected version 1 got 0
I20251211 20:52:52.569828 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 176) took 1926 ms. Trace:
I20251211 20:52:52.570185 3294 rpcz_store.cc:276] 1211 20:52:50.643239 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.643278 (+ 39us) service_pool.cc:225] Handling call
1211 20:52:52.486511 (+1843233us) ts_tablet_manager.cc:1151] Deleting tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce
1211 20:52:52.486523 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.519645 (+ 33122us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.556174 (+ 36529us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.569804 (+ 13630us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1842945}
I20251211 20:52:52.570794 3294 tablet_replica.cc:333] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:52.570837 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 3ecf0ea8ce19486c81d1d1a91d57a7ce (table default.Table_021d4fb1667a475d87424f23aa0d5e28 [id=6bbfcc2561f349519a69ae1f56016cc7]) successfully deleted
I20251211 20:52:52.570719 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 5e47af616e1b4d2db3780c1d67cdcb47 (table default.Table_296bdf30a11541c389afe3df0b0f667a [id=052edcbde60c41449d468c8a449fc3f1]). Expected version 1 got 0
I20251211 20:52:52.571152 3294 raft_consensus.cc:2243] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:52.572440 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 94658fcf512a48d0a992a7e2e0f7c176 (table default.Table_c9d86e82aebb4ef0bc83789720352e38 [id=97377a8ef56c4cdebe95d15955207404]). Expected version 1 got 0
I20251211 20:52:52.573037 3294 raft_consensus.cc:2272] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:52.575636 1166 ts_tablet_manager.cc:1434] T 83067b139f504bc191a515feed0ec771 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20251211 20:52:52.577651 3294 ts_tablet_manager.cc:1916] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:52.578653 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 156) took 3469 ms. Trace:
I20251211 20:52:52.578298 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet aa50ddb2789b4be3bfa9aedef9287109 (table default.Table_44f971096d1e41f480e594985ffd5c17 [id=d4e1349374a346a5b8be180f9bdc329a]). Expected version 1 got 0
I20251211 20:52:52.578790 2962 rpcz_store.cc:276] 1211 20:52:49.108682 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.111809 (+ 3127us) service_pool.cc:225] Handling call
1211 20:52:52.318945 (+3207136us) ts_tablet_manager.cc:1151] Deleting tablet f3855ffb51a74101b26fd28429e65bc2
1211 20:52:52.318968 (+ 23us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.424432 (+105464us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.492191 (+ 67759us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.578505 (+ 86314us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3198897}
I20251211 20:52:52.579174 2962 tablet_replica.cc:333] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:52.579397 2962 raft_consensus.cc:2243] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:52.579742 2962 raft_consensus.cc:2272] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:52.580063 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 8dc4e730951144d78b77280f6cce6d0e (table default.Table_890c9027415a44ccaf95c0adede51992 [id=7bc8a2b91fe54b91b6594b371fea7bd3]). Expected version 2 got 0
I20251211 20:52:52.580420 2962 ts_tablet_manager.cc:1916] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:52.581379 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 13269a20421240d1a98215129443248b (table default.Table_d887c953026046dc883ede774a58fd8e [id=1d8a79f716a0447cb0393f4759178598]). Expected version 1 got 0
I20251211 20:52:52.582634 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 51d12723f5194afb97d2c05fcbaf3776 (table default.Table_823505200e3142dbbe8cd56ac78daf55 [id=ec0bb10933fa4529bef3580bee6409e7]). Expected version 2 got 0
I20251211 20:52:52.584508 901 raft_consensus.cc:3060] T 94658fcf512a48d0a992a7e2e0f7c176 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:52.584328 1168 tablet_bootstrap.cc:492] T 66cdc6f297b24cfba3a3a0ae9f45d12d P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:52.585255 1168 ts_tablet_manager.cc:1403] T 66cdc6f297b24cfba3a3a0ae9f45d12d P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.054s user 0.003s sys 0.001s
I20251211 20:52:52.586045 1166 tablet_bootstrap.cc:492] T dfd194cf00f549c8937cb6ec40987240 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:52.586635 1168 raft_consensus.cc:359] T 66cdc6f297b24cfba3a3a0ae9f45d12d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.586812 1168 raft_consensus.cc:385] T 66cdc6f297b24cfba3a3a0ae9f45d12d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.586858 1168 raft_consensus.cc:740] T 66cdc6f297b24cfba3a3a0ae9f45d12d P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.587164 960 heartbeater.cc:344] Connected to a master server at 127.0.104.60:42087
W20251211 20:52:52.574975 982 leader_election.cc:343] T 1c4b2c22bd4c456eb77af45576de466f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.588111 982 leader_election.cc:304] T 1c4b2c22bd4c456eb77af45576de466f P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.587349 1168 consensus_queue.cc:260] T 66cdc6f297b24cfba3a3a0ae9f45d12d P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.588441 3448 raft_consensus.cc:2749] T 1c4b2c22bd4c456eb77af45576de466f P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:52.589068 1168 ts_tablet_manager.cc:1434] T 66cdc6f297b24cfba3a3a0ae9f45d12d P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
W20251211 20:52:52.589071 982 leader_election.cc:343] T 13ceabb190494e718a52892faaa53986 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.589197 982 leader_election.cc:304] T 13ceabb190494e718a52892faaa53986 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.589462 3448 raft_consensus.cc:2749] T 13ceabb190494e718a52892faaa53986 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:52.590075 1168 tablet_bootstrap.cc:492] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:52.594748 1168 tablet_bootstrap.cc:654] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.596241 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 46a2a070cf6a42fe8bc54a42eb802fbc (table default.Table_4e1f03b9d87b4fe59dbcb3a947ee299d [id=8e66ea58499b40e59294a135324aa527]). Expected version 1 got 0
I20251211 20:52:52.602844 3295 ts_tablet_manager.cc:1929] T bd2bf1113dfe4b67b263b56f3d3db07d P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:52.602985 3295 log.cc:1199] T bd2bf1113dfe4b67b263b56f3d3db07d P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/bd2bf1113dfe4b67b263b56f3d3db07d
I20251211 20:52:52.603355 3295 ts_tablet_manager.cc:1950] T bd2bf1113dfe4b67b263b56f3d3db07d P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:52.605471 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 5773c13034714a008c8d7dbe777b4828 (table default.Table_66eb1b021eed464f82f9cd223c88297b [id=94ff09564cd649c7a2d0e94b0f3c7b3a]). Expected version 1 got 0
W20251211 20:52:52.607784 982 leader_election.cc:343] T 1b8acfdf1a26427aa9863280dc39b818 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.607957 982 leader_election.cc:304] T 1b8acfdf1a26427aa9863280dc39b818 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.608206 3448 raft_consensus.cc:2749] T 1b8acfdf1a26427aa9863280dc39b818 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:52.609572 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 146) took 1671 ms. Trace:
I20251211 20:52:52.609709 3295 rpcz_store.cc:276] 1211 20:52:50.938416 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.938473 (+ 57us) service_pool.cc:225] Handling call
1211 20:52:52.526888 (+1588415us) ts_tablet_manager.cc:1151] Deleting tablet bd2bf1113dfe4b67b263b56f3d3db07d
1211 20:52:52.526908 (+ 20us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.582953 (+ 56045us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.602687 (+ 19734us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.609530 (+ 6843us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1588187}
I20251211 20:52:52.609876 3448 raft_consensus.cc:493] T fcf5afcfd7f142d29ff91272913e5fcb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.608745 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 18f125dd9efc401181f0644dad7aeffe (table default.Table_fa6592c2e2624d81ad21abb13ed36045 [id=9b48fc2b39db4fd5a4e5da19e34427dc]). Expected version 1 got 0
I20251211 20:52:52.610021 3448 raft_consensus.cc:515] T fcf5afcfd7f142d29ff91272913e5fcb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.610162 1783 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet bd2bf1113dfe4b67b263b56f3d3db07d (table default.Table_ccd52fe248db4491820429e77b22f465 [id=a3f079fc1d054841b369c45cdae7bdfb]) successfully deleted
I20251211 20:52:52.610477 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 147) took 1614 ms. Trace:
I20251211 20:52:52.610627 3448 leader_election.cc:290] T fcf5afcfd7f142d29ff91272913e5fcb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:52.610847 3295 rpcz_store.cc:276] 1211 20:52:50.996309 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.996379 (+ 70us) service_pool.cc:225] Handling call
1211 20:52:52.610335 (+1613956us) ts_tablet_manager.cc:1151] Deleting tablet 5ddf7c1972564ef88e3c0a0bd1b430e0
1211 20:52:52.610345 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.610444 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1613732}
W20251211 20:52:52.611265 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 5ddf7c1972564ef88e3c0a0bd1b430e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5ddf7c1972564ef88e3c0a0bd1b430e0 already in progress: creating tablet
I20251211 20:52:52.611637 3295 tablet_replica.cc:333] T 18b977f463d5420186c007b38110e07e P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:52.611886 3295 raft_consensus.cc:2243] T 18b977f463d5420186c007b38110e07e P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:52.612066 3295 raft_consensus.cc:2272] T 18b977f463d5420186c007b38110e07e P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:52.612602 3295 ts_tablet_manager.cc:1916] T 18b977f463d5420186c007b38110e07e P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:52.613792 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 78f77d8916d84edabbe95ce17d8df2c1 (table default.Table_e2af0e2fbd3d46d595136c7a7bf24e4c [id=ddc12ae6d8ed4587b5ef4da3ef81a055]). Expected version 1 got 0
I20251211 20:52:52.616415 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet ff4d5ec1f87247e896f9c099c532265a (table default.Table_3ac4746e17264f5a93890f3600505781 [id=475e0a1ec0344d10addbcfa42fe05b68]). Expected version 1 got 0
I20251211 20:52:52.617903 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fcf5afcfd7f142d29ff91272913e5fcb" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:52.618745 983 leader_election.cc:343] T fcf5afcfd7f142d29ff91272913e5fcb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.621002 3295 ts_tablet_manager.cc:1929] T 18b977f463d5420186c007b38110e07e P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:52.624270 3295 log.cc:1199] T 18b977f463d5420186c007b38110e07e P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/18b977f463d5420186c007b38110e07e
I20251211 20:52:52.624760 3295 ts_tablet_manager.cc:1950] T 18b977f463d5420186c007b38110e07e P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:52.625339 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 148) took 1628 ms. Trace:
I20251211 20:52:52.625535 3295 rpcz_store.cc:276] 1211 20:52:50.997283 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.997340 (+ 57us) service_pool.cc:225] Handling call
1211 20:52:52.611524 (+1614184us) ts_tablet_manager.cc:1151] Deleting tablet 18b977f463d5420186c007b38110e07e
1211 20:52:52.611535 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.615416 (+ 3881us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.620865 (+ 5449us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.625259 (+ 4394us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1613982}
I20251211 20:52:52.626235 1783 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 18b977f463d5420186c007b38110e07e (table default.Table_37520ef8d4194d8aa3eadef57196a291 [id=a7e8389b5a9a4b01b7a6c327fd2022b9]) successfully deleted
I20251211 20:52:52.627168 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 149) took 1628 ms. Trace:
I20251211 20:52:52.627410 3295 rpcz_store.cc:276] 1211 20:52:50.998152 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.998200 (+ 48us) service_pool.cc:225] Handling call
1211 20:52:52.626908 (+1628708us) ts_tablet_manager.cc:1151] Deleting tablet 41627cb27be8443fa37e514f9b3825a6
1211 20:52:52.626944 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.627086 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1628526}
W20251211 20:52:52.628113 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 41627cb27be8443fa37e514f9b3825a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 41627cb27be8443fa37e514f9b3825a6 already in progress: creating tablet
I20251211 20:52:52.628700 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 150) took 1629 ms. Trace:
I20251211 20:52:52.628865 3295 rpcz_store.cc:276] 1211 20:52:50.998970 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.999079 (+ 109us) service_pool.cc:225] Handling call
1211 20:52:52.628456 (+1629377us) ts_tablet_manager.cc:1151] Deleting tablet a4ccf9a4ecbc4ca6978a189cf3a15893
1211 20:52:52.628490 (+ 34us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.628591 (+ 101us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1629181}
W20251211 20:52:52.629415 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet a4ccf9a4ecbc4ca6978a189cf3a15893 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a4ccf9a4ecbc4ca6978a189cf3a15893 already in progress: creating tablet
I20251211 20:52:52.630040 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 151) took 1630 ms. Trace:
I20251211 20:52:52.632637 3294 ts_tablet_manager.cc:1929] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:52.632758 3294 log.cc:1199] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/3562b5effa124378beac7ccf0d27ee7e
I20251211 20:52:52.633116 3294 ts_tablet_manager.cc:1950] T 3562b5effa124378beac7ccf0d27ee7e P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:52.633363 1166 tablet_bootstrap.cc:654] T dfd194cf00f549c8937cb6ec40987240 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.633426 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 177) took 1989 ms. Trace:
I20251211 20:52:52.633512 3294 rpcz_store.cc:276] 1211 20:52:50.643528 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.643565 (+ 37us) service_pool.cc:225] Handling call
1211 20:52:52.570678 (+1927113us) ts_tablet_manager.cc:1151] Deleting tablet 3562b5effa124378beac7ccf0d27ee7e
1211 20:52:52.570721 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.611985 (+ 41264us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.632512 (+ 20527us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.633407 (+ 895us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1926956}
I20251211 20:52:52.633996 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 3562b5effa124378beac7ccf0d27ee7e (table default.Table_7f5df390c11943918af61066a7c67c66 [id=157a0b5022e445e68ac805168ebc586f]) successfully deleted
I20251211 20:52:52.634403 3294 tablet_replica.cc:333] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:52.634603 3294 raft_consensus.cc:2243] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:52.635010 3294 raft_consensus.cc:2272] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:52.635752 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet bc6a1c84501d4e179cd2828b50fddc23 (table default.Table_8bb5b228d60e4d29a41d9ba40c33d45d [id=6ef058fada9941629e0ea4d1906abdac]). Expected version 1 got 0
I20251211 20:52:52.636139 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet f3855ffb51a74101b26fd28429e65bc2 (table default.Table_9349d035f81c4ae0bc0bc8625d0b116c [id=a923a0eb52d84adfbf2fbabfeef9ac15]) successfully deleted
I20251211 20:52:52.636266 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 3dfea28c44d84c68850db65a30d52ee6 (table default.Table_c504f8f3796f43b697488bd604268543 [id=2af76b9d32094a4ca656060a4478e08a]). Expected version 1 got 0
I20251211 20:52:52.636839 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "683d0f2068fb4a6ebc9c169ad7b455ce" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:52.636858 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 4c7b74f370b042ee8575f821c07dc085 (table default.Table_731826f26f9940b8afe425d694b642d9 [id=17dd9cea9ac844128ff9f82adb3f919f]). Expected version 1 got 0
I20251211 20:52:52.637032 1033 raft_consensus.cc:3060] T 683d0f2068fb4a6ebc9c169ad7b455ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:52.637125 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet f00128ba84c245d9a0e35f26bc842b3e (table default.Table_6f8c133c7a364b93af2c83d8911a0ac3 [id=6a21c38ee7224a02b937df088494f5d8]). Expected version 1 got 0
I20251211 20:52:52.637434 3397 raft_consensus.cc:493] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.637578 3397 raft_consensus.cc:515] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.638768 3397 leader_election.cc:290] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:52.638881 1160 tablet_bootstrap.cc:492] T 79cc81e91e1846038621af7a0a9f7a2e P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:52.639074 1160 ts_tablet_manager.cc:1403] T 79cc81e91e1846038621af7a0a9f7a2e P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.124s user 0.005s sys 0.026s
I20251211 20:52:52.640906 1160 raft_consensus.cc:359] T 79cc81e91e1846038621af7a0a9f7a2e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:52.640997 847 leader_election.cc:343] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: ccc3ddd4b7f74f11a3835997cb7ae8d0
I20251211 20:52:52.641111 1160 raft_consensus.cc:385] T 79cc81e91e1846038621af7a0a9f7a2e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.641185 1160 raft_consensus.cc:740] T 79cc81e91e1846038621af7a0a9f7a2e P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.641765 1033 raft_consensus.cc:2468] T 683d0f2068fb4a6ebc9c169ad7b455ce P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:52.630223 3295 rpcz_store.cc:276] 1211 20:52:50.999760 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.999807 (+ 47us) service_pool.cc:225] Handling call
1211 20:52:52.629753 (+1629946us) ts_tablet_manager.cc:1151] Deleting tablet a56f1a7bd5464d289bfd018c826ee574
1211 20:52:52.629765 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.629861 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1629757}
W20251211 20:52:52.644940 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet a56f1a7bd5464d289bfd018c826ee574 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a56f1a7bd5464d289bfd018c826ee574 already in progress: creating tablet
I20251211 20:52:52.642802 1160 consensus_queue.cc:260] T 79cc81e91e1846038621af7a0a9f7a2e P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.648233 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b85af79c2574443998e847bc19c53b69 (table default.Table_fa3f2f115cd043a9aa6be2b070d320cb [id=15fad5e1cf8d4ea08dae8275a29a11dd]). Expected version 1 got 0
I20251211 20:52:52.652346 1168 tablet_bootstrap.cc:492] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:52.652562 1168 ts_tablet_manager.cc:1403] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.063s user 0.001s sys 0.004s
I20251211 20:52:52.653396 1168 raft_consensus.cc:359] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.653661 1168 raft_consensus.cc:385] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.653759 1168 raft_consensus.cc:740] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.653996 1168 consensus_queue.cc:260] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.654466 1168 ts_tablet_manager.cc:1434] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:52.654812 1168 tablet_bootstrap.cc:492] T aeff1fbabf7845dc9863f59f286ddcec P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:52.656194 1160 ts_tablet_manager.cc:1434] T 79cc81e91e1846038621af7a0a9f7a2e P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.017s user 0.002s sys 0.000s
I20251211 20:52:52.657543 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 6212b3b4425543168837946a54d60834 (table default.Table_8c30c4151d484db0bb9e2b9ca262248d [id=2d489cd7d73e403d84bf10960e6a2dfb]). Expected version 1 got 0
I20251211 20:52:52.658756 3294 ts_tablet_manager.cc:1916] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:52.660099 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 33fd09ea06ee47908b85911e4ee79708 (table default.Table_3c3cd48c8fe5411e96cdf811d492925d [id=30620a01c27f456bac05fa7c02a2cb4d]). Expected version 2 got 0
I20251211 20:52:52.663120 1168 tablet_bootstrap.cc:654] T aeff1fbabf7845dc9863f59f286ddcec P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.664243 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 3c865c99e9314f5f91658b9126da4d39 (table default.Table_0d7df3fba38849a49b65d8290812b95c [id=7f966d5cdad3423bb770414802a3cf7b]). Expected version 1 got 0
I20251211 20:52:52.664558 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4388893262b54ad8be1bdf82516031a5" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:52.664747 1033 raft_consensus.cc:2468] T 4388893262b54ad8be1bdf82516031a5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:52.664685 3294 ts_tablet_manager.cc:1929] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:52.665113 3294 log.cc:1199] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/06b8f8e46e524545aeab6c931efaf116
I20251211 20:52:52.665459 3294 ts_tablet_manager.cc:1950] T 06b8f8e46e524545aeab6c931efaf116 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:52.665759 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 178) took 2008 ms. Trace:
I20251211 20:52:52.665831 3294 rpcz_store.cc:276] 1211 20:52:50.657690 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.657749 (+ 59us) service_pool.cc:225] Handling call
1211 20:52:52.634349 (+1976600us) ts_tablet_manager.cc:1151] Deleting tablet 06b8f8e46e524545aeab6c931efaf116
1211 20:52:52.634359 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.661734 (+ 27375us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.664580 (+ 2846us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.665742 (+ 1162us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1976385}
I20251211 20:52:52.666177 3294 tablet_replica.cc:333] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:52.666361 3294 raft_consensus.cc:2243] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:52.666343 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 06b8f8e46e524545aeab6c931efaf116 (table default.Table_bf0900ef3fb8447c834a43e509caa105 [id=861b00629ceb42588a1d1be34153b792]) successfully deleted
I20251211 20:52:52.666620 3294 raft_consensus.cc:2272] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:52.667141 3294 ts_tablet_manager.cc:1916] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:52.667641 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 8675e2a18f0f4a0e95a87005cf82d8b4 (table default.Table_bd9633c55bd242c18aac3a7f2c18a329 [id=767f50bee5464316a18f73e64bab817a]). Expected version 2 got 0
I20251211 20:52:52.670056 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fcf5afcfd7f142d29ff91272913e5fcb" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:52.671630 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet b115d737d7fc483d9f5a1f040b511bae (table default.Table_6d94053661f142b79292e19752df02df [id=c5be59a2c3934b3789641ceeb2d4d6e2]). Expected version 2 got 0
I20251211 20:52:52.674917 2962 ts_tablet_manager.cc:1929] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:52.675223 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet da81bf8854a24954ba4548185812295d (table default.Table_18262e22c26d4d66ba4c6134fbde0368 [id=da8cf2b99b42409091484a9513623d6f]). Expected version 1 got 0
I20251211 20:52:52.677359 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet df457783b2534b3ea23bb1fd529ce268 (table default.Table_6e9f3ca164384cc8ac4501deb3f0c6aa [id=342c2469d6e74e3e9217236e3aa863d6]). Expected version 1 got 0
I20251211 20:52:52.678226 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 5453f68789fd4e89b69e98860050865b (table default.Table_b7bf8c5ad5304175a1f2f6a03887f725 [id=50d88b663a4d4891b113aee9c5906699]). Expected version 1 got 0
I20251211 20:52:52.679355 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 46cb1d16a0e94b3a833f32c635213fd2 (table default.Table_3482fbfc5b144a7198a7d01aa003cb6b [id=1bdaf5ce0f6b4b99a038f9f9efb828f6]). Expected version 1 got 0
I20251211 20:52:52.680099 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 152) took 1672 ms. Trace:
I20251211 20:52:52.680366 1160 tablet_bootstrap.cc:492] T de508d79f30f489cb745401f53115165 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:52.681058 710 leader_election.cc:304] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.680490 3295 rpcz_store.cc:276] 1211 20:52:51.000495 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:51.000540 (+ 45us) service_pool.cc:225] Handling call
1211 20:52:52.646287 (+1645747us) ts_tablet_manager.cc:1151] Deleting tablet 06ba98a726424dd89450053c5c5793f4
1211 20:52:52.672868 (+ 26581us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.672991 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1645170}
W20251211 20:52:52.681571 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 06ba98a726424dd89450053c5c5793f4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06ba98a726424dd89450053c5c5793f4 already in progress: creating tablet
I20251211 20:52:52.681685 710 leader_election.cc:304] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.682188 2959 raft_consensus.cc:2804] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:52.682296 2959 raft_consensus.cc:697] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:52.682531 3353 raft_consensus.cc:2804] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:52.682585 3353 raft_consensus.cc:493] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:52.682500 2959 consensus_queue.cc:237] T 683d0f2068fb4a6ebc9c169ad7b455ce P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.682636 3353 raft_consensus.cc:3060] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:52.682085 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 153) took 1680 ms. Trace:
I20251211 20:52:52.684813 3294 ts_tablet_manager.cc:1929] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:52.684926 3294 log.cc:1199] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/47779034ad89466e9e1801987128b4dd
I20251211 20:52:52.684521 3295 rpcz_store.cc:276] 1211 20:52:51.001169 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:51.001212 (+ 43us) service_pool.cc:225] Handling call
1211 20:52:52.681836 (+1680624us) ts_tablet_manager.cc:1151] Deleting tablet e8db5a11a69e4554bc689c38ab57b28a
1211 20:52:52.681847 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.681940 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1680479}
I20251211 20:52:52.685088 3353 raft_consensus.cc:515] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.685271 3294 ts_tablet_manager.cc:1950] T 47779034ad89466e9e1801987128b4dd P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
W20251211 20:52:52.685461 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet e8db5a11a69e4554bc689c38ab57b28a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e8db5a11a69e4554bc689c38ab57b28a already in progress: creating tablet
I20251211 20:52:52.685588 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 179) took 2020 ms. Trace:
I20251211 20:52:52.685664 3294 rpcz_store.cc:276] 1211 20:52:50.665430 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.665482 (+ 52us) service_pool.cc:225] Handling call
1211 20:52:52.666131 (+2000649us) ts_tablet_manager.cc:1151] Deleting tablet 47779034ad89466e9e1801987128b4dd
1211 20:52:52.666140 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.669958 (+ 3818us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.684714 (+ 14756us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.685572 (+ 858us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2000457}
I20251211 20:52:52.685665 3353 leader_election.cc:290] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:52.685951 3294 tablet_replica.cc:333] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:52.686069 902 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4388893262b54ad8be1bdf82516031a5" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:52.686137 3294 raft_consensus.cc:2243] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:52.686158 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 47779034ad89466e9e1801987128b4dd (table default.Table_441ecee3b8204f9fa8d22146340e5303 [id=a5b6a2883bda49738a07b6daea5e7310]) successfully deleted
I20251211 20:52:52.686281 3294 raft_consensus.cc:2272] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:52.686465 3528 tablet.cc:1722] T fd43a27db69044b68826cda87e16a086 P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:52.687377 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 154) took 1684 ms. Trace:
W20251211 20:52:52.686587 711 leader_election.cc:343] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.686810 3294 ts_tablet_manager.cc:1916] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:52.688309 3295 rpcz_store.cc:276] 1211 20:52:51.001827 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:51.001876 (+ 49us) service_pool.cc:225] Handling call
1211 20:52:52.686463 (+1684587us) ts_tablet_manager.cc:1151] Deleting tablet 83067b139f504bc191a515feed0ec771
1211 20:52:52.686473 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.686557 (+ 84us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1683826}
W20251211 20:52:52.688850 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 83067b139f504bc191a515feed0ec771 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 83067b139f504bc191a515feed0ec771 already in progress: creating tablet
I20251211 20:52:52.689509 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 155) took 1686 ms. Trace:
I20251211 20:52:52.689635 3295 rpcz_store.cc:276] 1211 20:52:51.002459 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:51.002503 (+ 44us) service_pool.cc:225] Handling call
1211 20:52:52.689207 (+1686704us) ts_tablet_manager.cc:1151] Deleting tablet fbc2d4aea084492c8270ad1d87096e58
1211 20:52:52.689217 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.689295 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1686461}
W20251211 20:52:52.690117 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet fbc2d4aea084492c8270ad1d87096e58 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fbc2d4aea084492c8270ad1d87096e58 already in progress: creating tablet
I20251211 20:52:52.690752 3295 tablet_replica.cc:333] T fa79b67b9eee45e2b41c4a90967606cc P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:52.690553 3529 tablet.cc:1722] T fd43a27db69044b68826cda87e16a086 P eb0f1933d9a74fccaa4618047fc05c41: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:52.691017 3295 raft_consensus.cc:2243] T fa79b67b9eee45e2b41c4a90967606cc P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:52.691289 3295 raft_consensus.cc:2272] T fa79b67b9eee45e2b41c4a90967606cc P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:52.691957 3295 ts_tablet_manager.cc:1916] T fa79b67b9eee45e2b41c4a90967606cc P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:52.696249 1160 tablet_bootstrap.cc:654] T de508d79f30f489cb745401f53115165 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:52.699349 847 leader_election.cc:343] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: ccc3ddd4b7f74f11a3835997cb7ae8d0
I20251211 20:52:52.699501 847 leader_election.cc:304] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:52.700142 3397 raft_consensus.cc:2749] T ccc3ddd4b7f74f11a3835997cb7ae8d0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:52.705726 906 raft_consensus.cc:3060] T 497c10742f01476db885bd70c4d2c491 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:52.711755 3294 ts_tablet_manager.cc:1929] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:52.712276 3294 log.cc:1199] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/cce44b80c97948e1af7ac68e77cccfd6
I20251211 20:52:52.713819 3294 ts_tablet_manager.cc:1950] T cce44b80c97948e1af7ac68e77cccfd6 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:52.715183 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 180) took 2037 ms. Trace:
I20251211 20:52:52.715565 3294 rpcz_store.cc:276] 1211 20:52:50.677872 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.677938 (+ 66us) service_pool.cc:225] Handling call
1211 20:52:52.685907 (+2007969us) ts_tablet_manager.cc:1151] Deleting tablet cce44b80c97948e1af7ac68e77cccfd6
1211 20:52:52.685916 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.699081 (+ 13165us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.711334 (+ 12253us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.715161 (+ 3827us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2007738}
I20251211 20:52:52.716790 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet cce44b80c97948e1af7ac68e77cccfd6 (table default.Table_32ebd46bd3cc460986098c79c9b67ca7 [id=1bd6b7930f2447659da365b7c06b1640]) successfully deleted
I20251211 20:52:52.718045 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 182) took 2037 ms. Trace:
I20251211 20:52:52.718487 3294 rpcz_store.cc:276] 1211 20:52:50.680156 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.683513 (+ 3357us) service_pool.cc:225] Handling call
1211 20:52:52.717616 (+2034103us) ts_tablet_manager.cc:1151] Deleting tablet 80c566822fa94ada983aa7266df4bf9d
1211 20:52:52.717628 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.718027 (+ 399us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2033784}
W20251211 20:52:52.719463 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 80c566822fa94ada983aa7266df4bf9d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 80c566822fa94ada983aa7266df4bf9d already in progress: creating tablet
I20251211 20:52:52.720139 3294 tablet_replica.cc:333] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:52.720927 3294 raft_consensus.cc:2243] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:52.721545 3294 raft_consensus.cc:2272] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:52.723611 2962 log.cc:1199] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/efe4d6bb07374294b8ab50a9fe605f87
I20251211 20:52:52.723704 3294 ts_tablet_manager.cc:1916] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:52.724031 2962 ts_tablet_manager.cc:1950] T efe4d6bb07374294b8ab50a9fe605f87 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:52.676419 1166 tablet_bootstrap.cc:492] T dfd194cf00f549c8937cb6ec40987240 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:52.697782 3423 raft_consensus.cc:493] T 07e4336171714da8adc21977ce33ce50 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.724632 3423 raft_consensus.cc:515] T 07e4336171714da8adc21977ce33ce50 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.724970 1160 tablet_bootstrap.cc:492] T de508d79f30f489cb745401f53115165 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:52.725148 1160 ts_tablet_manager.cc:1403] T de508d79f30f489cb745401f53115165 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.052s user 0.002s sys 0.004s
I20251211 20:52:52.725279 3423 leader_election.cc:290] T 07e4336171714da8adc21977ce33ce50 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:52.726210 902 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "07e4336171714da8adc21977ce33ce50" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:52.726318 1166 ts_tablet_manager.cc:1403] T dfd194cf00f549c8937cb6ec40987240 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.140s user 0.000s sys 0.005s
I20251211 20:52:52.726001 1160 raft_consensus.cc:359] T de508d79f30f489cb745401f53115165 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:52.726729 983 leader_election.cc:343] T 07e4336171714da8adc21977ce33ce50 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.727048 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 834b84a557b04e78b5d2c8c84520a644 (table default.Table_909bbe6da1a648d68199fb27424e4342 [id=cc99459a5039424dbc192ff748b6708b]). Expected version 2 got 0
I20251211 20:52:52.727447 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet e89d4d76bf64453892a4d0b55fcc3454 (table default.Table_fba99384e9f14160b3e3504f8d1d0b07 [id=01bba75f854d4c338d6c09a446dd7499]). Expected version 1 got 0
I20251211 20:52:52.727707 1166 raft_consensus.cc:359] T dfd194cf00f549c8937cb6ec40987240 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.727902 1166 raft_consensus.cc:385] T dfd194cf00f549c8937cb6ec40987240 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.727986 1166 raft_consensus.cc:740] T dfd194cf00f549c8937cb6ec40987240 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.728228 1166 consensus_queue.cc:260] T dfd194cf00f549c8937cb6ec40987240 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.728658 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 7be44df39a8f4d6da793479d425fa684 (table default.Table_f6a10556ff104db5bd5ecc9fc380e228 [id=f75813add54e46189ff1bbed82d83ba7]). Expected version 1 got 0
I20251211 20:52:52.728839 1166 ts_tablet_manager.cc:1434] T dfd194cf00f549c8937cb6ec40987240 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:52.729432 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet fd43a27db69044b68826cda87e16a086 (table default.Table_7833c67862334235b0e752bfcf44ff81 [id=f1d32002c612403a82bdff5baf209b9e]). Expected version 2 got 0
I20251211 20:52:52.731101 1160 raft_consensus.cc:385] T de508d79f30f489cb745401f53115165 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.731264 1160 raft_consensus.cc:740] T de508d79f30f489cb745401f53115165 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.731707 1160 consensus_queue.cc:260] T de508d79f30f489cb745401f53115165 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.734122 1160 ts_tablet_manager.cc:1434] T de508d79f30f489cb745401f53115165 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20251211 20:52:52.736294 1160 tablet_bootstrap.cc:492] T 3f1840e4b5214ddb878935a79b214eae P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:52.741185 3295 ts_tablet_manager.cc:1929] T fa79b67b9eee45e2b41c4a90967606cc P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:52.745570 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 4fe5c534b572491c911aae01eaa79bae (table default.Table_c1f9dca5c2144d50b67681dcc6800e77 [id=14fa8debfe5443ce81587f02dfa15847]). Expected version 2 got 0
I20251211 20:52:52.746989 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 161) took 3637 ms. Trace:
I20251211 20:52:52.747138 2962 rpcz_store.cc:276] 1211 20:52:49.109745 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.134799 (+ 25054us) service_pool.cc:225] Handling call
1211 20:52:52.579089 (+3444290us) ts_tablet_manager.cc:1151] Deleting tablet efe4d6bb07374294b8ab50a9fe605f87
1211 20:52:52.579099 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.583467 (+ 4368us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.674800 (+ 91333us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.746959 (+ 72159us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3444066}
I20251211 20:52:52.747524 2962 tablet_replica.cc:333] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:52.747776 2962 raft_consensus.cc:2243] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:52.747990 2962 raft_consensus.cc:2272] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:52.748030 3294 ts_tablet_manager.cc:1929] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:52.748617 2962 ts_tablet_manager.cc:1916] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:52.748551 3294 log.cc:1199] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/bb4004212441473483a209d23c0800ce
I20251211 20:52:52.749840 1166 tablet_bootstrap.cc:492] T 88970b575ba943129729eceef5636519 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:52.749994 3294 ts_tablet_manager.cc:1950] T bb4004212441473483a209d23c0800ce P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:52.750116 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 171d40a8a9124f9bb1b38797fef8e5d5 (table default.Table_e157133a993643a586e90be0e591ea6f [id=5d98f157d4eb47bf825a3ecd427eb2e5]). Expected version 2 got 0
I20251211 20:52:52.751389 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 183) took 1955 ms. Trace:
I20251211 20:52:52.751777 3294 rpcz_store.cc:276] 1211 20:52:50.795388 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.795443 (+ 55us) service_pool.cc:225] Handling call
1211 20:52:52.719991 (+1924548us) ts_tablet_manager.cc:1151] Deleting tablet bb4004212441473483a209d23c0800ce
1211 20:52:52.720003 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.734944 (+ 14941us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.747546 (+ 12602us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.751211 (+ 3665us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1924221}
I20251211 20:52:52.747078 1160 tablet_bootstrap.cc:654] T 3f1840e4b5214ddb878935a79b214eae P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.747215 3295 log.cc:1199] T fa79b67b9eee45e2b41c4a90967606cc P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/fa79b67b9eee45e2b41c4a90967606cc
I20251211 20:52:52.752911 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet bb4004212441473483a209d23c0800ce (table default.Table_55b2abeacec041418b6359c21e87b3d8 [id=75609081b0c7488eb388faac157db3e5]) successfully deleted
I20251211 20:52:52.753384 3295 ts_tablet_manager.cc:1950] T fa79b67b9eee45e2b41c4a90967606cc P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:52.753686 1166 tablet_bootstrap.cc:654] T 88970b575ba943129729eceef5636519 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.753916 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 184) took 1949 ms. Trace:
I20251211 20:52:52.752287 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet c081396b492049b996ac4661e1318388 (table default.Table_00ae5c111de94ba0a2680ad22c726e16 [id=95c9ed1ec1834e2dbfd3198887b75369]). Expected version 1 got 0
I20251211 20:52:52.754202 3294 rpcz_store.cc:276] 1211 20:52:50.804399 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.804457 (+ 58us) service_pool.cc:225] Handling call
1211 20:52:52.753378 (+1948921us) ts_tablet_manager.cc:1151] Deleting tablet c97d410272a947de8b5d7fb40fe4777d
1211 20:52:52.753394 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.753800 (+ 406us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1948591}
I20251211 20:52:52.755405 3294 tablet_replica.cc:333] T 59e955aa07c14fbbb347b88eca59a74e P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
W20251211 20:52:52.756009 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet c97d410272a947de8b5d7fb40fe4777d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c97d410272a947de8b5d7fb40fe4777d already in progress: creating tablet
I20251211 20:52:52.756634 3294 raft_consensus.cc:2243] T 59e955aa07c14fbbb347b88eca59a74e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:52.757207 3294 raft_consensus.cc:2272] T 59e955aa07c14fbbb347b88eca59a74e P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:52.756901 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet 2e800d8cc5ea4535a065d7d9243326b4 (table default.Table_083c478bedec45a5bf5203e4ed333860 [id=8baf0ea87d734d58896adbd0d6ef99c2]). Expected version 1 got 0
I20251211 20:52:52.759466 3294 ts_tablet_manager.cc:1916] T 59e955aa07c14fbbb347b88eca59a74e P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:52.760751 1806 catalog_manager.cc:5746] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177) does not have the latest schema for tablet d43210ec61af423ebde624397dea9f2e (table default.Table_af0df4d1ee144455a6041e5a6a0b9b80 [id=7c5b2e33fa1c4b12b3941577360908a8]). Expected version 3 got 0
I20251211 20:52:52.763870 3537 tablet.cc:1722] T fd43a27db69044b68826cda87e16a086 P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251211 20:52:52.764179 1045 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4388893262b54ad8be1bdf82516031a5" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
W20251211 20:52:52.764322 982 leader_election.cc:343] T fcf5afcfd7f142d29ff91272913e5fcb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.764359 1045 raft_consensus.cc:3060] T 4388893262b54ad8be1bdf82516031a5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:52.764427 982 leader_election.cc:304] T fcf5afcfd7f142d29ff91272913e5fcb P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.765130 3423 raft_consensus.cc:2749] T fcf5afcfd7f142d29ff91272913e5fcb P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:52.766000 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "07e4336171714da8adc21977ce33ce50" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:52.766933 1045 raft_consensus.cc:2468] T 4388893262b54ad8be1bdf82516031a5 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
W20251211 20:52:52.769412 1786 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 03184dcc56dd4e92adc7d0ad22aa8d8b (table default.Table_68261841754847ad8d22e86c29886144 [id=697f23a617f64762bf406b386cde7473], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 03184dcc56dd4e92adc7d0ad22aa8d8b consensus state has no leader
I20251211 20:52:52.774791 1160 tablet_bootstrap.cc:492] T 3f1840e4b5214ddb878935a79b214eae P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:52.777804 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet efe4d6bb07374294b8ab50a9fe605f87 (table default.Table_987db791387d4fa79e285fae8bddce52 [id=9538422c0e9e48a5ae065754c84ba581]) successfully deleted
I20251211 20:52:52.778223 1168 tablet_bootstrap.cc:492] T aeff1fbabf7845dc9863f59f286ddcec P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:52.778430 1168 ts_tablet_manager.cc:1403] T aeff1fbabf7845dc9863f59f286ddcec P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.124s user 0.011s sys 0.001s
I20251211 20:52:52.778669 710 leader_election.cc:304] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.779160 3505 raft_consensus.cc:2804] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:52.779237 1168 raft_consensus.cc:359] T aeff1fbabf7845dc9863f59f286ddcec P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.779523 1168 raft_consensus.cc:385] T aeff1fbabf7845dc9863f59f286ddcec P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.779639 1168 raft_consensus.cc:740] T aeff1fbabf7845dc9863f59f286ddcec P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.779891 1168 consensus_queue.cc:260] T aeff1fbabf7845dc9863f59f286ddcec P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.779408 3505 raft_consensus.cc:697] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:52.780303 3505 consensus_queue.cc:237] T 4388893262b54ad8be1bdf82516031a5 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.780885 1168 ts_tablet_manager.cc:1434] T aeff1fbabf7845dc9863f59f286ddcec P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:52.781158 1168 tablet_bootstrap.cc:492] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:52.776691 1160 ts_tablet_manager.cc:1403] T 3f1840e4b5214ddb878935a79b214eae P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.041s user 0.001s sys 0.004s
I20251211 20:52:52.782512 1168 tablet_bootstrap.cc:654] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.784235 1166 tablet_bootstrap.cc:492] T 88970b575ba943129729eceef5636519 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:52.786039 1166 ts_tablet_manager.cc:1403] T 88970b575ba943129729eceef5636519 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.036s user 0.001s sys 0.001s
I20251211 20:52:52.786768 1166 raft_consensus.cc:359] T 88970b575ba943129729eceef5636519 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.786969 1166 raft_consensus.cc:385] T 88970b575ba943129729eceef5636519 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.787048 1166 raft_consensus.cc:740] T 88970b575ba943129729eceef5636519 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.787285 1166 consensus_queue.cc:260] T 88970b575ba943129729eceef5636519 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.787824 1166 ts_tablet_manager.cc:1434] T 88970b575ba943129729eceef5636519 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:52.790236 1160 raft_consensus.cc:359] T 3f1840e4b5214ddb878935a79b214eae P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.799328 3294 ts_tablet_manager.cc:1929] T 59e955aa07c14fbbb347b88eca59a74e P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:52.799876 1166 tablet_bootstrap.cc:492] T bc4e7362e6d045da81bf39f4135f500e P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:52.799894 3294 log.cc:1199] T 59e955aa07c14fbbb347b88eca59a74e P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/59e955aa07c14fbbb347b88eca59a74e
I20251211 20:52:52.801183 3294 ts_tablet_manager.cc:1950] T 59e955aa07c14fbbb347b88eca59a74e P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:52.793867 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 156) took 1790 ms. Trace:
I20251211 20:52:52.802052 3295 rpcz_store.cc:276] 1211 20:52:51.003256 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:51.003302 (+ 46us) service_pool.cc:225] Handling call
1211 20:52:52.690387 (+1687085us) ts_tablet_manager.cc:1151] Deleting tablet fa79b67b9eee45e2b41c4a90967606cc
1211 20:52:52.690396 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.710283 (+ 19887us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.739829 (+ 29546us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.793752 (+ 53923us) spinlock_profiling.cc:243] Waited 39.6 ms on lock 0x61800000c518. stack: 00007f8ff9005ee6 00007f8ff9005ca0 00007f8ff7fe13ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f8ff8fee81d 00007f8ff8fef88e 00007f900cc2fbdd 00007f900cbae193 00007f9008ddebc3 00007f9008e11744 00007f9008df2240 00007f9008df1fcd 0000000000a8970c 00007f8ff904c20f
1211 20:52:52.793826 (+ 74us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":91511936,"tablet-delete.queue_time_us":1686915}
I20251211 20:52:52.802749 1783 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet fa79b67b9eee45e2b41c4a90967606cc (table default.Table_8f7e347b9ac64793b1073dfa955ebe21 [id=a6ab64c3e0d94462b716f2d153bc3314]) successfully deleted
I20251211 20:52:52.802917 1166 tablet_bootstrap.cc:654] T bc4e7362e6d045da81bf39f4135f500e P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.803294 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 157) took 1799 ms. Trace:
W20251211 20:52:52.792227 982 leader_election.cc:343] T 07e4336171714da8adc21977ce33ce50 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.803471 3295 rpcz_store.cc:276] 1211 20:52:51.003929 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:51.003972 (+ 43us) service_pool.cc:225] Handling call
1211 20:52:52.803130 (+1799158us) ts_tablet_manager.cc:1151] Deleting tablet 13ceabb190494e718a52892faaa53986
1211 20:52:52.803141 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.803256 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1799023}
I20251211 20:52:52.803637 982 leader_election.cc:304] T 07e4336171714da8adc21977ce33ce50 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
W20251211 20:52:52.804075 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 13ceabb190494e718a52892faaa53986 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 13ceabb190494e718a52892faaa53986 already in progress: creating tablet
I20251211 20:52:52.804296 3533 raft_consensus.cc:2749] T 07e4336171714da8adc21977ce33ce50 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:52.804292 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 186) took 1994 ms. Trace:
I20251211 20:52:52.804560 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 158) took 1800 ms. Trace:
I20251211 20:52:52.804719 3295 rpcz_store.cc:276] 1211 20:52:51.004512 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:51.004556 (+ 44us) service_pool.cc:225] Handling call
1211 20:52:52.804369 (+1799813us) ts_tablet_manager.cc:1151] Deleting tablet 6663f13691a24fb991b48f543d9de76b
1211 20:52:52.804378 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.804495 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1799670}
W20251211 20:52:52.805190 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 6663f13691a24fb991b48f543d9de76b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6663f13691a24fb991b48f543d9de76b already in progress: creating tablet
I20251211 20:52:52.804739 3294 rpcz_store.cc:276] 1211 20:52:50.809806 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.809840 (+ 34us) service_pool.cc:225] Handling call
1211 20:52:52.755158 (+1945318us) ts_tablet_manager.cc:1151] Deleting tablet 59e955aa07c14fbbb347b88eca59a74e
1211 20:52:52.755269 (+ 111us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.779974 (+ 24705us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.798941 (+ 18967us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.804162 (+ 5221us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1945124}
I20251211 20:52:52.805624 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 159) took 1800 ms. Trace:
I20251211 20:52:52.805778 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 59e955aa07c14fbbb347b88eca59a74e (table default.Table_f84cd3de863b446894133bb40aaf1a39 [id=f4d608cc41d0408486d3f6b0a09383a4]) successfully deleted
I20251211 20:52:52.805790 3295 rpcz_store.cc:276] 1211 20:52:51.005131 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:51.005171 (+ 40us) service_pool.cc:225] Handling call
1211 20:52:52.805512 (+1800341us) ts_tablet_manager.cc:1151] Deleting tablet 10746162da4640e6a2a891467690b770
1211 20:52:52.805521 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.805592 (+ 71us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1800193}
W20251211 20:52:52.806128 1783 catalog_manager.cc:4965] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): delete failed for tablet 10746162da4640e6a2a891467690b770 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 10746162da4640e6a2a891467690b770 already in progress: creating tablet
I20251211 20:52:52.806185 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 187) took 1969 ms. Trace:
I20251211 20:52:52.806499 3295 tablet_replica.cc:333] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:52.806771 3295 raft_consensus.cc:2243] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:52.806517 3294 rpcz_store.cc:276] 1211 20:52:50.836674 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.836720 (+ 46us) service_pool.cc:225] Handling call
1211 20:52:52.805808 (+1969088us) ts_tablet_manager.cc:1151] Deleting tablet 6029c1fd563d4023a8cf7cfb8ffa0876
1211 20:52:52.805819 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.806074 (+ 255us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1968890}
I20251211 20:52:52.807040 3295 raft_consensus.cc:2272] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:52.807340 1780 catalog_manager.cc:4952] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 6029c1fd563d4023a8cf7cfb8ffa0876 because the tablet was not found. No further retry: Not found: Tablet not found: 6029c1fd563d4023a8cf7cfb8ffa0876
I20251211 20:52:52.807749 3295 ts_tablet_manager.cc:1916] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:52.807763 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 188) took 1970 ms. Trace:
I20251211 20:52:52.808133 3294 rpcz_store.cc:276] 1211 20:52:50.837037 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.837073 (+ 36us) service_pool.cc:225] Handling call
1211 20:52:52.807350 (+1970277us) ts_tablet_manager.cc:1151] Deleting tablet d0673c37c2fd47eea8ab10d9e91b8d7a
1211 20:52:52.807452 (+ 102us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.807745 (+ 293us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1970141}
W20251211 20:52:52.808938 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet d0673c37c2fd47eea8ab10d9e91b8d7a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d0673c37c2fd47eea8ab10d9e91b8d7a already in progress: creating tablet
I20251211 20:52:52.809334 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 189) took 1971 ms. Trace:
I20251211 20:52:52.809607 3294 rpcz_store.cc:276] 1211 20:52:50.837471 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.837508 (+ 37us) service_pool.cc:225] Handling call
1211 20:52:52.808941 (+1971433us) ts_tablet_manager.cc:1151] Deleting tablet 516f7181aaac48cdadf9fbc08d0d3f57
1211 20:52:52.808950 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.809223 (+ 273us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1971182}
W20251211 20:52:52.810550 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 516f7181aaac48cdadf9fbc08d0d3f57 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 516f7181aaac48cdadf9fbc08d0d3f57 already in progress: creating tablet
I20251211 20:52:52.810912 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 190) took 1970 ms. Trace:
I20251211 20:52:52.811094 1160 raft_consensus.cc:385] T 3f1840e4b5214ddb878935a79b214eae P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.811188 3294 rpcz_store.cc:276] 1211 20:52:50.840785 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.840828 (+ 43us) service_pool.cc:225] Handling call
1211 20:52:52.810520 (+1969692us) ts_tablet_manager.cc:1151] Deleting tablet 4776c456b4d74c90b1fc71d57b62a461
1211 20:52:52.810528 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.810795 (+ 267us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1969409}
I20251211 20:52:52.812750 1160 raft_consensus.cc:740] T 3f1840e4b5214ddb878935a79b214eae P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
W20251211 20:52:52.813107 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 4776c456b4d74c90b1fc71d57b62a461 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4776c456b4d74c90b1fc71d57b62a461 already in progress: creating tablet
I20251211 20:52:52.813485 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 192) took 1971 ms. Trace:
I20251211 20:52:52.813743 3294 rpcz_store.cc:276] 1211 20:52:50.841555 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.841594 (+ 39us) service_pool.cc:225] Handling call
1211 20:52:52.812053 (+1970459us) ts_tablet_manager.cc:1151] Deleting tablet 6a61cf8f5de14adaa4a1de5bddde9a2f
1211 20:52:52.812063 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.813375 (+ 1312us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1970211}
W20251211 20:52:52.814740 1780 catalog_manager.cc:4952] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 6a61cf8f5de14adaa4a1de5bddde9a2f because the tablet was not found. No further retry: Not found: Tablet not found: 6a61cf8f5de14adaa4a1de5bddde9a2f
I20251211 20:52:52.815016 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 193) took 1973 ms. Trace:
I20251211 20:52:52.815284 3294 rpcz_store.cc:276] 1211 20:52:50.841859 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.841911 (+ 52us) service_pool.cc:225] Handling call
1211 20:52:52.814595 (+1972684us) ts_tablet_manager.cc:1151] Deleting tablet 9c5c937cc7db4ed18fe03f4a8a99a66a
1211 20:52:52.814606 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.814897 (+ 291us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1972455}
I20251211 20:52:52.816799 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 194) took 1967 ms. Trace:
W20251211 20:52:52.816968 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 9c5c937cc7db4ed18fe03f4a8a99a66a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9c5c937cc7db4ed18fe03f4a8a99a66a already in progress: creating tablet
I20251211 20:52:52.817070 3294 rpcz_store.cc:276] 1211 20:52:50.848911 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.848973 (+ 62us) service_pool.cc:225] Handling call
1211 20:52:52.816397 (+1967424us) ts_tablet_manager.cc:1151] Deleting tablet 0f8808bfd68841e4ae30818018040cf9
1211 20:52:52.816408 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.816688 (+ 280us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1967123}
W20251211 20:52:52.817878 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 0f8808bfd68841e4ae30818018040cf9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0f8808bfd68841e4ae30818018040cf9 already in progress: creating tablet
W20251211 20:52:52.813933 847 consensus_peers.cc:597] T bb4004212441473483a209d23c0800ce P eb0f1933d9a74fccaa4618047fc05c41 -> Peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Couldn't send request to peer 6f8d784fd1cd492bbe3809403ec34919. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb4004212441473483a209d23c0800ce. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:52.818465 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 195) took 1968 ms. Trace:
W20251211 20:52:52.818967 710 consensus_peers.cc:597] T 32f8d104f6a948d1a97a83769ea82449 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:52.818740 3294 rpcz_store.cc:276] 1211 20:52:50.849450 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.849493 (+ 43us) service_pool.cc:225] Handling call
1211 20:52:52.817995 (+1968502us) ts_tablet_manager.cc:1151] Deleting tablet 7d4e63f54d8443ae8322d4659a165e76
1211 20:52:52.818008 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.818349 (+ 341us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1968249}
W20251211 20:52:52.819666 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 7d4e63f54d8443ae8322d4659a165e76 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d4e63f54d8443ae8322d4659a165e76 already in progress: creating tablet
I20251211 20:52:52.820086 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 196) took 1970 ms. Trace:
I20251211 20:52:52.821703 3294 rpcz_store.cc:276] 1211 20:52:50.849812 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.849849 (+ 37us) service_pool.cc:225] Handling call
1211 20:52:52.819681 (+1969832us) ts_tablet_manager.cc:1151] Deleting tablet bb9fccb6981e446a939a28d7fdd38e25
1211 20:52:52.819692 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.819972 (+ 280us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1969582}
W20251211 20:52:52.822665 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet bb9fccb6981e446a939a28d7fdd38e25 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb9fccb6981e446a939a28d7fdd38e25 already in progress: creating tablet
I20251211 20:52:52.820416 1160 consensus_queue.cc:260] T 3f1840e4b5214ddb878935a79b214eae P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.823071 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 197) took 1972 ms. Trace:
I20251211 20:52:52.823406 3294 rpcz_store.cc:276] 1211 20:52:50.850142 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.850184 (+ 42us) service_pool.cc:225] Handling call
1211 20:52:52.822670 (+1972486us) ts_tablet_manager.cc:1151] Deleting tablet 449921ea74f04a9c88eb9e094e8226c8
1211 20:52:52.822679 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.823052 (+ 373us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1972354}
W20251211 20:52:52.824419 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 449921ea74f04a9c88eb9e094e8226c8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 449921ea74f04a9c88eb9e094e8226c8 already in progress: creating tablet
I20251211 20:52:52.824437 3533 raft_consensus.cc:493] T dfd194cf00f549c8937cb6ec40987240 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.824566 3533 raft_consensus.cc:515] T dfd194cf00f549c8937cb6ec40987240 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.824857 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 200) took 1911 ms. Trace:
I20251211 20:52:52.825443 3533 leader_election.cc:290] T dfd194cf00f549c8937cb6ec40987240 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:52.825204 3294 rpcz_store.cc:276] 1211 20:52:50.913165 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.913206 (+ 41us) service_pool.cc:225] Handling call
1211 20:52:52.824453 (+1911247us) ts_tablet_manager.cc:1151] Deleting tablet 41627cb27be8443fa37e514f9b3825a6
1211 20:52:52.824463 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.824743 (+ 280us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1908538}
I20251211 20:52:52.825848 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dfd194cf00f549c8937cb6ec40987240" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:52.825977 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dfd194cf00f549c8937cb6ec40987240" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:52.826280 983 leader_election.cc:343] T dfd194cf00f549c8937cb6ec40987240 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:52.826454 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 41627cb27be8443fa37e514f9b3825a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 41627cb27be8443fa37e514f9b3825a6 already in progress: creating tablet
I20251211 20:52:52.826547 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 201) took 1913 ms. Trace:
W20251211 20:52:52.826685 982 leader_election.cc:343] T dfd194cf00f549c8937cb6ec40987240 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.826768 982 leader_election.cc:304] T dfd194cf00f549c8937cb6ec40987240 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.826959 3533 raft_consensus.cc:2749] T dfd194cf00f549c8937cb6ec40987240 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:52.826817 3294 rpcz_store.cc:276] 1211 20:52:50.913429 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.913470 (+ 41us) service_pool.cc:225] Handling call
1211 20:52:52.826147 (+1912677us) ts_tablet_manager.cc:1151] Deleting tablet 87c4ec07e82d4bf79b4eeefe732674ff
1211 20:52:52.826157 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.826434 (+ 277us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1910177}
W20251211 20:52:52.827564 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 87c4ec07e82d4bf79b4eeefe732674ff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 87c4ec07e82d4bf79b4eeefe732674ff already in progress: creating tablet
I20251211 20:52:52.829253 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 198) took 1978 ms. Trace:
I20251211 20:52:52.829531 3294 rpcz_store.cc:276] 1211 20:52:50.850449 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.912285 (+ 61836us) service_pool.cc:225] Handling call
1211 20:52:52.828381 (+1916096us) ts_tablet_manager.cc:1151] Deleting tablet 9b99860ee6be45a0b358e686332d3137
1211 20:52:52.828521 (+ 140us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.829136 (+ 615us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1912290}
W20251211 20:52:52.830411 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 9b99860ee6be45a0b358e686332d3137 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9b99860ee6be45a0b358e686332d3137 already in progress: creating tablet
I20251211 20:52:52.830870 3294 tablet_replica.cc:333] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:52.831423 3294 raft_consensus.cc:2243] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:52.831882 1160 ts_tablet_manager.cc:1434] T 3f1840e4b5214ddb878935a79b214eae P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.050s user 0.000s sys 0.002s
I20251211 20:52:52.832094 3294 raft_consensus.cc:2272] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:52.833930 2962 ts_tablet_manager.cc:1929] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:52.834081 3294 ts_tablet_manager.cc:1916] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:52.836144 1160 tablet_bootstrap.cc:492] T 32f8d104f6a948d1a97a83769ea82449 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:52.838042 1160 tablet_bootstrap.cc:654] T 32f8d104f6a948d1a97a83769ea82449 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.839960 1168 tablet_bootstrap.cc:492] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:52.840133 1168 ts_tablet_manager.cc:1403] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.059s user 0.002s sys 0.001s
I20251211 20:52:52.840911 1168 raft_consensus.cc:359] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.841096 1168 raft_consensus.cc:385] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.841163 1168 raft_consensus.cc:740] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.841383 1168 consensus_queue.cc:260] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.841979 1168 ts_tablet_manager.cc:1434] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:52.842273 1168 tablet_bootstrap.cc:492] T 1b8acfdf1a26427aa9863280dc39b818 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:52.857435 1160 tablet_bootstrap.cc:492] T 32f8d104f6a948d1a97a83769ea82449 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:52.857674 1160 ts_tablet_manager.cc:1403] T 32f8d104f6a948d1a97a83769ea82449 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.000s
I20251211 20:52:52.861196 3294 ts_tablet_manager.cc:1929] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:52.861608 3294 log.cc:1199] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/4ce495233a5e4f3792faf44680df54ba
I20251211 20:52:52.863530 3294 ts_tablet_manager.cc:1950] T 4ce495233a5e4f3792faf44680df54ba P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:52.864794 1168 tablet_bootstrap.cc:654] T 1b8acfdf1a26427aa9863280dc39b818 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.866629 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 213) took 1951 ms. Trace:
I20251211 20:52:52.867035 3294 rpcz_store.cc:276] 1211 20:52:50.915504 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.917830 (+ 2326us) service_pool.cc:225] Handling call
1211 20:52:52.830483 (+1912653us) ts_tablet_manager.cc:1151] Deleting tablet 4ce495233a5e4f3792faf44680df54ba
1211 20:52:52.830803 (+ 320us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.846671 (+ 15868us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.860790 (+ 14119us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.866606 (+ 5816us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1912367}
I20251211 20:52:52.869315 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 4ce495233a5e4f3792faf44680df54ba (table default.Table_488913be2ade428593d6c0934e54a8f0 [id=f8adc6a8019d4b6b949eb34bde21031a]) successfully deleted
I20251211 20:52:52.868469 3294 tablet_replica.cc:333] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:52.871821 3294 raft_consensus.cc:2243] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:52.873071 3294 raft_consensus.cc:2272] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:52.874251 1160 raft_consensus.cc:359] T 32f8d104f6a948d1a97a83769ea82449 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.874483 1160 raft_consensus.cc:385] T 32f8d104f6a948d1a97a83769ea82449 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.874593 1160 raft_consensus.cc:740] T 32f8d104f6a948d1a97a83769ea82449 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.859545 3533 raft_consensus.cc:493] T 43044ba5660345eaa2b4e6562cc8d448 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.874749 3533 raft_consensus.cc:515] T 43044ba5660345eaa2b4e6562cc8d448 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.875267 3533 leader_election.cc:290] T 43044ba5660345eaa2b4e6562cc8d448 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:52.860503 1166 tablet_bootstrap.cc:492] T bc4e7362e6d045da81bf39f4135f500e P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:52.861603 3448 raft_consensus.cc:493] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6f8d784fd1cd492bbe3809403ec34919)
I20251211 20:52:52.875478 3448 raft_consensus.cc:515] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.875942 3448 leader_election.cc:290] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
W20251211 20:52:52.872092 983 consensus_peers.cc:597] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:52.873579 3423 consensus_queue.cc:1048] T 51d12723f5194afb97d2c05fcbaf3776 P e715a811a82446ce8d5bb4aafd5b81c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:52.876180 3294 ts_tablet_manager.cc:1916] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:52.876435 1166 ts_tablet_manager.cc:1403] T bc4e7362e6d045da81bf39f4135f500e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.077s user 0.015s sys 0.004s
I20251211 20:52:52.877241 1166 raft_consensus.cc:359] T bc4e7362e6d045da81bf39f4135f500e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.877403 1166 raft_consensus.cc:385] T bc4e7362e6d045da81bf39f4135f500e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.877468 1166 raft_consensus.cc:740] T bc4e7362e6d045da81bf39f4135f500e P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.877655 1166 consensus_queue.cc:260] T bc4e7362e6d045da81bf39f4135f500e P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.878245 1166 ts_tablet_manager.cc:1434] T bc4e7362e6d045da81bf39f4135f500e P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:52.878527 1166 tablet_bootstrap.cc:492] T 36d91772201d4c5b928d9dee3f583eb4 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:52.879964 1166 tablet_bootstrap.cc:654] T 36d91772201d4c5b928d9dee3f583eb4 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.878468 902 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "43044ba5660345eaa2b4e6562cc8d448" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:52.880936 1160 consensus_queue.cc:260] T 32f8d104f6a948d1a97a83769ea82449 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.881625 2962 log.cc:1199] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/32f8d104f6a948d1a97a83769ea82449
I20251211 20:52:52.882028 2962 ts_tablet_manager.cc:1950] T 32f8d104f6a948d1a97a83769ea82449 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:52.882779 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 162) took 3747 ms. Trace:
I20251211 20:52:52.884440 1168 tablet_bootstrap.cc:492] T 1b8acfdf1a26427aa9863280dc39b818 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:52.885214 1168 ts_tablet_manager.cc:1403] T 1b8acfdf1a26427aa9863280dc39b818 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.043s user 0.000s sys 0.003s
W20251211 20:52:52.883642 983 leader_election.cc:343] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Not found: Tablet not found: 8feee01fc0aa4b32be926ff2b617b894
W20251211 20:52:52.886411 983 leader_election.cc:343] T 43044ba5660345eaa2b4e6562cc8d448 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.886574 2962 rpcz_store.cc:276] 1211 20:52:49.135256 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.135302 (+ 46us) service_pool.cc:225] Handling call
1211 20:52:52.747463 (+3612161us) ts_tablet_manager.cc:1151] Deleting tablet 32f8d104f6a948d1a97a83769ea82449
1211 20:52:52.747473 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.806316 (+ 58843us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.833795 (+ 27479us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.882759 (+ 48964us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3611973}
I20251211 20:52:52.886934 2962 tablet_replica.cc:333] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:52.887305 2962 raft_consensus.cc:2243] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:52.887223 3295 ts_tablet_manager.cc:1929] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:52.887924 1160 ts_tablet_manager.cc:1434] T 32f8d104f6a948d1a97a83769ea82449 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.024s user 0.001s sys 0.001s
I20251211 20:52:52.888857 3295 log.cc:1199] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/92a71dff4ea8445d8484fc2747471148
I20251211 20:52:52.890277 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 32f8d104f6a948d1a97a83769ea82449 (table default.Table_aa83784f17a84101a53b3e111f31d93e [id=4b8f683aba234b40b129ba35b19873ca]) successfully deleted
I20251211 20:52:52.890801 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "43044ba5660345eaa2b4e6562cc8d448" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:52.891618 1160 tablet_bootstrap.cc:492] T f91487dd7e0f4c8b8483c2b83945da3b P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
W20251211 20:52:52.891973 982 leader_election.cc:343] T 43044ba5660345eaa2b4e6562cc8d448 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.890993 1168 raft_consensus.cc:359] T 1b8acfdf1a26427aa9863280dc39b818 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.892089 982 leader_election.cc:304] T 43044ba5660345eaa2b4e6562cc8d448 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.892251 1168 raft_consensus.cc:385] T 1b8acfdf1a26427aa9863280dc39b818 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20251211 20:52:52.892434 982 leader_election.cc:343] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 8feee01fc0aa4b32be926ff2b617b894
I20251211 20:52:52.892513 982 leader_election.cc:304] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.892736 3544 raft_consensus.cc:2749] T 43044ba5660345eaa2b4e6562cc8d448 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:52.892863 3423 raft_consensus.cc:2749] T 8feee01fc0aa4b32be926ff2b617b894 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:52:52.893128 1168 raft_consensus.cc:740] T 1b8acfdf1a26427aa9863280dc39b818 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.894169 1168 consensus_queue.cc:260] T 1b8acfdf1a26427aa9863280dc39b818 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.897554 1168 ts_tablet_manager.cc:1434] T 1b8acfdf1a26427aa9863280dc39b818 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20251211 20:52:52.898537 3295 ts_tablet_manager.cc:1950] T 92a71dff4ea8445d8484fc2747471148 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:52.902478 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 160) took 1896 ms. Trace:
I20251211 20:52:52.907944 1160 tablet_bootstrap.cc:654] T f91487dd7e0f4c8b8483c2b83945da3b P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.907979 2962 raft_consensus.cc:2272] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:52.908725 2962 ts_tablet_manager.cc:1916] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:52.909049 3294 ts_tablet_manager.cc:1929] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:52.910034 3294 log.cc:1199] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/1578887b2f5748cd83a6fbdec2f3fc99
I20251211 20:52:52.902693 3295 rpcz_store.cc:276] 1211 20:52:51.005768 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:51.005807 (+ 39us) service_pool.cc:225] Handling call
1211 20:52:52.806398 (+1800591us) ts_tablet_manager.cc:1151] Deleting tablet 92a71dff4ea8445d8484fc2747471148
1211 20:52:52.806406 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.840317 (+ 33911us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.885735 (+ 45418us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.902454 (+ 16719us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1800450}
I20251211 20:52:52.911696 3294 ts_tablet_manager.cc:1950] T 1578887b2f5748cd83a6fbdec2f3fc99 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:52.911864 1783 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 92a71dff4ea8445d8484fc2747471148 (table default.Table_c7e035d673434c588da10ea5a480f541 [id=6a07ca569d504b8292feabcab2e3f70c]) successfully deleted
I20251211 20:52:52.919162 1168 tablet_bootstrap.cc:492] T 1c4b2c22bd4c456eb77af45576de466f P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:52.920629 1168 tablet_bootstrap.cc:654] T 1c4b2c22bd4c456eb77af45576de466f P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.909181 3544 raft_consensus.cc:493] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6f8d784fd1cd492bbe3809403ec34919)
I20251211 20:52:52.922154 3544 raft_consensus.cc:515] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } }
I20251211 20:52:52.922816 3544 leader_election.cc:290] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:52.922865 3295 tablet_replica.cc:333] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:52.922835 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 202) took 2009 ms. Trace:
I20251211 20:52:52.921752 1166 tablet_bootstrap.cc:492] T 36d91772201d4c5b928d9dee3f583eb4 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
W20251211 20:52:52.923687 983 leader_election.cc:343] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Not found: Tablet not found: 3fc173c396ff4af5a8589cff18165e85
I20251211 20:52:52.923725 3295 raft_consensus.cc:2243] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus shutting down.
W20251211 20:52:52.923822 982 leader_election.cc:343] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 3fc173c396ff4af5a8589cff18165e85
I20251211 20:52:52.923916 3295 raft_consensus.cc:2272] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:52.923938 982 leader_election.cc:304] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.923991 1166 ts_tablet_manager.cc:1403] T 36d91772201d4c5b928d9dee3f583eb4 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.046s user 0.004s sys 0.000s
I20251211 20:52:52.923249 3294 rpcz_store.cc:276] 1211 20:52:50.913715 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.913759 (+ 44us) service_pool.cc:225] Handling call
1211 20:52:52.868319 (+1954560us) ts_tablet_manager.cc:1151] Deleting tablet 1578887b2f5748cd83a6fbdec2f3fc99
1211 20:52:52.868331 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.895975 (+ 27644us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.908636 (+ 12661us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.922712 (+ 14076us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1950174}
I20251211 20:52:52.924563 3295 ts_tablet_manager.cc:1916] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:52.924563 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 1578887b2f5748cd83a6fbdec2f3fc99 (table default.Table_328b7dd76e9e407c83b47a0f51cfcce5 [id=9e7c08f833e14a90b17a9b9f60b702de]) successfully deleted
I20251211 20:52:52.924161 3544 raft_consensus.cc:2749] T 3fc173c396ff4af5a8589cff18165e85 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:52:52.925711 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 203) took 2011 ms. Trace:
I20251211 20:52:52.926107 3294 rpcz_store.cc:276] 1211 20:52:50.913949 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.914004 (+ 55us) service_pool.cc:225] Handling call
1211 20:52:52.924758 (+2010754us) ts_tablet_manager.cc:1151] Deleting tablet 6cfa15533fad4aec84f8d9038227faf6
1211 20:52:52.924821 (+ 63us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.925591 (+ 770us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2006590}
W20251211 20:52:52.927132 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 6cfa15533fad4aec84f8d9038227faf6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6cfa15533fad4aec84f8d9038227faf6 already in progress: creating tablet
I20251211 20:52:52.927321 1166 raft_consensus.cc:359] T 36d91772201d4c5b928d9dee3f583eb4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.927488 1166 raft_consensus.cc:385] T 36d91772201d4c5b928d9dee3f583eb4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.927544 1166 raft_consensus.cc:740] T 36d91772201d4c5b928d9dee3f583eb4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.927852 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 204) took 2013 ms. Trace:
I20251211 20:52:52.928246 3294 rpcz_store.cc:276] 1211 20:52:50.914050 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.914093 (+ 43us) service_pool.cc:225] Handling call
1211 20:52:52.927185 (+2013092us) ts_tablet_manager.cc:1151] Deleting tablet 48820386e41b47129a13bdd5768f021b
1211 20:52:52.927196 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.927790 (+ 594us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2008968}
W20251211 20:52:52.929409 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 48820386e41b47129a13bdd5768f021b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 48820386e41b47129a13bdd5768f021b already in progress: creating tablet
I20251211 20:52:52.929966 3294 tablet_replica.cc:333] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:52.935174 1166 consensus_queue.cc:260] T 36d91772201d4c5b928d9dee3f583eb4 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.937105 3294 raft_consensus.cc:2243] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:52.937711 3294 raft_consensus.cc:2272] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:52.938681 3294 ts_tablet_manager.cc:1916] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:52.940284 3353 consensus_queue.cc:1048] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:52:52.943713 1166 ts_tablet_manager.cc:1434] T 36d91772201d4c5b928d9dee3f583eb4 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.018s user 0.002s sys 0.000s
W20251211 20:52:52.943847 847 consensus_peers.cc:597] T cce44b80c97948e1af7ac68e77cccfd6 P eb0f1933d9a74fccaa4618047fc05c41 -> Peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Couldn't send request to peer 6f8d784fd1cd492bbe3809403ec34919. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cce44b80c97948e1af7ac68e77cccfd6. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:52.946667 1166 tablet_bootstrap.cc:492] T a4ccf9a4ecbc4ca6978a189cf3a15893 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:52.946854 1160 tablet_bootstrap.cc:492] T f91487dd7e0f4c8b8483c2b83945da3b P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:52.947024 1160 ts_tablet_manager.cc:1403] T f91487dd7e0f4c8b8483c2b83945da3b P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.057s user 0.003s sys 0.000s
I20251211 20:52:52.947896 1160 raft_consensus.cc:359] T f91487dd7e0f4c8b8483c2b83945da3b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.948175 1160 raft_consensus.cc:385] T f91487dd7e0f4c8b8483c2b83945da3b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.948276 1160 raft_consensus.cc:740] T f91487dd7e0f4c8b8483c2b83945da3b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.948509 1166 tablet_bootstrap.cc:654] T a4ccf9a4ecbc4ca6978a189cf3a15893 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.948518 1160 consensus_queue.cc:260] T f91487dd7e0f4c8b8483c2b83945da3b P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.949335 1160 ts_tablet_manager.cc:1434] T f91487dd7e0f4c8b8483c2b83945da3b P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:52:52.949846 1160 tablet_bootstrap.cc:492] T 969a8587019c4202b6e705d126f83d44 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:52.952575 1160 tablet_bootstrap.cc:654] T 969a8587019c4202b6e705d126f83d44 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:52.954142 3505 raft_consensus.cc:493] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.954285 3505 raft_consensus.cc:515] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.954869 3505 leader_election.cc:290] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:52.955369 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29463a8f71f943a0927942235af3c2da" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:52.956105 711 leader_election.cc:343] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.956647 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29463a8f71f943a0927942235af3c2da" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:52.956830 1033 raft_consensus.cc:2468] T 29463a8f71f943a0927942235af3c2da P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:52.957129 710 leader_election.cc:304] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.957499 3353 raft_consensus.cc:2804] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:52.957579 3353 raft_consensus.cc:493] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:52.957654 3353 raft_consensus.cc:3060] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:52.960774 3353 raft_consensus.cc:515] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.961510 3353 leader_election.cc:290] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:52.961689 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29463a8f71f943a0927942235af3c2da" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:52.962005 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29463a8f71f943a0927942235af3c2da" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:52.962158 1033 raft_consensus.cc:3060] T 29463a8f71f943a0927942235af3c2da P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:52.962177 711 leader_election.cc:343] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:52.964620 1033 raft_consensus.cc:2468] T 29463a8f71f943a0927942235af3c2da P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:52.964965 710 leader_election.cc:304] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:52.965444 3353 raft_consensus.cc:2804] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:52.965687 3353 raft_consensus.cc:697] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:52.965942 3353 consensus_queue.cc:237] T 29463a8f71f943a0927942235af3c2da P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.968878 3397 raft_consensus.cc:493] T 32f8d104f6a948d1a97a83769ea82449 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.969000 3397 raft_consensus.cc:515] T 32f8d104f6a948d1a97a83769ea82449 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.969516 3397 leader_election.cc:290] T 32f8d104f6a948d1a97a83769ea82449 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:52.970199 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "32f8d104f6a948d1a97a83769ea82449" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:52.971053 2962 ts_tablet_manager.cc:1929] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:52.971164 2962 log.cc:1199] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/940dfd8781de444a9caa78dd363c7451
I20251211 20:52:52.971495 2962 ts_tablet_manager.cc:1950] T 940dfd8781de444a9caa78dd363c7451 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:52.972572 906 raft_consensus.cc:3060] T de508d79f30f489cb745401f53115165 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:52.973310 1168 tablet_bootstrap.cc:492] T 1c4b2c22bd4c456eb77af45576de466f P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:52.973423 1168 ts_tablet_manager.cc:1403] T 1c4b2c22bd4c456eb77af45576de466f P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.054s user 0.008s sys 0.003s
I20251211 20:52:52.974064 1168 raft_consensus.cc:359] T 1c4b2c22bd4c456eb77af45576de466f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.974215 1168 raft_consensus.cc:385] T 1c4b2c22bd4c456eb77af45576de466f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.974257 1168 raft_consensus.cc:740] T 1c4b2c22bd4c456eb77af45576de466f P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.974452 1168 consensus_queue.cc:260] T 1c4b2c22bd4c456eb77af45576de466f P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.975512 1168 ts_tablet_manager.cc:1434] T 1c4b2c22bd4c456eb77af45576de466f P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20251211 20:52:52.975755 847 leader_election.cc:343] T 32f8d104f6a948d1a97a83769ea82449 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 32f8d104f6a948d1a97a83769ea82449
I20251211 20:52:52.975867 847 leader_election.cc:304] T 32f8d104f6a948d1a97a83769ea82449 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:52.976269 3438 raft_consensus.cc:2749] T 32f8d104f6a948d1a97a83769ea82449 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:52.976589 3438 raft_consensus.cc:493] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:52.976701 3438 raft_consensus.cc:515] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.977193 3438 leader_election.cc:290] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:52.978843 847 leader_election.cc:343] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: 25454e67437541d9ac2bfab29bad55f9
I20251211 20:52:52.979722 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 163) took 3843 ms. Trace:
I20251211 20:52:52.979861 2962 rpcz_store.cc:276] 1211 20:52:49.135753 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.135796 (+ 43us) service_pool.cc:225] Handling call
1211 20:52:52.886879 (+3751083us) ts_tablet_manager.cc:1151] Deleting tablet 940dfd8781de444a9caa78dd363c7451
1211 20:52:52.886888 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.931311 (+ 44423us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.970942 (+ 39631us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.979696 (+ 8754us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3750923}
I20251211 20:52:52.980764 2959 consensus_queue.cc:1048] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251211 20:52:52.981419 847 leader_election.cc:343] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: 25454e67437541d9ac2bfab29bad55f9
I20251211 20:52:52.981521 847 leader_election.cc:304] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:52.981992 3397 raft_consensus.cc:2749] T 25454e67437541d9ac2bfab29bad55f9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:52.986943 1166 tablet_bootstrap.cc:492] T a4ccf9a4ecbc4ca6978a189cf3a15893 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:52.988181 1166 ts_tablet_manager.cc:1403] T a4ccf9a4ecbc4ca6978a189cf3a15893 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.042s user 0.003s sys 0.000s
I20251211 20:52:52.990049 1166 raft_consensus.cc:359] T a4ccf9a4ecbc4ca6978a189cf3a15893 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.991554 1166 raft_consensus.cc:385] T a4ccf9a4ecbc4ca6978a189cf3a15893 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:52.991806 1166 raft_consensus.cc:740] T a4ccf9a4ecbc4ca6978a189cf3a15893 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:52.993181 3294 ts_tablet_manager.cc:1929] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:52.993688 3294 log.cc:1199] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/bd2bf1113dfe4b67b263b56f3d3db07d
I20251211 20:52:52.995138 3294 ts_tablet_manager.cc:1950] T bd2bf1113dfe4b67b263b56f3d3db07d P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:52.997229 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 940dfd8781de444a9caa78dd363c7451 (table default.Table_17e2482cdad5493a888dd8bd1ba85a7f [id=926a3069e1e946beb5cb0d5ddcfc3660]) successfully deleted
I20251211 20:52:52.992189 1166 consensus_queue.cc:260] T a4ccf9a4ecbc4ca6978a189cf3a15893 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:52.999213 1166 ts_tablet_manager.cc:1434] T a4ccf9a4ecbc4ca6978a189cf3a15893 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20251211 20:52:53.000015 3544 raft_consensus.cc:493] T 83067b139f504bc191a515feed0ec771 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:53.000130 3544 raft_consensus.cc:515] T 83067b139f504bc191a515feed0ec771 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.000617 3544 leader_election.cc:290] T 83067b139f504bc191a515feed0ec771 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691)
I20251211 20:52:52.999891 1166 tablet_bootstrap.cc:492] T e8db5a11a69e4554bc689c38ab57b28a P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:53.002375 1166 tablet_bootstrap.cc:654] T e8db5a11a69e4554bc689c38ab57b28a P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:53.004102 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "83067b139f504bc191a515feed0ec771" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:53.006358 2962 tablet_replica.cc:333] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:52.996527 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 205) took 2081 ms. Trace:
I20251211 20:52:53.006583 2962 raft_consensus.cc:2243] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:53.005204 1168 tablet_bootstrap.cc:492] T 6663f13691a24fb991b48f543d9de76b P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:53.006562 3294 rpcz_store.cc:276] 1211 20:52:50.914585 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.914640 (+ 55us) service_pool.cc:225] Handling call
1211 20:52:52.929518 (+2014878us) ts_tablet_manager.cc:1151] Deleting tablet bd2bf1113dfe4b67b263b56f3d3db07d
1211 20:52:52.929579 (+ 61us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.972529 (+ 42950us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.992662 (+ 20133us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:52.996407 (+ 3745us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6176,"tablet-delete.queue_time_us":2011298}
I20251211 20:52:53.008978 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet bd2bf1113dfe4b67b263b56f3d3db07d (table default.Table_ccd52fe248db4491820429e77b22f465 [id=a3f079fc1d054841b369c45cdae7bdfb]) successfully deleted
I20251211 20:52:53.009378 1168 tablet_bootstrap.cc:654] T 6663f13691a24fb991b48f543d9de76b P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:53.010211 902 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "83067b139f504bc191a515feed0ec771" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:53.010819 711 consensus_peers.cc:597] T 4b07f110f25645799ed0c89737cd8279 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:53.012722 2962 raft_consensus.cc:2272] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251211 20:52:53.009330 982 leader_election.cc:343] T 83067b139f504bc191a515feed0ec771 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:53.013556 983 leader_election.cc:343] T 83067b139f504bc191a515feed0ec771 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:53.013948 983 leader_election.cc:304] T 83067b139f504bc191a515feed0ec771 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:53.014628 3544 raft_consensus.cc:2749] T 83067b139f504bc191a515feed0ec771 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:53.016085 1160 tablet_bootstrap.cc:492] T 969a8587019c4202b6e705d126f83d44 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:53.017620 1160 ts_tablet_manager.cc:1403] T 969a8587019c4202b6e705d126f83d44 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.068s user 0.004s sys 0.000s
I20251211 20:52:53.018371 1160 raft_consensus.cc:359] T 969a8587019c4202b6e705d126f83d44 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.021142 1160 raft_consensus.cc:385] T 969a8587019c4202b6e705d126f83d44 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:53.021265 1160 raft_consensus.cc:740] T 969a8587019c4202b6e705d126f83d44 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:53.021364 2962 ts_tablet_manager.cc:1916] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:53.021493 1160 consensus_queue.cc:260] T 969a8587019c4202b6e705d126f83d44 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:53.021948 711 consensus_peers.cc:597] T c112714331bf4bd7a1ddcca17f338d4f P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:53.024200 1160 ts_tablet_manager.cc:1434] T 969a8587019c4202b6e705d126f83d44 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20251211 20:52:53.024724 3544 raft_consensus.cc:493] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6f8d784fd1cd492bbe3809403ec34919)
I20251211 20:52:53.024847 3544 raft_consensus.cc:515] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.025056 3295 ts_tablet_manager.cc:1929] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251211 20:52:53.025149 3295 log.cc:1199] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/87863daf77304a399318cdf56da9f9bb
I20251211 20:52:53.025411 3544 leader_election.cc:290] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:53.025451 3295 ts_tablet_manager.cc:1950] T 87863daf77304a399318cdf56da9f9bb P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
W20251211 20:52:53.026335 983 leader_election.cc:343] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Not found: Tablet not found: f22e4d34cde24b37a025c1b5a1046b1b
I20251211 20:52:53.027863 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 214) took 2112 ms. Trace:
I20251211 20:52:53.027982 3294 rpcz_store.cc:276] 1211 20:52:50.915695 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.918044 (+ 2349us) service_pool.cc:225] Handling call
1211 20:52:53.009257 (+2091213us) ts_tablet_manager.cc:1151] Deleting tablet fbc2d4aea084492c8270ad1d87096e58
1211 20:52:53.027701 (+ 18444us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:53.027835 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2090936}
I20251211 20:52:53.028321 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 208) took 2113 ms. Trace:
I20251211 20:52:53.028396 3294 rpcz_store.cc:276] 1211 20:52:50.914845 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.914950 (+ 105us) service_pool.cc:225] Handling call
1211 20:52:53.028223 (+2113273us) ts_tablet_manager.cc:1151] Deleting tablet 4585e05d06ef42cfb06c4ceb7524fd71
1211 20:52:53.028232 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:53.028307 (+ 75us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2110003}
W20251211 20:52:53.028553 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet fbc2d4aea084492c8270ad1d87096e58 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fbc2d4aea084492c8270ad1d87096e58 already in progress: creating tablet
I20251211 20:52:53.028681 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 209) took 2113 ms. Trace:
I20251211 20:52:53.028755 3294 rpcz_store.cc:276] 1211 20:52:50.914927 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.915036 (+ 109us) service_pool.cc:225] Handling call
1211 20:52:53.028593 (+2113557us) ts_tablet_manager.cc:1151] Deleting tablet 06ba98a726424dd89450053c5c5793f4
1211 20:52:53.028601 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:53.028667 (+ 66us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2110330}
I20251211 20:52:53.028978 3294 tablet_replica.cc:333] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
W20251211 20:52:53.029004 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 4585e05d06ef42cfb06c4ceb7524fd71 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4585e05d06ef42cfb06c4ceb7524fd71 already in progress: creating tablet
I20251211 20:52:53.029153 3294 raft_consensus.cc:2243] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
W20251211 20:52:53.029364 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 06ba98a726424dd89450053c5c5793f4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06ba98a726424dd89450053c5c5793f4 already in progress: creating tablet
I20251211 20:52:53.029459 3294 raft_consensus.cc:2272] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:53.029982 3294 ts_tablet_manager.cc:1916] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:53.031900 1160 tablet_bootstrap.cc:492] T 5df87ea7d28340da84ef141c92efc710 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:53.033901 1160 tablet_bootstrap.cc:654] T 5df87ea7d28340da84ef141c92efc710 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:53.049098 711 consensus_peers.cc:597] T 062f980b65504d3baf99ae400e521775 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:53.049765 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 161) took 2043 ms. Trace:
I20251211 20:52:53.049918 3295 rpcz_store.cc:276] 1211 20:52:51.006351 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:51.006396 (+ 45us) service_pool.cc:225] Handling call
1211 20:52:52.912974 (+1906578us) ts_tablet_manager.cc:1151] Deleting tablet 87863daf77304a399318cdf56da9f9bb
1211 20:52:52.922244 (+ 9270us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:52.953704 (+ 31460us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:53.024952 (+ 71248us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:53.049743 (+ 24791us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1906208}
I20251211 20:52:53.050457 1783 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 87863daf77304a399318cdf56da9f9bb (table default.Table_28b13655ab2b47e68f2bd01d618a2b58 [id=9b92dd859a6641b8ade2b49515d1e810]) successfully deleted
I20251211 20:52:53.050858 3295 tablet_replica.cc:333] T 3562b5effa124378beac7ccf0d27ee7e P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:53.050696 3294 ts_tablet_manager.cc:1929] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:53.051187 3295 raft_consensus.cc:2243] T 3562b5effa124378beac7ccf0d27ee7e P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:53.051388 3295 raft_consensus.cc:2272] T 3562b5effa124378beac7ccf0d27ee7e P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:53.052356 3295 ts_tablet_manager.cc:1916] T 3562b5effa124378beac7ccf0d27ee7e P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:53.052742 1166 tablet_bootstrap.cc:492] T e8db5a11a69e4554bc689c38ab57b28a P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:53.052923 1166 ts_tablet_manager.cc:1403] T e8db5a11a69e4554bc689c38ab57b28a P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.053s user 0.002s sys 0.001s
I20251211 20:52:53.053689 1166 raft_consensus.cc:359] T e8db5a11a69e4554bc689c38ab57b28a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.053884 1166 raft_consensus.cc:385] T e8db5a11a69e4554bc689c38ab57b28a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:53.053974 1166 raft_consensus.cc:740] T e8db5a11a69e4554bc689c38ab57b28a P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:53.054220 1166 consensus_queue.cc:260] T e8db5a11a69e4554bc689c38ab57b28a P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.054819 1166 ts_tablet_manager.cc:1434] T e8db5a11a69e4554bc689c38ab57b28a P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:53.055145 1166 tablet_bootstrap.cc:492] T 48820386e41b47129a13bdd5768f021b P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:53.057395 1166 tablet_bootstrap.cc:654] T 48820386e41b47129a13bdd5768f021b P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:53.060245 3294 log.cc:1199] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/fa79b67b9eee45e2b41c4a90967606cc
I20251211 20:52:53.061813 3294 ts_tablet_manager.cc:1950] T fa79b67b9eee45e2b41c4a90967606cc P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:53.062645 1168 tablet_bootstrap.cc:492] T 6663f13691a24fb991b48f543d9de76b P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:53.062825 1168 ts_tablet_manager.cc:1403] T 6663f13691a24fb991b48f543d9de76b P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.058s user 0.006s sys 0.003s
I20251211 20:52:53.063544 1168 raft_consensus.cc:359] T 6663f13691a24fb991b48f543d9de76b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.063751 1168 raft_consensus.cc:385] T 6663f13691a24fb991b48f543d9de76b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:53.063861 1168 raft_consensus.cc:740] T 6663f13691a24fb991b48f543d9de76b P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:53.064134 1168 consensus_queue.cc:260] T 6663f13691a24fb991b48f543d9de76b P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.064618 1168 ts_tablet_manager.cc:1434] T 6663f13691a24fb991b48f543d9de76b P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:53.064894 1168 tablet_bootstrap.cc:492] T b115d737d7fc483d9f5a1f040b511bae P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:53.064821 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 211) took 2149 ms. Trace:
I20251211 20:52:53.065094 3294 rpcz_store.cc:276] 1211 20:52:50.915090 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.915121 (+ 31us) service_pool.cc:225] Handling call
1211 20:52:53.028939 (+2113818us) ts_tablet_manager.cc:1151] Deleting tablet fa79b67b9eee45e2b41c4a90967606cc
1211 20:52:53.028946 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:53.034129 (+ 5183us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:53.050036 (+ 15907us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:53.064709 (+ 14673us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2110631}
I20251211 20:52:53.066141 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet fa79b67b9eee45e2b41c4a90967606cc (table default.Table_8f7e347b9ac64793b1073dfa955ebe21 [id=a6ab64c3e0d94462b716f2d153bc3314]) successfully deleted
I20251211 20:52:53.066529 1168 tablet_bootstrap.cc:654] T b115d737d7fc483d9f5a1f040b511bae P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:53.067754 1160 tablet_bootstrap.cc:492] T 5df87ea7d28340da84ef141c92efc710 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:53.067932 1160 ts_tablet_manager.cc:1403] T 5df87ea7d28340da84ef141c92efc710 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.036s user 0.003s sys 0.003s
I20251211 20:52:53.068620 1160 raft_consensus.cc:359] T 5df87ea7d28340da84ef141c92efc710 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.068822 1160 raft_consensus.cc:385] T 5df87ea7d28340da84ef141c92efc710 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:53.068912 1160 raft_consensus.cc:740] T 5df87ea7d28340da84ef141c92efc710 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:53.069171 1160 consensus_queue.cc:260] T 5df87ea7d28340da84ef141c92efc710 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
W20251211 20:52:53.050583 982 leader_election.cc:343] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: f22e4d34cde24b37a025c1b5a1046b1b
I20251211 20:52:53.069712 1160 ts_tablet_manager.cc:1434] T 5df87ea7d28340da84ef141c92efc710 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:53.069891 982 leader_election.cc:304] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:53.070370 3544 raft_consensus.cc:2749] T f22e4d34cde24b37a025c1b5a1046b1b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:52:53.072347 2962 ts_tablet_manager.cc:1929] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:53.072500 2962 log.cc:1199] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/d69fb1f6d9594afcbfb844860b41a377
I20251211 20:52:53.072865 2962 ts_tablet_manager.cc:1950] T d69fb1f6d9594afcbfb844860b41a377 P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:53.073674 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 164) took 3937 ms. Trace:
I20251211 20:52:53.073798 2962 rpcz_store.cc:276] 1211 20:52:49.136119 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.136157 (+ 38us) service_pool.cc:225] Handling call
1211 20:52:53.006307 (+3870150us) ts_tablet_manager.cc:1151] Deleting tablet d69fb1f6d9594afcbfb844860b41a377
1211 20:52:53.006320 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:53.045009 (+ 38689us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:53.072255 (+ 27246us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:53.073654 (+ 1399us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3847183}
I20251211 20:52:53.074266 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 166) took 3902 ms. Trace:
I20251211 20:52:53.074369 2962 rpcz_store.cc:276] 1211 20:52:49.171353 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.171437 (+ 84us) service_pool.cc:225] Handling call
1211 20:52:53.074108 (+3902671us) ts_tablet_manager.cc:1151] Deleting tablet ca89f09cc0954389b679e8bb8b5e4dfb
1211 20:52:53.074117 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:53.074249 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3902414}
I20251211 20:52:53.074640 2962 tablet_replica.cc:333] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:53.074846 2962 raft_consensus.cc:2243] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:53.075032 2962 raft_consensus.cc:2272] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:53.075604 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet d69fb1f6d9594afcbfb844860b41a377 (table default.Table_2edfd010f1ef40e8be655908fdc4800e [id=b24d18520fdd40e8bb721899ce7ac734]) successfully deleted
I20251211 20:52:53.066819 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 210) took 2151 ms. Trace:
I20251211 20:52:53.075553 2962 ts_tablet_manager.cc:1916] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:53.076145 1780 catalog_manager.cc:4965] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): delete failed for tablet ca89f09cc0954389b679e8bb8b5e4dfb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ca89f09cc0954389b679e8bb8b5e4dfb already in progress: creating tablet
I20251211 20:52:53.075907 3294 rpcz_store.cc:276] 1211 20:52:50.915000 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.915031 (+ 31us) service_pool.cc:225] Handling call
1211 20:52:53.066303 (+2151272us) ts_tablet_manager.cc:1151] Deleting tablet 83067b139f504bc191a515feed0ec771
1211 20:52:53.066412 (+ 109us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:53.066705 (+ 293us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2147892}
W20251211 20:52:53.076706 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 83067b139f504bc191a515feed0ec771 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 83067b139f504bc191a515feed0ec771 already in progress: creating tablet
I20251211 20:52:53.076773 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 212) took 2161 ms. Trace:
I20251211 20:52:53.076916 3294 rpcz_store.cc:276] 1211 20:52:50.915164 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.915207 (+ 43us) service_pool.cc:225] Handling call
1211 20:52:53.076553 (+2161346us) ts_tablet_manager.cc:1151] Deleting tablet e8db5a11a69e4554bc689c38ab57b28a
1211 20:52:53.076566 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:53.076753 (+ 187us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2158083}
W20251211 20:52:53.077454 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet e8db5a11a69e4554bc689c38ab57b28a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e8db5a11a69e4554bc689c38ab57b28a already in progress: creating tablet
I20251211 20:52:53.077523 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 207) took 2162 ms. Trace:
I20251211 20:52:53.077656 3294 rpcz_store.cc:276] 1211 20:52:50.914763 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.914821 (+ 58us) service_pool.cc:225] Handling call
1211 20:52:53.077320 (+2162499us) ts_tablet_manager.cc:1151] Deleting tablet e709051411354522bdb327c34dbf533a
1211 20:52:53.077361 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:53.077507 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2158871}
I20251211 20:52:53.078126 3294 tablet_replica.cc:333] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
W20251211 20:52:53.078174 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet e709051411354522bdb327c34dbf533a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e709051411354522bdb327c34dbf533a already in progress: creating tablet
I20251211 20:52:53.078470 3294 raft_consensus.cc:2243] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:53.078740 3294 raft_consensus.cc:2272] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:53.080878 3294 ts_tablet_manager.cc:1916] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:53.083905 1160 tablet_bootstrap.cc:492] T f2ae084982f44fdf8bfecdd2bc4c01b9 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:53.085345 1160 tablet_bootstrap.cc:654] T f2ae084982f44fdf8bfecdd2bc4c01b9 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:53.085932 711 consensus_peers.cc:597] T e59f8afe1834485d9fc64615e7bfe174 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:53.092809 3438 raft_consensus.cc:493] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:53.092968 3438 raft_consensus.cc:515] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.093519 3438 leader_election.cc:290] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:53.094440 847 leader_election.cc:343] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Not found: Tablet not found: fb030a62796248869d46b682656eaeb0
W20251211 20:52:53.094818 847 leader_election.cc:343] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: fb030a62796248869d46b682656eaeb0
I20251211 20:52:53.094904 847 leader_election.cc:304] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:53.095329 3438 raft_consensus.cc:2749] T fb030a62796248869d46b682656eaeb0 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:53.098776 1166 tablet_bootstrap.cc:492] T 48820386e41b47129a13bdd5768f021b P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:53.098961 1166 ts_tablet_manager.cc:1403] T 48820386e41b47129a13bdd5768f021b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.044s user 0.004s sys 0.000s
I20251211 20:52:53.099797 1166 raft_consensus.cc:359] T 48820386e41b47129a13bdd5768f021b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.099996 1166 raft_consensus.cc:385] T 48820386e41b47129a13bdd5768f021b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:53.100075 1166 raft_consensus.cc:740] T 48820386e41b47129a13bdd5768f021b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:53.100314 1166 consensus_queue.cc:260] T 48820386e41b47129a13bdd5768f021b P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.100826 3353 raft_consensus.cc:493] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:53.100951 3353 raft_consensus.cc:515] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.101523 3353 leader_election.cc:290] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:53.101881 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a56f1a7bd5464d289bfd018c826ee574" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:53.102044 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a56f1a7bd5464d289bfd018c826ee574" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:53.102190 1033 raft_consensus.cc:2468] T a56f1a7bd5464d289bfd018c826ee574 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
W20251211 20:52:53.102348 711 leader_election.cc:343] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:53.102521 710 leader_election.cc:304] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:53.102928 3353 raft_consensus.cc:2804] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:53.102996 3353 raft_consensus.cc:493] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:53.103048 3353 raft_consensus.cc:3060] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:53.105505 3353 raft_consensus.cc:515] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.106431 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a56f1a7bd5464d289bfd018c826ee574" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:53.106472 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a56f1a7bd5464d289bfd018c826ee574" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:53.106607 1033 raft_consensus.cc:3060] T a56f1a7bd5464d289bfd018c826ee574 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:52:53.106905 711 leader_election.cc:343] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:53.107060 3294 ts_tablet_manager.cc:1929] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:53.107450 3294 log.cc:1199] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/260bdeb73439437cbf234d20f7264ab3
I20251211 20:52:53.107684 3353 leader_election.cc:290] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:53.108887 3294 ts_tablet_manager.cc:1950] T 260bdeb73439437cbf234d20f7264ab3 P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:53.109411 1033 raft_consensus.cc:2468] T a56f1a7bd5464d289bfd018c826ee574 P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:53.109854 710 leader_election.cc:304] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:53.110260 3353 raft_consensus.cc:2804] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:53.110481 3353 raft_consensus.cc:697] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:53.110764 3353 consensus_queue.cc:237] T a56f1a7bd5464d289bfd018c826ee574 P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.117256 1166 ts_tablet_manager.cc:1434] T 48820386e41b47129a13bdd5768f021b P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.018s user 0.001s sys 0.000s
I20251211 20:52:53.118645 1168 tablet_bootstrap.cc:492] T b115d737d7fc483d9f5a1f040b511bae P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:53.118850 1168 ts_tablet_manager.cc:1403] T b115d737d7fc483d9f5a1f040b511bae P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.054s user 0.003s sys 0.000s
I20251211 20:52:53.119621 1168 raft_consensus.cc:359] T b115d737d7fc483d9f5a1f040b511bae P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.119859 1168 raft_consensus.cc:385] T b115d737d7fc483d9f5a1f040b511bae P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:53.119944 1168 raft_consensus.cc:740] T b115d737d7fc483d9f5a1f040b511bae P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:53.120209 1168 consensus_queue.cc:260] T b115d737d7fc483d9f5a1f040b511bae P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.120738 1168 ts_tablet_manager.cc:1434] T b115d737d7fc483d9f5a1f040b511bae P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:53.120790 1166 tablet_bootstrap.cc:492] T ba0e3a6b2699493ba9b9627b7d944b74 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:53.123313 3544 raft_consensus.cc:493] T ab4e7a9062134e489d6f9abe6ff26fb4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:53.123461 3544 raft_consensus.cc:515] T ab4e7a9062134e489d6f9abe6ff26fb4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.124092 3544 leader_election.cc:290] T ab4e7a9062134e489d6f9abe6ff26fb4 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:53.128015 3295 ts_tablet_manager.cc:1929] T 3562b5effa124378beac7ccf0d27ee7e P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:53.128351 3295 log.cc:1199] T 3562b5effa124378beac7ccf0d27ee7e P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/3562b5effa124378beac7ccf0d27ee7e
I20251211 20:52:53.129403 3295 ts_tablet_manager.cc:1950] T 3562b5effa124378beac7ccf0d27ee7e P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:53.130533 3544 raft_consensus.cc:493] T 88970b575ba943129729eceef5636519 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:53.130692 3544 raft_consensus.cc:515] T 88970b575ba943129729eceef5636519 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.131268 3544 leader_election.cc:290] T 88970b575ba943129729eceef5636519 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:53.133859 768 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ab4e7a9062134e489d6f9abe6ff26fb4" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:53.134311 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "88970b575ba943129729eceef5636519" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:53.134405 982 leader_election.cc:343] T ab4e7a9062134e489d6f9abe6ff26fb4 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:52:53.134814 982 leader_election.cc:343] T 88970b575ba943129729eceef5636519 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:53.135674 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 215) took 2181 ms. Trace:
I20251211 20:52:53.135772 3294 rpcz_store.cc:276] 1211 20:52:50.954685 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.954738 (+ 53us) service_pool.cc:225] Handling call
1211 20:52:53.078046 (+2123308us) ts_tablet_manager.cc:1151] Deleting tablet 260bdeb73439437cbf234d20f7264ab3
1211 20:52:53.078057 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:53.093588 (+ 15531us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:53.106665 (+ 13077us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:53.135652 (+ 28987us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2123098}
I20251211 20:52:53.136142 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 216) took 2181 ms. Trace:
I20251211 20:52:53.136241 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 260bdeb73439437cbf234d20f7264ab3 (table default.Table_10e55a82b5d1413b8b68da9df264da06 [id=7c485abbfb12494c89c81b0b51965923]) successfully deleted
I20251211 20:52:53.136212 3294 rpcz_store.cc:276] 1211 20:52:50.955063 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.955104 (+ 41us) service_pool.cc:225] Handling call
1211 20:52:53.136034 (+2180930us) ts_tablet_manager.cc:1151] Deleting tablet a4ccf9a4ecbc4ca6978a189cf3a15893
1211 20:52:53.136044 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:53.136127 (+ 83us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2180792}
I20251211 20:52:53.136447 3294 tablet_replica.cc:333] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:53.136632 3294 raft_consensus.cc:2243] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:53.136765 1168 tablet_bootstrap.cc:492] T 13ceabb190494e718a52892faaa53986 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
W20251211 20:52:53.136760 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet a4ccf9a4ecbc4ca6978a189cf3a15893 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a4ccf9a4ecbc4ca6978a189cf3a15893 already in progress: creating tablet
I20251211 20:52:53.136936 3294 raft_consensus.cc:2272] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:53.137491 3294 ts_tablet_manager.cc:1916] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:53.137993 3565 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 11f5f94732f049d0be4c97cebfffcb05)
I20251211 20:52:53.138383 1168 tablet_bootstrap.cc:654] T 13ceabb190494e718a52892faaa53986 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:53.142217 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ab4e7a9062134e489d6f9abe6ff26fb4" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:53.143327 1166 tablet_bootstrap.cc:654] T ba0e3a6b2699493ba9b9627b7d944b74 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:53.144261 910 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "88970b575ba943129729eceef5636519" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:53.144241 3565 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1830b0c58cfe495989f3d911dda0409a" member_type: VOTER last_known_addr { host: "127.0.104.62" port: 41123 } } peers { permanent_uuid: "11f5f94732f049d0be4c97cebfffcb05" member_type: VOTER last_known_addr { host: "127.0.104.61" port: 40749 } } peers { permanent_uuid: "9230f94a6a1a49cead51aeea9f069087" member_type: VOTER last_known_addr { host: "127.0.104.60" port: 42087 } }
W20251211 20:52:53.143805 983 leader_election.cc:343] T ab4e7a9062134e489d6f9abe6ff26fb4 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:53.144474 983 leader_election.cc:304] T ab4e7a9062134e489d6f9abe6ff26fb4 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:53.144868 3544 raft_consensus.cc:2749] T ab4e7a9062134e489d6f9abe6ff26fb4 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:53.145174 983 leader_election.cc:343] T 88970b575ba943129729eceef5636519 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:53.145270 983 leader_election.cc:304] T 88970b575ba943129729eceef5636519 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:53.145356 1829 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1830b0c58cfe495989f3d911dda0409a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1304 } } ignore_live_leader: false dest_uuid: "11f5f94732f049d0be4c97cebfffcb05" is_pre_election: true
I20251211 20:52:53.145601 3544 raft_consensus.cc:2749] T 88970b575ba943129729eceef5636519 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:53.146193 3565 leader_election.cc:290] T 00000000000000000000000000000000 P 1830b0c58cfe495989f3d911dda0409a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11f5f94732f049d0be4c97cebfffcb05 (127.0.104.61:40749), 9230f94a6a1a49cead51aeea9f069087 (127.0.104.60:42087)
I20251211 20:52:53.147054 3294 ts_tablet_manager.cc:1929] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:53.147922 3294 log.cc:1199] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/3acfef6f082641a199b80eefcfc772ac
I20251211 20:52:53.151293 3295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50110 (request call id 162) took 2144 ms. Trace:
I20251211 20:52:53.151479 3295 rpcz_store.cc:276] 1211 20:52:51.007017 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:51.007064 (+ 47us) service_pool.cc:225] Handling call
1211 20:52:53.050806 (+2043742us) ts_tablet_manager.cc:1151] Deleting tablet 3562b5effa124378beac7ccf0d27ee7e
1211 20:52:53.050817 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:53.070692 (+ 19875us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:53.127858 (+ 57166us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:53.151253 (+ 23395us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2043603}
I20251211 20:52:53.151567 3294 ts_tablet_manager.cc:1950] T 3acfef6f082641a199b80eefcfc772ac P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:53.152297 1783 catalog_manager.cc:4985] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): tablet 3562b5effa124378beac7ccf0d27ee7e (table default.Table_7f5df390c11943918af61066a7c67c66 [id=157a0b5022e445e68ac805168ebc586f]) successfully deleted
I20251211 20:52:53.153250 3295 tablet_replica.cc:333] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41: stopping tablet replica
I20251211 20:52:53.154667 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 217) took 2199 ms. Trace:
I20251211 20:52:53.155853 3295 raft_consensus.cc:2243] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:53.158035 3295 raft_consensus.cc:2272] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:53.158560 1160 tablet_bootstrap.cc:492] T f2ae084982f44fdf8bfecdd2bc4c01b9 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:53.159149 1160 ts_tablet_manager.cc:1403] T f2ae084982f44fdf8bfecdd2bc4c01b9 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.075s user 0.006s sys 0.010s
I20251211 20:52:53.155503 3294 rpcz_store.cc:276] 1211 20:52:50.955545 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.955611 (+ 66us) service_pool.cc:225] Handling call
1211 20:52:53.136407 (+2180796us) ts_tablet_manager.cc:1151] Deleting tablet 3acfef6f082641a199b80eefcfc772ac
1211 20:52:53.136414 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:53.141416 (+ 5002us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:53.146795 (+ 5379us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:53.154532 (+ 7737us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2180666}
I20251211 20:52:53.163187 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet 3acfef6f082641a199b80eefcfc772ac (table default.Table_320e1edc81a842849ac04e448aa2e2d4 [id=1c8eceb8a080464192ef7ab198728cd5]) successfully deleted
I20251211 20:52:53.163767 1160 raft_consensus.cc:359] T f2ae084982f44fdf8bfecdd2bc4c01b9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.164052 3294 tablet_replica.cc:333] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:53.164543 1160 raft_consensus.cc:385] T f2ae084982f44fdf8bfecdd2bc4c01b9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:53.164965 1160 raft_consensus.cc:740] T f2ae084982f44fdf8bfecdd2bc4c01b9 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f1933d9a74fccaa4618047fc05c41, State: Initialized, Role: FOLLOWER
I20251211 20:52:53.167392 2962 ts_tablet_manager.cc:1929] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:53.167203 3294 raft_consensus.cc:2243] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Raft consensus shutting down.
I20251211 20:52:53.166368 1160 consensus_queue.cc:260] T f2ae084982f44fdf8bfecdd2bc4c01b9 P eb0f1933d9a74fccaa4618047fc05c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.168489 3295 ts_tablet_manager.cc:1916] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251211 20:52:53.169874 710 consensus_peers.cc:597] T efe4d6bb07374294b8ab50a9fe605f87 P 6f8d784fd1cd492bbe3809403ec34919 -> Peer e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): Couldn't send request to peer e715a811a82446ce8d5bb4aafd5b81c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: efe4d6bb07374294b8ab50a9fe605f87. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:53.169745 3294 raft_consensus.cc:2272] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:53.170120 2962 log.cc:1199] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-2/wal/wals/01104975fd8b470aa026578b97ff6b8d
I20251211 20:52:53.170605 2962 ts_tablet_manager.cc:1950] T 01104975fd8b470aa026578b97ff6b8d P e715a811a82446ce8d5bb4aafd5b81c7: Deleting consensus metadata
I20251211 20:52:53.172827 3294 ts_tablet_manager.cc:1916] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:53.173122 1160 ts_tablet_manager.cc:1434] T f2ae084982f44fdf8bfecdd2bc4c01b9 P eb0f1933d9a74fccaa4618047fc05c41: Time spent starting tablet: real 0.013s user 0.003s sys 0.000s
I20251211 20:52:53.176832 1166 tablet_bootstrap.cc:492] T ba0e3a6b2699493ba9b9627b7d944b74 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:53.176975 1166 ts_tablet_manager.cc:1403] T ba0e3a6b2699493ba9b9627b7d944b74 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.056s user 0.001s sys 0.003s
I20251211 20:52:53.173489 2962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36220 (request call id 167) took 4001 ms. Trace:
I20251211 20:52:53.177841 2962 rpcz_store.cc:276] 1211 20:52:49.171875 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:49.171925 (+ 50us) service_pool.cc:225] Handling call
1211 20:52:53.074600 (+3902675us) ts_tablet_manager.cc:1151] Deleting tablet 01104975fd8b470aa026578b97ff6b8d
1211 20:52:53.074608 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:53.127731 (+ 53123us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:53.167248 (+ 39517us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:53.173470 (+ 6222us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3902520}
I20251211 20:52:53.177999 1166 raft_consensus.cc:359] T ba0e3a6b2699493ba9b9627b7d944b74 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.178330 1168 tablet_bootstrap.cc:492] T 13ceabb190494e718a52892faaa53986 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:53.178404 1166 raft_consensus.cc:385] T ba0e3a6b2699493ba9b9627b7d944b74 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:53.178467 1166 raft_consensus.cc:740] T ba0e3a6b2699493ba9b9627b7d944b74 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:53.178681 1168 ts_tablet_manager.cc:1403] T 13ceabb190494e718a52892faaa53986 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.042s user 0.005s sys 0.000s
I20251211 20:52:53.178756 1166 consensus_queue.cc:260] T ba0e3a6b2699493ba9b9627b7d944b74 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.179418 1166 ts_tablet_manager.cc:1434] T ba0e3a6b2699493ba9b9627b7d944b74 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251211 20:52:53.179749 1166 tablet_bootstrap.cc:492] T a5b2f4079315477bb5862629b7896f80 P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:53.179782 1780 catalog_manager.cc:4985] TS e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845): tablet 01104975fd8b470aa026578b97ff6b8d (table default.Table_5c5941c768984fc49f485d17cb1a485d [id=433155eccb834498a3b4e213051f8bbe]) successfully deleted
I20251211 20:52:53.181900 1166 tablet_bootstrap.cc:654] T a5b2f4079315477bb5862629b7896f80 P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:53.178267 2962 tablet_replica.cc:333] T d53b074865f4490693152a247b0fffdc P e715a811a82446ce8d5bb4aafd5b81c7: stopping tablet replica
I20251211 20:52:53.182974 2962 raft_consensus.cc:2243] T d53b074865f4490693152a247b0fffdc P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:53.183156 2962 raft_consensus.cc:2272] T d53b074865f4490693152a247b0fffdc P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:53.183981 2962 ts_tablet_manager.cc:1916] T d53b074865f4490693152a247b0fffdc P e715a811a82446ce8d5bb4aafd5b81c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:53.180347 1168 raft_consensus.cc:359] T 13ceabb190494e718a52892faaa53986 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.190692 1168 raft_consensus.cc:385] T 13ceabb190494e718a52892faaa53986 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:53.190922 1168 raft_consensus.cc:740] T 13ceabb190494e718a52892faaa53986 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:53.191094 3544 raft_consensus.cc:493] T ba0e3a6b2699493ba9b9627b7d944b74 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:53.191234 3544 raft_consensus.cc:515] T ba0e3a6b2699493ba9b9627b7d944b74 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.191381 1168 consensus_queue.cc:260] T 13ceabb190494e718a52892faaa53986 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.191861 3544 leader_election.cc:290] T ba0e3a6b2699493ba9b9627b7d944b74 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:53.192261 910 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ba0e3a6b2699493ba9b9627b7d944b74" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:53.192744 983 leader_election.cc:343] T ba0e3a6b2699493ba9b9627b7d944b74 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:53.194329 1168 ts_tablet_manager.cc:1434] T 13ceabb190494e718a52892faaa53986 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.015s user 0.003s sys 0.000s
I20251211 20:52:53.194772 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ba0e3a6b2699493ba9b9627b7d944b74" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
W20251211 20:52:53.195351 982 leader_election.cc:343] T ba0e3a6b2699493ba9b9627b7d944b74 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:53.195473 982 leader_election.cc:304] T ba0e3a6b2699493ba9b9627b7d944b74 P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:53.196774 1168 tablet_bootstrap.cc:492] T e95b0d8c507a4779a087c8933d464228 P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:53.200166 3544 raft_consensus.cc:2749] T ba0e3a6b2699493ba9b9627b7d944b74 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:53.200410 3544 raft_consensus.cc:493] T 48820386e41b47129a13bdd5768f021b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:53.200542 3544 raft_consensus.cc:515] T 48820386e41b47129a13bdd5768f021b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.201118 3544 leader_election.cc:290] T 48820386e41b47129a13bdd5768f021b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177)
I20251211 20:52:53.201452 910 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "48820386e41b47129a13bdd5768f021b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:53.201923 983 leader_election.cc:343] T 48820386e41b47129a13bdd5768f021b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:53.202394 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "48820386e41b47129a13bdd5768f021b" candidate_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:53.207129 1168 tablet_bootstrap.cc:654] T e95b0d8c507a4779a087c8933d464228 P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
W20251211 20:52:53.207818 982 leader_election.cc:343] T 48820386e41b47129a13bdd5768f021b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:53.207942 982 leader_election.cc:304] T 48820386e41b47129a13bdd5768f021b P e715a811a82446ce8d5bb4aafd5b81c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e715a811a82446ce8d5bb4aafd5b81c7; no voters: 6f8d784fd1cd492bbe3809403ec34919, eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:53.208673 3505 raft_consensus.cc:493] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:53.208712 3544 raft_consensus.cc:2749] T 48820386e41b47129a13bdd5768f021b P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:53.208827 3505 raft_consensus.cc:515] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.209396 3505 leader_election.cc:290] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:53.210237 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "12aaf618d6474f599458026308d7afdd" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:53.210772 711 leader_election.cc:343] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:53.211692 1040 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "12aaf618d6474f599458026308d7afdd" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:53.211916 1040 raft_consensus.cc:2468] T 12aaf618d6474f599458026308d7afdd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:53.212284 710 leader_election.cc:304] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:53.212728 3505 raft_consensus.cc:2804] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:53.212823 3505 raft_consensus.cc:493] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:53.212904 3505 raft_consensus.cc:3060] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:53.215435 3505 raft_consensus.cc:515] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.217226 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "12aaf618d6474f599458026308d7afdd" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:53.217566 3505 leader_election.cc:290] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
W20251211 20:52:53.217784 711 leader_election.cc:343] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:53.218286 961 heartbeater.cc:499] Master 127.0.104.61:40749 was elected leader, sending a full tablet report...
I20251211 20:52:53.220543 2959 raft_consensus.cc:493] T c549c2bfebd64b87a4fc8ce8210e14f6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:53.220726 2959 raft_consensus.cc:515] T c549c2bfebd64b87a4fc8ce8210e14f6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.221349 2959 leader_election.cc:290] T c549c2bfebd64b87a4fc8ce8210e14f6 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:53.221696 906 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c549c2bfebd64b87a4fc8ce8210e14f6" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
W20251211 20:52:53.222205 711 leader_election.cc:343] T c549c2bfebd64b87a4fc8ce8210e14f6 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:53.228226 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "12aaf618d6474f599458026308d7afdd" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:53.228432 1042 raft_consensus.cc:3060] T 12aaf618d6474f599458026308d7afdd P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:53.229504 1040 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c549c2bfebd64b87a4fc8ce8210e14f6" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:53.229713 1040 raft_consensus.cc:2468] T c549c2bfebd64b87a4fc8ce8210e14f6 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:53.230952 710 leader_election.cc:304] T c549c2bfebd64b87a4fc8ce8210e14f6 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:53.231304 3438 raft_consensus.cc:493] T 3f1840e4b5214ddb878935a79b214eae P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:53.231451 3438 raft_consensus.cc:515] T 3f1840e4b5214ddb878935a79b214eae P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.232100 3438 leader_election.cc:290] T 3f1840e4b5214ddb878935a79b214eae P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:53.233006 773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3f1840e4b5214ddb878935a79b214eae" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8d784fd1cd492bbe3809403ec34919" is_pre_election: true
I20251211 20:52:53.233243 3576 raft_consensus.cc:2804] T c549c2bfebd64b87a4fc8ce8210e14f6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:53.235034 1037 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3f1840e4b5214ddb878935a79b214eae" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:53.235874 847 leader_election.cc:304] T 3f1840e4b5214ddb878935a79b214eae P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:53.235934 1042 raft_consensus.cc:2468] T 12aaf618d6474f599458026308d7afdd P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:53.236297 3438 raft_consensus.cc:2749] T 3f1840e4b5214ddb878935a79b214eae P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:52:53.237283 1160 tablet_bootstrap.cc:492] T 7b06cd366de5494780b5ac81abbf0702 P eb0f1933d9a74fccaa4618047fc05c41: Bootstrap starting.
I20251211 20:52:53.236879 710 leader_election.cc:304] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters: eb0f1933d9a74fccaa4618047fc05c41
I20251211 20:52:53.238164 2959 raft_consensus.cc:2804] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:53.238320 2959 raft_consensus.cc:697] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
I20251211 20:52:53.238586 2959 consensus_queue.cc:237] T 12aaf618d6474f599458026308d7afdd P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.241611 3528 tablet.cc:1722] T d43210ec61af423ebde624397dea9f2e P 6f8d784fd1cd492bbe3809403ec34919: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:53.243023 3537 tablet.cc:1722] T d43210ec61af423ebde624397dea9f2e P e715a811a82446ce8d5bb4aafd5b81c7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251211 20:52:53.243098 3573 raft_consensus.cc:493] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:53.243461 3573 raft_consensus.cc:515] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.246902 1168 tablet_bootstrap.cc:492] T e95b0d8c507a4779a087c8933d464228 P 6f8d784fd1cd492bbe3809403ec34919: No bootstrap required, opened a new log
I20251211 20:52:53.251327 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "993616e226274c229a5924fbe69911af" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:53.251554 1033 raft_consensus.cc:2468] T 993616e226274c229a5924fbe69911af P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 0.
I20251211 20:52:53.252024 710 leader_election.cc:304] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:53.252281 3436 raft_consensus.cc:2804] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:52:53.252383 3436 raft_consensus.cc:493] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:52:53.252480 3436 raft_consensus.cc:3060] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:53.254563 3397 raft_consensus.cc:493] T f91487dd7e0f4c8b8483c2b83945da3b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:52:53.254704 3397 raft_consensus.cc:515] T f91487dd7e0f4c8b8483c2b83945da3b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.254990 3436 raft_consensus.cc:515] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.255126 1168 ts_tablet_manager.cc:1403] T e95b0d8c507a4779a087c8933d464228 P 6f8d784fd1cd492bbe3809403ec34919: Time spent bootstrapping tablet: real 0.059s user 0.006s sys 0.000s
I20251211 20:52:53.257441 3436 leader_election.cc:290] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:53.258814 1033 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "993616e226274c229a5924fbe69911af" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7"
I20251211 20:52:53.259006 1033 raft_consensus.cc:3060] T 993616e226274c229a5924fbe69911af P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:53.260118 1166 tablet_bootstrap.cc:492] T a5b2f4079315477bb5862629b7896f80 P e715a811a82446ce8d5bb4aafd5b81c7: No bootstrap required, opened a new log
I20251211 20:52:53.260249 1166 ts_tablet_manager.cc:1403] T a5b2f4079315477bb5862629b7896f80 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent bootstrapping tablet: real 0.081s user 0.004s sys 0.000s
I20251211 20:52:53.260974 1166 raft_consensus.cc:359] T a5b2f4079315477bb5862629b7896f80 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.261128 1166 raft_consensus.cc:385] T a5b2f4079315477bb5862629b7896f80 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:53.261175 1166 raft_consensus.cc:740] T a5b2f4079315477bb5862629b7896f80 P e715a811a82446ce8d5bb4aafd5b81c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715a811a82446ce8d5bb4aafd5b81c7, State: Initialized, Role: FOLLOWER
I20251211 20:52:53.261353 1166 consensus_queue.cc:260] T a5b2f4079315477bb5862629b7896f80 P e715a811a82446ce8d5bb4aafd5b81c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.261646 1033 raft_consensus.cc:2468] T 993616e226274c229a5924fbe69911af P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8d784fd1cd492bbe3809403ec34919 in term 1.
I20251211 20:52:53.261832 1166 ts_tablet_manager.cc:1434] T a5b2f4079315477bb5862629b7896f80 P e715a811a82446ce8d5bb4aafd5b81c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:52:53.262128 1166 tablet_bootstrap.cc:492] T 84ed434868aa437a83455b1605deb26c P e715a811a82446ce8d5bb4aafd5b81c7: Bootstrap starting.
I20251211 20:52:53.264056 1166 tablet_bootstrap.cc:654] T 84ed434868aa437a83455b1605deb26c P e715a811a82446ce8d5bb4aafd5b81c7: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:53.264827 3573 leader_election.cc:290] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:53.267372 3397 leader_election.cc:290] T f91487dd7e0f4c8b8483c2b83945da3b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:53.267854 710 leader_election.cc:304] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7; no voters:
I20251211 20:52:53.269160 3575 raft_consensus.cc:2804] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:52:53.277062 1042 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f91487dd7e0f4c8b8483c2b83945da3b" candidate_uuid: "eb0f1933d9a74fccaa4618047fc05c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" is_pre_election: true
I20251211 20:52:53.277302 1042 raft_consensus.cc:2393] T f91487dd7e0f4c8b8483c2b83945da3b P e715a811a82446ce8d5bb4aafd5b81c7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb0f1933d9a74fccaa4618047fc05c41 in current term 1: Already voted for candidate 6f8d784fd1cd492bbe3809403ec34919 in this term.
I20251211 20:52:53.270490 824 heartbeater.cc:499] Master 127.0.104.61:40749 was elected leader, sending a full tablet report...
I20251211 20:52:53.271147 3576 raft_consensus.cc:493] T c549c2bfebd64b87a4fc8ce8210e14f6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20251211 20:52:53.276412 711 consensus_peers.cc:597] T 116b21a182474ca191c4a89e9474677d P 6f8d784fd1cd492bbe3809403ec34919 -> Peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Couldn't send request to peer eb0f1933d9a74fccaa4618047fc05c41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:52:53.281543 3576 raft_consensus.cc:3060] T c549c2bfebd64b87a4fc8ce8210e14f6 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:53.281594 910 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "993616e226274c229a5924fbe69911af" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
I20251211 20:52:53.278190 1168 raft_consensus.cc:359] T e95b0d8c507a4779a087c8933d464228 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.283720 1168 raft_consensus.cc:385] T e95b0d8c507a4779a087c8933d464228 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:52:53.284051 1168 raft_consensus.cc:740] T e95b0d8c507a4779a087c8933d464228 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Initialized, Role: FOLLOWER
I20251211 20:52:53.284296 1168 consensus_queue.cc:260] T e95b0d8c507a4779a087c8933d464228 P 6f8d784fd1cd492bbe3809403ec34919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.295711 3575 raft_consensus.cc:697] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d784fd1cd492bbe3809403ec34919, State: Running, Role: LEADER
W20251211 20:52:53.296141 847 leader_election.cc:343] T f91487dd7e0f4c8b8483c2b83945da3b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): Not found: Tablet not found: f91487dd7e0f4c8b8483c2b83945da3b
I20251211 20:52:53.296270 847 leader_election.cc:304] T f91487dd7e0f4c8b8483c2b83945da3b P eb0f1933d9a74fccaa4618047fc05c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f1933d9a74fccaa4618047fc05c41; no voters: 6f8d784fd1cd492bbe3809403ec34919, e715a811a82446ce8d5bb4aafd5b81c7
I20251211 20:52:53.297011 3398 raft_consensus.cc:3060] T f91487dd7e0f4c8b8483c2b83945da3b P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:52:53.298622 3575 consensus_queue.cc:237] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.280682 3294 ts_tablet_manager.cc:1929] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:53.300060 3294 log.cc:1199] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-0/wal/wals/d9cf0d92ccc7419cb6e7694e8e5c7bfb
I20251211 20:52:53.299532 3576 raft_consensus.cc:515] T c549c2bfebd64b87a4fc8ce8210e14f6 P 6f8d784fd1cd492bbe3809403ec34919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3" port: 36845 } }
I20251211 20:52:53.300467 3294 ts_tablet_manager.cc:1950] T d9cf0d92ccc7419cb6e7694e8e5c7bfb P 6f8d784fd1cd492bbe3809403ec34919: Deleting consensus metadata
I20251211 20:52:53.301450 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 219) took 2345 ms. Trace:
I20251211 20:52:53.301561 3294 rpcz_store.cc:276] 1211 20:52:50.956281 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.956317 (+ 36us) service_pool.cc:225] Handling call
1211 20:52:53.163618 (+2207301us) ts_tablet_manager.cc:1151] Deleting tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb
1211 20:52:53.163635 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:53.207642 (+ 44007us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:53.280542 (+ 72900us) tablet_metadata.cc:641] Metadata flushed
1211 20:52:53.301400 (+ 20858us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2206903}
I20251211 20:52:53.302018 3294 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56136 (request call id 220) took 2345 ms. Trace:
I20251211 20:52:53.302124 3294 rpcz_store.cc:276] 1211 20:52:50.956829 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:52:50.956885 (+ 56us) service_pool.cc:225] Handling call
1211 20:52:53.301849 (+2344964us) ts_tablet_manager.cc:1151] Deleting tablet 10746162da4640e6a2a891467690b770
1211 20:52:53.301858 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1211 20:52:53.301983 (+ 125us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2344845}
I20251211 20:52:53.302403 3294 tablet_replica.cc:333] T 13ceabb190494e718a52892faaa53986 P 6f8d784fd1cd492bbe3809403ec34919: stopping tablet replica
I20251211 20:52:53.302629 3294 raft_consensus.cc:2243] T 13ceabb190494e718a52892faaa53986 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251211 20:52:53.302812 3294 raft_consensus.cc:2272] T 13ceabb190494e718a52892faaa53986 P 6f8d784fd1cd492bbe3809403ec34919 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251211 20:52:53.304087 1780 catalog_manager.cc:4985] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): tablet d9cf0d92ccc7419cb6e7694e8e5c7bfb (table default.Table_678702734654439b9b10658a0b6b3421 [id=b5b24783442f46f7a87c9758c1ed1397]) successfully deleted
W20251211 20:52:53.304458 1780 catalog_manager.cc:4965] TS 6f8d784fd1cd492bbe3809403ec34919 (127.0.104.1:42177): delete failed for tablet 10746162da4640e6a2a891467690b770 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 10746162da4640e6a2a891467690b770 already in progress: creating tablet
W20251211 20:52:53.297474 711 leader_election.cc:343] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:53.305727 3576 leader_election.cc:290] T c549c2bfebd64b87a4fc8ce8210e14f6 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Requested vote from peers eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691), e715a811a82446ce8d5bb4aafd5b81c7 (127.0.104.3:36845)
I20251211 20:52:53.307268 1168 ts_tablet_manager.cc:1434] T e95b0d8c507a4779a087c8933d464228 P 6f8d784fd1cd492bbe3809403ec34919: Time spent starting tablet: real 0.051s user 0.000s sys 0.002s
I20251211 20:52:53.307557 1168 tablet_bootstrap.cc:492] T 200b6db115964f3c8882b514c794698d P 6f8d784fd1cd492bbe3809403ec34919: Bootstrap starting.
I20251211 20:52:53.308388 1796 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 0302cc8613934c43a0650f081cf98797 (table default.Table_bf827b216f1b43c196923dc1828ff0d0 [id=ed371e073250439fa44330c7101e9387]). Expected version 1 got 0
I20251211 20:52:53.309187 1796 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 1f4eb1a174c648c897ae2b7612e44585 (table default.Table_e70873b71ba24d7eae548e2acfd243b3 [id=87adcc552a704d56bd2fe812fb3b791f]). Expected version 1 got 0
I20251211 20:52:53.309381 1168 tablet_bootstrap.cc:654] T 200b6db115964f3c8882b514c794698d P 6f8d784fd1cd492bbe3809403ec34919: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:53.309549 1796 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet a7afcce522494c9ca1d80e6eea913e6e (table default.Table_cb6a4bac7b694f33a37de2ee78c91ca0 [id=8940f18527c249a9baefc170ed82f45a]). Expected version 1 got 0
I20251211 20:52:53.309831 1796 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 84bb3583857848459e0e865283de623d (table default.Table_92281c5fcfee4355a79215ffedeee588 [id=6e5fe0a78e4b48f98dac028802f1ddf4]). Expected version 4 got 0
I20251211 20:52:53.310160 1796 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 7a299ee03d0e48aba7c4f71e3e0f381a (table default.Table_6402a801d48e4a53827b1b62b6cdaba1 [id=6876565d3a9c48ce841b830cf75139ff]). Expected version 3 got 0
I20251211 20:52:53.310621 901 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "993616e226274c229a5924fbe69911af" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41" is_pre_election: true
I20251211 20:52:53.312532 901 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c549c2bfebd64b87a4fc8ce8210e14f6" candidate_uuid: "6f8d784fd1cd492bbe3809403ec34919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f1933d9a74fccaa4618047fc05c41"
W20251211 20:52:53.312955 711 leader_election.cc:343] T 993616e226274c229a5924fbe69911af P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:53.313704 3398 raft_consensus.cc:2749] T f91487dd7e0f4c8b8483c2b83945da3b P eb0f1933d9a74fccaa4618047fc05c41 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:52:53.313774 711 leader_election.cc:343] T c549c2bfebd64b87a4fc8ce8210e14f6 P 6f8d784fd1cd492bbe3809403ec34919 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:52:53.315647 3295 ts_tablet_manager.cc:1929] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251211 20:52:53.316375 3295 log.cc:1199] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41: Deleting WAL directory at /tmp/dist-test-task5TWsQc/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1765486337660080-416-0/minicluster-data/ts-1/wal/wals/4f807596fa6a4e66b89566b5d5dd1080
I20251211 20:52:53.316982 3295 ts_tablet_manager.cc:1950] T 4f807596fa6a4e66b89566b5d5dd1080 P eb0f1933d9a74fccaa4618047fc05c41: Deleting consensus metadata
I20251211 20:52:53.317621 3294 ts_tablet_manager.cc:1916] T 13ceabb190494e718a52892faaa53986 P 6f8d784fd1cd492bbe3809403ec34919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251211 20:52:53.317540 1160 tablet_bootstrap.cc:654] T 7b06cd366de5494780b5ac81abbf0702 P eb0f1933d9a74fccaa4618047fc05c41: Neither blocks nor log segments found. Creating new log.
I20251211 20:52:53.327064 1160 tablet_bootstrap.cc:492] T 7b06cd366de5494780b5ac81abbf0702 P eb0f1933d9a74fccaa4618047fc05c41: No bootstrap required, opened a new log
I20251211 20:52:53.327277 1160 ts_tablet_manager.cc:1403] T 7b06cd366de5494780b5ac81abbf0702 P eb0f1933d9a74fccaa4618047fc05c41: Time spent bootstrapping tablet: real 0.092s user 0.008s sys 0.000s
I20251211 20:52:53.327620 1796 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 5b3e8ec3faf445ac9b746e65dc4dbfe0 (table default.Table_7fe80ff5483a4b0292091119980f6639 [id=3c3462a0c09748cea0d2ffb544ae2f13]). Expected version 1 got 0
I20251211 20:52:53.328094 1796 catalog_manager.cc:5746] TS eb0f1933d9a74fccaa4618047fc05c41 (127.0.104.2:34691) does not have the latest schema for tablet 7ffa06c4a7ee48b0b337a27aff090592 (table default.Table_647731023b044ebb8954f7fdcc3faf97 [id=4b91abd6e9e94a8a941b33e62e23c083]). Expected version 1 got 0
I20251211 20:52:53.328136 1160 raft_consensus.cc:359] T 7b06cd366de5494780b5ac81abbf0702 P eb0f1933d9a74fccaa4618047fc05c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0f1933d9a74fccaa4618047fc05c41" member_type: VOTER last_known_addr { host: "127.0.104.2" port: 34691 } } peers { permanent_uuid: "6f8d784fd1cd492bbe3809403ec34919" member_type: VOTER last_known_addr { host: "127.0.104.1" port: 42177 } } peers { permanent_uuid: "e715a811a82446ce8d5bb4aafd5b81c7" member_type: VOTER last_known_addr { host: "127.0.104.3"